Building a Software Factory with Gas City

So you’ve been pretty busy. You’re a maintainer of Gas City and went from saying that you’re not a developer to doing a lot of developer type work. One thing I guess I’ll start with is what have you learned from doing open source work on Gas City or other projects

that you would not have expected?

Hmm. Well, okay, this might be a kind of odd one, but I so I I do some level of open source work for Gas City, which is also built on Beads. and it’s all sort of agentically maintained, which is its whole

other side of open source. So my previous like understanding of open source, like as somebody who was coming from not being an engineer, not being a developer was, you know, it’s very gated and you know, you’re not really you shouldn’t you have to fully, fully, fully understand the system and before you can do anything, otherwise you’re you should just fix a typo or something like that.

But then I had read I think it was yeah, Steve Yegge’s blog post. I forgot what it was called, but it was about agentic maintaining. And he, you know, was he wrote Beads and he’s, you know, handles maintaining it. And he was talking about his approach, which was to basically take the good. So it was a it was around using.

community ideas and input to make a product better. And so I kind of started with that philosophy when I found Gas City.

And I I had really really been interested in software factories and trying to automate some of the development flow and just like experimenting and playing around with these things. But I had been like before reading that blog post, I was like, ooh, but I shouldn’t touch this. Like I I have no business getting in into this at all. but what I did was I was trying to use it and then as I was using the system, my

mayor, which I I don’t know, maybe I should talk a little bit about what Gas City kind of is, to ki to give some context for folks. Yeah, so

So

right, so Beads is kind of like Jira for agents, like tasks and issues and management and whatnot, maintaining a state between tasks. And then gas town, maybe I think more people might be familiar with that, was like an implementation of a software factory that was built on top of Beads, but in a very sort of opinionated way, with a mayor and deacon and polecats and all of these like crazy like words for like different agents with different roles.

And I guess yes, exactly. Exactly. Very whimsical, but also kind of hard to like

keep all the abstractions in your head. so that was all by Steve. and then

Chris Sells and Julian Knutsen. They are the CEO and CTO of Gas City, which is an open source project and now becoming an enterprise project, which is basically all the building blocks of Gas Town, basically a software development kit for building a software factory. And so you can create.

like your own variant of Gas Town. It doesn’t need to have polecats or deacon you know, I the my the way my city looks is different from what Gas Town looks like. so but I still have a mayor. Like there’s there’s still like some things like seem to be baked in to to Gas City that you know, maintain some of that. but the mayor everyone I think so.

I yeah. See

I think everybody needs this, like, I don’t know if it has to be the it doesn’t have to be called the mayor, but the it’s basically the way it’s run is by an orchestrator agent. So that was all just to sort of get us all up to speed on like the vocabulary terminology. So running, you know, I was building my city, not really knowing what I was doing, running it with my mayor. Mayor would tell me, hey, there’s this issue that we like that we’ve encountered, right?

And so you should file an upstream bug. And me, having worked with these agents for you know over a year now, I know not to trust them. So I’m like, really? Is this is this really a problem with Gas City or is this an us problem?

and this is relevant for just like how working with like the agentic open source sort of communities ends up being. so I use all of my you know different tools and setup and whatnot to really try to isolate whether, okay, this is in fact an issue that is upstream. so I I go to file like my first issue with Gas City.

And I felt kind of empowered to like do that because of the like ethos or whatever that Steve had mentioned with like Beads of like, okay, it’s it’s an open welcome community. You’re like even if your mayor is wrong or or you didn’t quite understand the system, this finding and input is still gonna be like valuable because of the approach that they’re using to maintaining it, which isn’t like a developer sort of like hunched over the keyboard, like, this another.

Person

who’s like submitting this obnoxious thing. It’s like they have their own sort of automated triage system and like their agents with more context around how they want the system to run to take that in. and then I I open my first PR to fix it. Cause it’s just like, okay, you have you submit an issue, but now

You don’t really need to wait for like a maintainer to write the code to fix your issue. You you have the agents and whatnot. And you, as the as the person who found the issue, probably have the most like useful context available to fix it. so I I did all I did all of that. and then and yeah, blah, blah, blah. And and then like Julian like merged my PR, and I was like, my God. Like I I actually contributed to an open source like project.

and I s I looked at his approach and like what happened, right? That that was able to like that I was able to go from issue to like producing that PR to then it being merged and I saw he was using multifamily model like review setup, like a lot of that.

was being recorded, like it was available to me to see what the findings were, like what was flagged as an issue and then what was sort of f like what was good and what was bad, basically. So I decided I would take it upon myself to be like, okay, I don’t want Julian to have to I don’t want Julian’s agents to have to do that. Like I want I want to figure out what I can actually extract from this.

To make it so like I basically like made it like a game for myself in a way of like, I’m gonna try

so it’s just like okay, the first PR I saw like basically then I had the data of like, okay, here.

you know, I felt good, like it got merged and everything, but I was like, we have we have some level of observability into this system. We can see what good means to Julian based on how his agents respond to our inputs, right?

So I sent my agents to look through all of like the Git history and like commits and like all of the reviews that were all like all like because all of the age the review agents were like commenting on these PRs and everything. So have all of that. And I built out these like this PR pipeline review skill that then I would use on my own PRs before they would send. And then I was like, you know, super happy whenever some of my, you know, it just just got accepted. and bait

and then I started.

So did you ha

was there a progression where you went from like like some number of comments down to no comments accepted right away?

Yeah, yeah. I can’t r I can’t yeah,

exactly. Like I I can’t remember what that sort of timeline looked like or the count, but I it was getting so like I was able to like really increase that throughput and then start like triaging. So what I would do is I built like a triage system as well for like the issues in Gas City. and then I would start writing PRs for those issues.

And then and so I basically I was doing enough of this that Julian was like, let’s make you a maintainer of of Gas City. So that’s just kind of like that’s how I ended up a a maintainer of this like open source project. and honestly, like when that was happening and kind of still now, like I had very like I would say like a minimal even like understanding of like how Gas City was working. Like I just got it to work and I got my agents to like

implement a system that had enough quality to like pass through the merge gates effectively. but like to to have the level of like understanding of the system to be able to do that, like they were they weren’t actually coupled. Like I was learning the system on the side with like different techniques.

but like to actually make effective PRs and issues like didn’t actually really require like a thorough understanding of the system, which I thought was very interesting and and might be kind of heretical to to say.

yes, I’ve I’ve gained more understanding of the system, though then I’ve like seen like, my god, why like why are things the way they are, right? It’s sort of like that sort of like reveal. and I had cause a part of the complexity is Gas City is, you know, a million lines of like Go or whatever. And then it’s it has a dependency on Beads.

which is its own sort of complex sort of code base. And then Beads depends on Dolt, which like I had struggled with, I still struggle with, like understanding.

The requirements behind that and like when is it a Dolt issue? When is it a Beads issue? When is it a Gas City issue? So I spent quite a bit of time trying to like tease some of those apart and then bake them that understanding like into my automated skills so that I wouldn’t necessarily have to hold all of that in my head, but I could basically transfer it to my agents when doing some of these reviews.

Mm.

Yeah, so it’s now

In in a lot of ways it’s more like sophisticated in a lot of other ways that’s made it more susceptible to breaking than what I had before. Cause before I had some I had like a skill called like GC triage, and then a separate one called like GC ship. So it’s just like triage something and then like

once I or like GC PR write or whatever. There’s like three different ones. And so it was very like clear and and it would be directed by me essentially. Like I would control, okay, gonna do this triage now, gonna have it write the PR, gonna have it go through the process. So like what that looked like for the

PR chip was I encoded a bunch of rules. And like this is still kind of what it looks like. I encoded a bunch of rules based on how effective code was like being merged and and retained within the code base. And so I had to meet those rules and then has to go through like several different layers of review. and then if it’s like

There’s also checks to see whether it touches certain parts of the code base that if those parts of the code base were to fail, would have a more significant sort of blast radius versus a very simple fix or like feature support ad or something like that. so like all of that is still encoded within my system, but the complexity and sophistication part of it is I was putting it into like a formula, like using Gas City itself, as opposed to before it was like

A skill that I would manually run with Claude Code Codex. and then so I was just like, okay, I I should be using Gas City to do this and automate it more. but it’s how I sort of started, and I I’ve written a a bit about like some of like the durability and distributed systems sort of side of things, but it’s like how I started uncovering like all of the sneaky ways that think like these systems can fail when you try to automate them.

And how, you know, they really want a human in the loop. It’s very frustrating. And so trying to like figure out all these different band-aids of

Like making no, like no, I I do not want to be in this loop. Like I I’m trying to automate this part away. Like, please. And so you like add in like another agent or something, like check on it. And then something happens to that agent. And so it’s just like how much of it can you abstract away into like a cron job? And so like I have these like different sort of scripts that check for things breaking that I only would encounter like know to codify after actually experiencing those failures. so but that so I have

in my city, I have project leads essentially that are like so the mayor is like the higher highest level like orchestrator. Then I have different rigs, which are basically repos with Beads databases. and I have yeah. Yeah, each

them know about each other. I just yeah, kind of don’t.

‘Cause they they’re not related enough. but I have like the like the other other agents know about each other to the extent that that’s helpful. So the mayor knows about or mayor obviously knows about all the the agents below, but

The w mayor interacts more with my Gas City maintenance project lead who chose the name Mika. because I I read another of Steve’s blogs where he, I don’t know if you had read it, but he talks about like charters and laurels and all of these things. And I don’t know how much of it I buy into or whatnot, but it was a it was interesting enough for me that I was like, hey, Gas City.

Proje maintenance project lead. Here’s

Is this the one where they came

up with different laws and self-governing structure, essentially?

Yeah. K well it

well th this one was more about like I think it was model governance or mo I don’t remember the exact word that he used. But it was around giving them like the model like f especially for these longer running agents, like basically a shared sort of memory state.

across sessions and th the the agent would choose like their name and pronouns and like you would tell like have a like a ledger of the things that the agent did well and all like I don’t know like to the extent that any of that helps or or doesn’t but I was like willing to try this experiment at least. so, you know, I I had

I I basically pointed my mayor at Steve’s blog and I said, Let’s set this up, right? and with our project lead agents. and so I have a Gas City maintenance project lead and they chose Mika, though the mayor and some they

Mika chose they pronouns, but the mayor always refers to them as she and then like they they all don’t really respect each other’s pronouns, which is its own amusing sort of like weird social like

robo thing.

Do the agents respond to that?

Or they just yes?

To to like, hey, like like correcting their pronouns. No. I don’t

actually think they care all that much. yeah, they they like go in and out of like how they refer to one another. But I had my mayor interact with Mika, who’s the the maintenance pr yeah,

project lead. And then there’s a city infra project lead, which is basically cause the mayor was fighting all of these fires that were happening in my software factory.

So I wanted to extract some of those responsibilities out into a separate agent because I whenever I go look at the mayor, I just see these diffs flowing of like another fire that’s being I’m like, I wanted to be able no. so pulled all those responsibilities out. And that agent chose the name Rivet. just yeah,

it’s a cool name. my mayor chose the name Cairn, which is I think a very popular claudism.

to choose. I’ve seen so many projects like named Cairn.

yeah.

That was a hilarious read. Yeah. I don’t they it is that’s like another sort of aspect of it too, is just like these like as you’re swapping in models, the way that you interact with them like changes. And so trying to keep on top of how any of that breaks your existing workflows or not, getting the most out of it is its own sort of unique challenge, I think.

Yeah, yeah, my background is yeah,

my background is very different from I think a lot of people in tech currently. yeah, so I’ve always loved

learning

really and information and all of those things. So I’ll go back a little bit further. when I was in high school, I would spend like every day after school like at the library, like just reading books like s through like starting with section 500, which is like math. and that’s kind of how I discovered astronomy as this more technical like

Grounded in reality and physics sort of discipline. Cause before I was like, it’s pretty. And you look at the stars and all like the basically popularized version of astronomy. and so

That’s a good one.

I d I don’t I don’t know anything about telescopes.

Yeah, it’s it I don’t know, it’s nice. It’s it’s it’s yeah, it’s it’s probably

it it’s for lazy people like me. So ‘cause, you know.

Well like it you it requires minimal setup and it’s like odd like you have like a little controller

and you can like program in like where yeah,

yeah, exactly. So like there’s probably better deal like there are certainly better deals for like if you are willing to do the more manual movements and whatnot of the telescope and get a larger aperture.

But I’m super late like I’m really lazy. So I yeah, end up getting that one.

Do have

it in a window normally or is it put away? Yeah.

The telescope,

it I’m embarrassingly, it’s like in a closet right now. ‘Cause I have two small kids.

‘cause I got a solar filter for it and I was able to record a video of that. that was a couple of years ago. So

Mm.

right, that yeah, it’s usually like

that one’s like usually like a three-inch telescope where they’re like then the like little kid is like using it to spy on neighbors or something like that. Yes, yes. I haven’t had that happen yet. I’m still waiting on it.

but yeah, so it I got into like astronomy. I was also really just into computers though. Like I

I started using computers as soon as I could. Like in third grade, I was like on AIM and then I like never got off on I was like basically online, like just perpetually online since third grade. For better or for worse. Like I just am not a touch grass type person. so just like yeah, super into that.

And the and so were you were you programming as well or making making websites? What all

what did that look like?

I I yeah,

I w I made a f I I would make like forums and stuff like as a kid in like elementary school. I got in trouble in like middle school because this boy didn’t like me and so I made like an I hate Jake forum and

A A AP programming. Yeah, I was like

Java. Yeah. and I I think that was like one of the first times I like stayed up all night, like, doing w work ‘cause I was just like really energized by it. Even though it was like some stupid like making a bug like do a random walk or or whatever. I was just I was just always really into into using computers and

I think that was also like something that connected me with astronomy was like realizing that actually now, these days, like most of that work is done on a computer. Like it’s all you know, through like analysis and data analysis and like programs and whatnot. So yeah, I think astronomers are actually pretty far ahead when it comes to to computational work.

Yes. Yeah, like the stacks

of like, yeah, books and all of those things. Yeah, so Pi Python was my language of choice. Or it was it was what when I started in undergrad, I think I was using like Mathematica and then like MATLAB and some other like random languages like that. But when in my PhD program, the first programming language that I like really

like used rigorously or what whatnot was in an astronomical data analysis course, which is where they taught Python. I write really bad Python. I’m I’m so gl I’m just so happy that agents exist because I’m I’m not like one of the craftsman developers who’s like, no, it really needs to look just so it’s like if I can get to like the medium medium level of like syntax, like that’s awesome for me. Like that’s that’s super good.

Do you think that is actually an advantage in the agentic era, having a focus on getting things built or accomplishing something versus

the craft of software?

Yeah. yeah,

I do. I think that this current era at least really benefits people with an open mind who aren’t necessarily like really set in their previous ways. like there’s like I I also hold the belief that like software engineering fundamentals are very important. So I won’t say that that all of that should be thrown out the window at all.

But I do think that like the mind the mindset that I think is most benefited by what’s currently available right now are folks that are like really full of like wonder and curiosity and like want to just start building stuff. so I had I had recently read this book, which is kind of cringy. I guess I’ll I’ll caveat I’ll caveat it with that.

cause it’s it’s called like the like six working geniuses or something. Like I like working gene like geniuses just sounds like such a cringy like personality test type thing. Genii, yeah. So it should have been geni. Obviously let’s get that straight off the bat.

but like reading through it, like it it made a lot of sense to to me. Like if you get past like the this seems like a weird sort of corporate non whatever.

in terms of like relating to the different mindsets of how people work and where they sort of like draw their energy from. So there’s the s the the six, right? what are the six? Yeah,

I’m gonna teach I’m gonna teach ya. so it’s it’s wonder is the is the first one ‘cause that’s when you just start at the super high level.

thinking about all the possibilities and everything. And then invention, which is like, okay, now that you’ve come up with some ideas, let’s test test them, right? It’s like the prototyping.

Make sense that that follows

from wonder. I

Yes,

it follows from wonder. and then there’s two, I can’t remember like the order necessarily of them, but they’re enablement and galvanizing. And so enablement makes sense, right? So like as you’re doing work, you you are but basically the glue and identifying ways to like lift everybody else up around you, and you get energy from from that, and it getting done what needs done, no matter like what it is. And galvanizing galvanizing is like, okay, we’ve settled on the idea, like basically.

I don’t know. Actually, hold on. There’s I I got ahead of myself. After invention is discernment. So you judging. So somebody’s thinking through stuff, making prototypes. Then somebody’s like, that’s a terrible idea. You should not do that. Or like this is a great approach, you know, that that sort of area of working. Then that’s like when you like sort of

crystallize what’s gonna be worked on. And then it’s the yeah, enablement, getting it worked on, galvanizing, making sure the people are actually doing it. So like the cat herder, right? Or like the technical program manager or something like that. Somebody who’s like making sure everybody’s on track, product manager, project manager. And then

And then

I

think the last one is tenacity, which is like I’m awful at it. and that’s like it’s like g it’s like gaining energy from like completing things. Like you really just like wanna be handed a discrete set of like things that are like clear and they have objectives and everything. and you you just want to get it done. You just wanna finish it. so in this sort of like era, I think you’re really benefited if you draw energy from like

Wonder and invention, because the agents can like do it for you. Like once once they are given like the thing, like a a well sort of formulated project to do, they will do it. And you know, there you can obviously argue around like, okay, well, is it gonna be like a steaming pile of crap or or whatever? But

like you as if if if you are like basically I had heard in like the Pragmatic Engineer podcast, I think Gergely had

called it like a tactical tornado. Like if you’re like a tactical tornado, like this is great time for you because you come up with all of these ideas and things and then you just like can have this have this robot sort of do it for you. If on the other hand you are somebody who really loves to like get into the weeds and the details and make sure every single thing is just so and whatnot, you’re probably gonna have a bad time.

as well and so like that’s one area. But to in in fairness to the people who complain about agents and everything, they i if if that person over here is not being careful and like making sure that what they’re guiding the agents to do is gonna be actual quality, the person down over here is left with a bunch of crap to to to deal with.

like I think I saw like the Shopify CEO say, like, everybody’s making more work with like these slop cannons and everything. And it’s true if if the people over here aren’t caring about it. and there’s also a difference too in the folks who are like the like this there’s a spectrum in how responsible like the person is.

for that quality and like what happens, like if it breaks. Right. so like the folks that are on call, right, for like emergencies and things, they’re gonna have a way different experience and like perspective on agentic outputs compared to somebody, maybe like me, who’s like, yay, I’m over here making like fun prototypes or like these different like

experimental projects and doing research and learning and all of this stuff. Like I’m not contributing to a bank or, you know, those other sorts of systems where you really probably should be looking at each line of code and like ha like making sure that the agent didn’t do something ridiculous.

Yeah, I wonder

what is the future gonna look like in industries where you needed this super reliable code are there were gonna be pressures to use agents and what does that

yeah.

cause you’ll s like cause Reddit, I think as a whole, is is kind of anti-AI in a lot of ways. You can find pockets of it where there’s like people in the hype circle, but like broadly, it just seems yeah, exactly. Yeah, r/ClaudeAI or

a agentic coding, vibe coders or whatever or whatever. Like generally it just seems that way. and that’s also kind of true of scientists.

as well. I think there’s like kind of a moral tinge, to some of this, like a political tinge. It’s it’s yeah, it’s interesting.

Sci scientists

like not wanting to use it to accelerate their work or in general?

I think there’s just there’s

a lot of like just there’s scientists tend to be very vocal about ethics and then there’s been a lot of discourse around the ethics of AI use. And so I think it can be an easier path of like kind of instead of

looking f deeper into it to sort of take this sort of moral ground and say, I I wouldn’t use that or like that that sort of thing without like actually sort of digging in into it and like s i it it I think it’s just in this current era, like across a lot of people, there’s not a lot of room for nuance in like grey areas. It’s like very polarized. At least in the States, like I’m not I can’t really necessarily like speak

or like other cultures. but just something I’ve sort of observed generally. and you know, so like the the water use of like data centers and like like all of these so like there’s lots of like just like that sort of thing. But then there’s also it’s not to say that all scientists are anti AI or anything. There’s like terminal bench science

And

there’s like an like AI group at the Center for Astrophysics where you know, I was working. There like there’s and there’s like a push for AI from NASA and and things like that, but it’s just like it’s it’s not a gung ho sort it just feels like there’s still this lot like a lot of education and like enablements and like argument that’s happening still within that area.

in a in a pretty similar way to like what we see with developers. It’s like a kind of you know there’s a lot of parallels I think in like that sort of technical mindset. a and how it’s affected peer review. there’s like lots of parallels there I think too.

I I don’t have an answer for that, right? Like, I mean it ultimately it requires somehow for like the contr like the problem is like the way the information is being conveyed, I think.

and and controlled and there’s like all this just like hype and like doom and gloom and like all of like so there’s like I could say like what I think people should how people ought to be thinking about this but like I am not the fire hose of information that’s being just like shoved into people’s faces from either side from whatever they’re clicking on. Right. So like you’d have to fix social media and like

Yeah, I it’s tough because I mean, I spent my career as a software engineer learning programming languages and

I mean, even I’ve spent a lot of time as an engineering manager as well, so it’s some times when I haven’t been deep in the code and just managing

Yeah.

But I think that’s okay. Like I I think that’s also something that I would okay. I I’ll I’ll c I’ll come back to like my auntie what I’m gonna say. But I I think that if like you’re questioning like, okay, what what’s the what is the value that I’m adding now? Or like what can I like just keep like I would just say keep trying to

do more and more, right? Like how like find those boundaries where like the agent actually can’t replicate like what what you’re trying to do and like f sort of fill in those gaps there. like I’ve just found like it can do so many like

sort of surprising things and it’s it doesn’t really necessarily matter that I am not the one like typing in all of the code that like puts that together. It’s just like the agent wasn’t gonna come up with that idea like I did. so it’s just like there’s still a lot of value in that. And then in discerning it, like making sure that it’s doing a thing that like I wanted.

So like for me, it’s easy for me to like not tie my value to that because it never was there. Right. Like it was I was never gonna be like writing the best Rust. Nobody is. There’s who knows Rust? One person knows Rust and then the rest of us are hopelessly

dependent on it. Right. yeah, so I I think there’s just like a lot that we can do to just like go higher up in the levels.

And there’s a lot of value that we could like add there. And that but that again kind of goes back to like the wonder and invention sort of side of things. Like if that’s what you’re good at and that’s what you like doing, then like yay. But if it’s not, so the other the thing that I was gonna so I wrote a couple tasks for Terminal-Bench-Science, which I was really excited about.

I was like, this seems like an excellent combination of like my skill sets of like I had this James Webb Space Telescope program that I had done and I had some Cassini data analysis and whatnot. And I was like, this this seems like I can really like get it into a task format because it was like using Harbor and I have lots of experience like with evals and benchmarks and and that.

Harbor

is the eval suite software, right? Yep.

Yeah, the yeah, the runner, like the

the framework. so I did that and it was and this and I was getting it all ready to go into like the v0.1 that but then you know there were ser certain issues and whatnot. it was I made the task hard enough that at the time like GPT 5.6 Sol and Fable 5 like couldn’t solve it.

they there was just like a one little like judgment pieces that I just like couldn’t quite get. And I was like, yes, like I did it. I I beat these agents. good, I could put in, you know, I I could contribute to how, you know, the reinforcement learning or however those other things are for like making agents that are better at science. Like, yay. but then Astra

And in 5.1 came around and like they just they they figured it out and I was I was just like really demotivated. Like I still am demotivated. I’m like, I like what it like what am I trying to do here with with with like where’s my role in science right now? if they’re just able to I mean I mean ultimately it it would require for me to step back and think a more b like a bigger picture a

complex

sort of like tasks and it’s not like I I couldn’t do that or anything. But it was it was just like sort of a demoralizing moment and an exciting moment. It’s like wow they can do this, but also like three or whatever years ago I spent like months trying to figure this out and this agent did it in 15 minutes. Like are you kidding me? so like I can kind of empathize

with I think what what some of the developers are feeling.

Yeah, the I mean I still think

That having the knowledge and having it be across different areas is still valuable because

Like you’re saying at some point, you’re the one giving the context or the instructions to the agents and they don’t know what they don’t know.

I

I used to have a skill that I would use. I don’t really have it in rotation anymore, called pre-mortem.

where I would have the agent like help me, like uncover some of those unknown unknowns, which I I think it is still a valuable thing to do outside of then whatever the agent does, like for you to then have that in your brain and think through how you.

give the task to the agent ‘cause what I had found after doing that is like the agents are such nerds they would like try to then overcompensate for like all of these potentialities. and just like were so risk averse and w and would just like basically build in all of like it would just like make getting anything done like impossible. yeah.

Why don’t

you use it as much anymore?

‘Cause

of that, yeah. Cause of like ‘cause it it was just like over sort of indexing on on the pre-mortem findings. And like J s basically taking that in as like the full sort of like basically giving it way too much weight. Like I could probably like if I wanted to go back in there and like make modifications to it, I what I would do is like run still run the pre-mortem and then

do a review on that of like what is like really critical to like be mindful of and sort of triage that because it was giving sort of all of it equal weight. regardless of like how actually likely important it would be.

Yeah. Yeah. I still think it’s valuable

though, because there’s just so much that can go wrong in these complex systems and like emergent behaviors, which you can’t always predict. And so what I try to do is also

like bake in that fact of like having like more sort of graceful degradation of like when things go wrong or like it more like alerts of like like here’s the expected behavior, if that doesn’t happen, like I need it needs to be escalated to me. Cause there were just so many instances where

something would break and then I wouldn’t know for like days or whatever. But now now it’s on the other now it’s gone the other way for me. Yes. I my Slack that I have set up connected to my agents, just like full of alerts. I’m like, God, like I don’t need all these alerts. Cause like if you have too many alerts, then you’re gonna ignore them. So it’s still like a yeah, a learning process, I think.

Yeah, yeah.

Well, so it’s it’s it’s more of a kind of like a dashboard in a way. it it’s called Agent City. and each channel is associated with a project or a rig. and then the project leads have handles there. So like you can’t make

user like the agents aren’t users. I basically have them I get around it by making user groups. And so I have like a whole like I basically created a Slackbot app thing or whatever

that’s like associated with the entire Gas City and then separated out each of the like agent identities into their own user groups so that then they could be tagged and that’s like mapped out. and then I just have like automated

I have like a decisions ledger thing that gets like posted there. And I have like a Slackbot task that like it parses through all of the alerts and things that come through the channels and like gives me like a morning report.

and some like recommendations of like replies and things like that. it’s still a work in progress though. I’m not enti like I I’m I’m not gonna say that I have like the observability and like communication layer solved at all.

Is this

part of Gas City or is this your own project?

Both. So like the way Gas City works is it’s it’s it’s like an SDK, right? So it’s a building blocks. But then one of the great aspects of it is you can create packs so that other people can like basically pull in your customizations. or you can share them and things like that. So there’s like a there’s a Gas City repo and then there’s like a Gas City dash packs repo.

and so some of these are like maintained by the maintainers or like I think Julian and Chris came up with like a web form now for other folks to like submit theirs to be sort of vetted to be in the more public-facing stuff. But I have a Slack integration pack. so if like somebody is running their own city, then they can pull in the Slack Gas City pack and then in theory.

be able to to run that, you know. I only have so much time to be testing this on other machines or or whatever. So it’s, you know, at your own risk, have your agent fix it and then open PRs if there are any issues.

So do you have this

all run locally on your machine or do you have a remote Mac Mini somewhere or server running this?

I I yeah, I

never got I never got into the Mac Mini craze. so I have a MacBook Air that is remotely connected to a Linux machine. that I use it the and we also have like network attached storage, which my husband got for me as a birthday present because we are giant nerds. So

Nice. Very nice. Yes. Was that w did you

request that or did he n guess that you wanted it? Wow. you know you’re meant for each other.

He thought he thought he he guessed that I wanted it. Yeah. So yeah. Yeah. We my god. Yeah. Yeah.

No, we he’s at Nvidia. is a developer that he was a technical program manager and he’s now like he div does like agentic development of like different libraries and things like that. I got into agents before him though.

It I’m not saying this to brag, I’m saying this because he didn’t understand my horrible addiction until like a few months later. And I don’t know, I’m like, you shouldn’t have judged me so much. Like you’re just as addicted to them now.

He

he he’s the reason I like started using like ChatGPT like back in what was it? Like big yeah, 2023, Yeah, it

Right, yeah. A few a few

things were going on at the same time.

Yeah.

but like he was the reason I like found out about this and I was like, my gosh, this is in like around that time, like when that it was basically like better Stack Overflow or whatever or like another

Yes.

Or I th or if you wanted to write a song in the style of Blink-182 about anything any topic on your mind.

Exactly. Exactly. Yeah. Yeah. We’re

we’re very into Suno now. So

Cool,

yeah, it’s you can get them get them excited about any topic you want with Suno.

Yeah, yeah, I do a lot of projects with the the kiddos. ‘cause my my oldest is six and he’s very like I didn’t have to really I don’t know. he’s very into Zelda and Magic the Gathering. I didn’t have to do much to get

this obsession to happen. It’s just very lucky for me. so he like co-designed with me a Zelda set.

for for Magic the Gathering. And we’ve been like I I made this whole like Magic the Gathering like play app and and things and we are like playing that together. I made like a Wheel of Fortune app that he’ll like ask to play. In yeah in the Yeah.

So in the Magic the Gathering app, it’s I

use there’s like a already made rules engine called Forge which I use in the back end but it’s like what is it GLP or something so there’s like a bridge so it doesn’t like yeah to play to play the rules and everything but

So well Flurry Rush in the Magic set is it’s like Flurry Rush two

whenever the creature is like blocked by a creature with power like greater than it, it will do two damage to like that creature.

Is that a

rule that exists in other sets or that you created for this okay?

I created that rule. Yeah. So like

I have so there’s a separate like little rules engine that I built to like be hybrid with Forge. to then work with any sort of like generated cards and mechanics and things. so it was like a f it’s like a fun sort of like data like experiment too. because it like many, many years ago,

‘Cause I’ve always just sort of been into Magic the Gathering. When I was in my PhD program, I was like, like these Jason or like these Magic cards, it’s just like JSON data. Like it’s just metadata. Like, what can I do to like do a data analysis project to like learn more like try to like from first principles. Like I I So like there was like a week or something that and I was like, I don’t know. I’m I I gave up because it was just tedious, like doing that sort of like

I

I’m I’m not good at the syn like the lower level syntax sort of stuff. I’m like more like bigger picture sort of thing. So then like when agents came around and I was like, I can actually I can actually start doing this sort of thing. So it’s like a whole lab.

Well and where even would you

have I guess you would have needed somewhere that had all the cards and all the data in the first okay.

Well, I had that. So I at least I built out that part.

So ‘cause it’s like Scryf Scryfall and some other like places, Seventeen Lands. There’s there’s also like gameplay data and stuff like you can pull in. You can get the data,

yeah. But then it was a matter of like needing to figure out what features would be like most potentially relevant or like then I wanted to visualize it and I wanted to like make drafts and bots and things happen, which which now does. Like I have that set up and it

looks great and it plays great. I can play all my nostalgic sets and everything. yeah, so it’s just like a super fun time.

my god. Yeah. It it just wouldn’t be fun. It would it it just like I mean

I I figured out pretty quickly it wasn’t gonna be fun. And so

I stopped doing it. But but this yeah, this this one is fun. And I I also started I started trying to make out my own UI and everything, but then I realized like actually Magic the Gathering Online already had the perfect UI and they for for like the representing all of like the

information and everything. So I was like, why would I try to reinvent that? Like or like make a worse one? And that’s kind of like my philosophy to a lot of different things. It’s like, if this already exists, like why why am I trying to reinvent it? Like why not just pull in all of like the learnings from this previous thing? Like my, you know, I I don’t need I’m not gonna have a better approach. I can build on it, maybe like from my own things. but it’s why I didn’t want to make a benchmark.

Like when I was tasked with that for Sourcegraph, I was like, surely there’s a benchmark that could already meet the the requirements here. ‘cause I I was I was still able to take and like build on on them, but I I would do like to like steal things to the extent that they exist for my needs.

I was always I guess.

Right, right. I know, because that’s a whole other thing of like the gatekeepiness that exists within engineering cultures, which has made me question my anyway. so

My undergraduate

degree is in comparative literature.

Yeah. Yeah.

Yeah. Y’all yeah, it it’s really helpful to have, I think, a a variety of different experiences. but yeah, so I I was like a pure planetary science researcher working at Southwest Research Institute, observing asteroids like with James Webb Space Telescope, doing Cassini data analysis, Europa Clipper, Artemis program, blah blah blah, NASA scientist. and then

As I had mentioned, like ChatGPT became a thing and like information, like all that stuff was like very intriguing to me.

And there was like a job opening for an academic search engine called the astrophysics data system, which is a search engine that’s used by every single astronomer. which I had been using it, and NASA was funding them to expand to not just cover astronomy, but to cover all the disciplines. So NASA doesn’t just do astronomy. Planetary science is like the space exploration side of things. So I would be the planetary science

Project scientists, but they’re also covering Earth science, which is like an order of magnitude, more like content and and information and people and whatnot. And then also biological, physical sciences, so like human space flight and like growing plants in space and things like that. and then heliophysics, so like studies of the sun. and the role was like very intriguing to me, because you’d get 30% of your time to do whatever planetary science you want.

so I don’t know how many people know this, but like the way scientists usually are funded is through like soft money, which means you have to write proposals and have and win them and

It’s a very competitive and it’s like very anxiety inducing. and you’re kind of at the whims a bit too of like whatever the review panel or like decadal survey or whatever want. So having 30% of your time to do whatever science you want is like extremely awesome. yeah, without having to write proposals, yes, no.

And the other percent would be spent on the project. So like the

helping develop the the search engine, the doing the community engagement. So like all of the things that I l like as I was definitely a big leader in like the planetary community, I I loved community engagement and all of those things. and then I was loving like the information like the like there was a lot of opportunity to

leverage like they have all of this like full text of like all these papers and metadata and things and now that we have this like emerging technology, just like really exciting to to figure out new ways to use that to like make planetary scientists work easier essentially and connect and also like have more interdisciplinary connections, like make all of that more discoverable.

So

when was that?

So

that was 2023? Yeah.

Yeah, yes, yeah. So I was yeah, I was like r I was running

like a Gordon research seminar on origins of solar systems thing in like the summer of 2023 at Mount Holyoke in Massachusetts and they had I had gotten to like the panel stage or whatever.

So still a thing. we had an exposure.

To COVID, like the day before my like in-person presentation was supposed to happen. so I

I had like driven out to like a hotel that I was staying at to like then go do my talk and whatever. but I had to stay, I had to do all the interviews and like the presentation from my like on Zoom from my hotel room, because of COVID still still

a thing. yeah, so I but I got I got that role as the project scientist for planetary science.

And yeah, it was one of like the best jobs I’ve ever had. It was really awesome. People were great. and like a lot of why I had ended up looking outward at industry jobs at tech is because NASA

was like just the government was just ridiculous. the funds were like always at risk of being cut in half. my role was basically on the chopping block. I was trying to like figure out alternative ways to like maybe get funding to like support the project and whatnot.

And just wasn’t it it wasn’t feeling very like safe. Not that tech is safe, I guess.

I

feel like, especially in the last well

but but for me, like ultimately it was so it was a combination of that, and then I just wasn’t enjoying planetary science anymore in the same way that I was enjoying development and building. I was just like vibe coding all the time at that time.

And just like really into the information retrieval side of things and search. Like I had joined the search relevance community. Like I saw you had Doug on one of your recent episodes. Yeah. Yeah. Yeah. Doug Doug was like

one of my first introductions to the search relevance like side of things ‘cause I read his book.

And I just became I became obsessed like with with that side of things, in a way that I just was no longer with planetary science.

Yeah. so yeah, I guess I can give a little bit of back story too, with like s when I was looking at other roles, I found Sourcegraph through that search relevance community because the person who leads the women in search commun group there, she was a developer at Sourcegraph. And Sourcegraph like

At the time, and now is also, but like this kind of changed for a little bit. the time when I was looking at it, I was like, this is a code search company, and I do search, and you know, this this is really cool. Let’s let’s look at that. and but then in my recruiter call, Trevor,

He’s like, actually, we’re a coding agent company now. Have you heard of Amp? And I was like, no, but I love coding agents. Let’s do this. so anyway, for the first three months of my role there, I was a solutions engineer for Amp. and then and then there was a company split and everything, and sort of went more back over. But like for the for that first sort of part, like there was like almost like I wasn’t

really like there was such a divide in the company. All right. So I was like not really doing any search like at all. it

so

like code search versus agentic development.

Yeah, like

so so like a lot I think a lot of people, including the prospects and and customers is thought had this like assumption like, Amp has this sort of like Sourcegraph stuff baked into it. It’s like no Amp isn’t entirely like it uses you know, grep and whatever. Like there was like a different sort of skill subagent, like the librarian and things, but it was like none of that was

really in any way related to how Sourcegraph like handled the search and like what all of that was. But I think what helped me for like on the information retrieval side of things is like having this understanding of of context and relevance and and all of those things for like getting the most out of setting up like the environment for the agents. though I I

I created my own sort of like research agents and things, which I think the information retrieval side really helped. Not necessarily so much as much for like the coding agents, because I would just sort of use whatever was out of the box for those, with maybe some skills and whatnot, with like an understanding of context engineering. but it did help with my

like looking into like supporting the Sourcegraph MCP and like those sorts of sides of things. But I was never really like an engineer there. And I really wanted to do more in the side of like the search and information retrieval, like research side of things, but just like sort of never panned out there.

So by the t by the time this episode is released, you’ll you’ll probably be you’ll be on to the next. You’re

you’re joining Omni, right.

Yeah,

so Omni is awesome and they had like a fantastic candidate experience. Like that’s a whole other episode I could talk ‘cause I did

Yeah. I feel like

if you’re not like Anthropic these days, if your interview process is too long, it’s probably you’re probably missing out on a lot of candidates.

Yeah. Yeah.

Man, the NVIDIA one so long. Anyway, but yeah, Omni. So their their interview process, like

basically had two different ways of hiring. One was like for open requisitions, which like these are roles like with very specific needs that they have. And then the other is just like sort of people shaped roles. Which is like what I would always want. Like cause I have a very weird background and like set of skills and things that are like I feel like are always in flux. So having

so like they basically had me write a paragraph of like what I wanted to be doing.

it sort of like worked with me to figure out what that would look like in terms of scope and like who I’d be reporting to, where I would sit within the organization and like all of those things.

Reach out to us

and tell us what you think you want and we’ll figure it out. Yeah.

Yeah, yeah. Well

like well so I don’t know so I don’t know how it if it’s like that level ‘cause like for me in my own experience VP of marketing had reached out to me because my role at Sourcegraph, like I sat within the marketing org for not really any reason other than headcount.

And like that’s like as just a separate thing. Like I

so I became I I kind of was put into a developer relations sort of scope because that’s the thing that was closest within marketing to what I do. So then I get, you know, marketing inbound. so and I I do a lot of like technical communication and and things like that. So it’s just like very broad. but in talking with them, like they kind of immediately realized like, we need to

like make a person shaped role like for you because of all of the like engineering type interests that you have. So I went through like several I talked to several different people, all of them were just like really awesome. The culture was great. and like I had like one assignment which was like more around the marketing side, but then they had realized like, no, actually since since your paragraph that you gave us like was way more technical. Let’s give you this technical assignment.

which they then also like paid me for. which is, you know, always nice when a company is willing to do that. yeah, yeah, so like we’re gonna like pay back

your like the resources that you use and your time and everything. and it was like a a benchmark project. So I appreciated it wasn’t LeetCode and it wasn’t

Also, like I know some companies are trying out different things of like let’s do agentic coding interviews, but like that also bakes in some of their biases around however they think agen decoding should work. And I I ran into some of those issues and stuff too.

Yeah, those

can be tough depending on it’s like what do you value in agentic coding, right?

Yeah, exactly. Exactly. Exactly. Cause like in one of my

like basically I I need to like ask and understand a lot of like what the task is up front. So it’s not necess I’m not necessarily like optimizing for opening a PR in like the first five minutes or whatever, like the the time box time. Like ‘cause I it just doesn’t seem right to me. but like who knows like yeah.

Yeah, yeah. Right,

right. and I think like a lot of people just still don’t like necessarily know how to evaluate like other other humans for different roles and like what they actually want. that’s also like another that was one of my worst candidate experiences where the job description was written as one thing, but they were clearly looking for somebody of a different type.

And so it was just sort of a very mismatched sort of there. but anyway, with with Omni worked out sort of great. They like were totally on board with the paragraph that I gave them and we’ll be reporting to the CTO, being in the engineering organization and get to be an AI engineer, which is exactly what I wanna be doing.

Awesome.

Yes. What other other than starting your new role, what I guess what are you most excited about in AI and agents these days?

Like I

co-founded a structured outputs company and it was like it’s a structured outputs, we’ve had this, I mean and with small

Right. Right. Is it like so I’m still trying to figure out

is just like is it because of how it’s packaged and how easy it is, like versus like, you know, where where’s the actual lift gonna potentially come from? though I am really excited to watch like that space for, you know, this is one flavor of their models, like the decision models, but they’re all sort of intended to be

machine consumable versus like requiring the human loop. So I’m I’m I’m kind of excited about that.

For a lot of like the software factory sort of and like automation side of things, because realizing like, right, obviously models that are trained with reinforcement learning with human feedback require a human in the loop. And so that’s why we are all forced to be in the loop still, even though we are like desperately trying to remove ourselves from the loop, be really interesting to see how those building blocks come together.

Exactly. Exactly.

Yeah. Cause I I had that similar thing with I had made this like fine-tuned model trying to go from like natural language to structured query syntax for the SciX search engine. And that, you know, there’s lots of labeling and like the all these like different things you have to do and like lots of friction there. It’s just like how much of that can be potentially just like swapped out with a single line.

Right.

Now everybody knows about it. Like it’s it’s really cool to see all of the like different use cases, even though some of them aren’t the right use cases. Like Diogo’s like, stop doing that.

Yeah. All right. Yeah.

Yeah. Yeah, it’s and it’s just it’s more another tool you have to build the system, which it’s

I think that’s

Yeah, and I it’s the f funniest thing I find is like all the different projects that are implemented in like twenty four hours because now everyone has agents to build things to.

Yeah, exactly.

Yeah, it’s exciting for sure.

Thank

you for having me.