The New Sport of Programming

I’m Dan Gerlanc and welcome to Agents and Engineers, the podcast about agentic AI and software development. Today, our guest is Matt Rocklin. He is a well known open source developer, the creator of Dask, among several other open source projects. Most recently, he

created Coiled Computing, which is a VC funded startup that he is still running, though in a slightly different form today, which maybe we’ll talk a little bit about as well. But all in all, great to have you here today, Matt.

Sure, yeah, that’s been a lot of fun. maybe I’ll take a step back briefly and just describe Coiled. So Coiled was really based around Dask, it’s an open source project that is parallel computing in Python. We were like a cloud SaaS company, sort of on the path of Databricks, but we like failed to get anywhere near there. maybe like six months ago, I decided to sort of stop the sort of ambitious side of the company.

We sort of bought back shares from investors and we now just run it as like a small lifestyle company. And I say we generously. It’s me working five or ten hours a week and a couple of other guys working five or ten hours a week. so it’s we went from being like a exciting but struggling large company at peak like fifty people, to like, you know, three part-time engineers. We now make more money than we ever made and we’re like way more efficient than we ever were and like much smoother than we ever were.

So it’s like a it’s a good company, just not in like the VC funded way. But yeah, I think it’s it’s a it’s actually fun to run a normal company because like the money you make is yours and the time you spend is yours. And so you care a lot about efficiency. And I think AI has been really effective at helping me do sort of normal basic things you’d expect, like when there’s an error, look through logs and find issues. But those things you wouldn’t expect, like I’ve got AI connected to

you know, Slack and email for customer support, but also to like QuickBooks and our bank and you know, my calendar and lots of different things. And I actually found that giving agents sort of that broad access results in sort of interesting, interesting crossovers. Like agents will have opinions on our legal documents, for example, given things they’ve learned about our code base. And I would I would encourage people maybe to think more broadly about how they use AI.

Beyond strictly engineering, I think AI is actually pretty good at non engineering tasks like accounting, like legal. I think it’s also especially good when you combine sort of in sort of cross disciplines. So like understanding our accounting system and our database allowed AI to find actually some like pretty serious flaws in our billing code. So the thing that I sort of really get excited about that sort of like broad context, cross disciplinary.

That’s been really fun.

Yeah,

so I maintain a Git repository of markdown files with context, which has, you know, information about how connect everything and where everything is and what’s important. You know, all the customers are in there and my thoughts on them and employees and all sorts of things. And then whenever a question occurs, I ask an agent posted in that repository. So that’s sort of the an ad hoc workflow. And also just every day have a a daily a daily brief. So every morning I get at seven AM.

Sort of for the last twenty four hours, what’s happened? And it shows me, you know, what customers have been more active or less active relative to recent history, if there’s been any outages, any errors that have occurred. And then from that other thing spawn. But that’s sort of like daily cadence is maybe the heartbeat today. And it’s a separate cadence that runs monthly for, you know, month end close, overages, etc. So mostly ad hoc, but I think the like the daily update does a good job of

Triggering some work. We don’t react to anything today. In some consulting work that I do, I’ve like set up agents that react to, you know, failures. Coiled is Coiled is mature and stable enough, fortunately, that we actually don’t have a whole lot of like failures that occur. And if they did occur, I wouldn’t want, I think, agents to act on them just yet. Everything’s still in read-only mode. We haven’t I haven’t gone to the point where I’m confident in sort of taking action.

I have n

I have no idea. Yeah, I don’t know.

I the honest truth is I have no idea. Like actually it’s it’s interesting. Like I I work with far fewer people now than I used to. Like agents are good enough that most tasks I just do on my own or with agents, which is great. But actually have like I think a far less exposure to how other people are working than I used to. we like we used to need each other a lot more. And that used to cause a lot of communication and sharing. We don’t any we don’t really need each other that much anymore.

Yeah, I would hope that other startups at least are using agents in this capacity. I mentioned I’m doing some consulting work and like we’re definitely very AI forward there. When I hear about other large companies, they seem to be excited about AI, but maybe less far along than what you would what you would hope for in sort of a fast moving startup.

Sure.

I don’t think there was an intentionality. Wasn’t it like a direct causality there? if there if I were to search for a connection though, I would say that whenever you’re running a startup, it’s hard. It’s like you’re always it’s it’s a higher risk, higher reward activity running a startup, right? You might become a billionaire and you’re probably just wasting your time. And that’s hard to do. I think that’s always happening under the backdrop of opportunity costs. I think that.

Today, maybe opportunity costs are very high because there’s so much going on. and so you know, I think I became a little disenchanted with try with shooting for the moon with Coiled. And that becomes something that you are you’re more likely to stop that shooting for the moon process if the sort of broader context around you seems quite rich. so like.

When I when for example, when I when I left Coiled I I for published a blog post and I, you know, said, Cool, I’m I’m done. There were like three or four different companies reaching out with with salaries that were far beyond what I thought were possible. Just because like the employment market today for, you know, sharp individuals is very, very strong, right? You know, people working at Anthropic or taking home, you know, fifty million dollar, you know, pay packages. So

Did I choose to become a lifestyle company with Coiled because of AI? No. Did I choose to stop striving as strongly? maybe because of AI, because of all the other stuff that was going on.

I just mean that in the broader economic context, there’s a lot of activity. And when you’re doing one thing, you can’t do other things. you know, everyone today working at their job should step back and think, wait a minute, something else better that I should be doing. And you know, in an economic downturn, the answer is usually no. You should not be doing anything else. You should stay at your job. But like right now, like there’s plenty of opportunity that’s like out there and crazy and interesting. It’s an interesting gold rush time.

Not sure it’s a good idea to quit your job, but like there’s certainly weird opportunities that are present. If you ever wanted to start a company or start a new endeavor, now’s a good time to do it. It’s never been easier. yeah, there’s just more churn happening. We’re in a high churn era, and I think high churn eras benefit people who are able to take some risk.

Yeah. So it’s you know, match instability in the environment with in th with flexibility in yourself, maybe.

Yeah, no. I I think when I first started writing software, I wrote like early in my career I wrote w problems like tools or multiple dispatch. I was very cared about languages. I like was into clojure. I cared about like the art and craft of writing code. I think over time that I had like I lost that appreciation and I became much more pragmatic around code. Yeah, no, I think I’m like more addicted to the productivity that writing software can provide.

And less to the art form. I can like appreciate the art form, but it’s not something that I I endeavor to practice. And so I’m I am pretty happy with the transition that has occurred. yeah, and it’s also just like it’s intellectually fascinating. Like my brain, I’ve had to like rewire my brain in various ways in the last year. And that has been a an exciting endeavor from not just a productivity standpoint, but also from like a a mental awareness standpoint. So

Yeah.

What what

do you mean by that in terms of rewiring your brand?

I think that programming historically has been you know, like 10% inspiration, 90% perspiration, right? You like have an idea and then you have to like work on that idea. And there’s like a craft to build the thing. and that I mean, there’s still some perspiration, but it’s like, you know, it’s 80-20 now. It’s like 80% inspiration, 20% perspiration. And so you really have to like, even more than before, I think you have to like shape your brain to think well.

Maybe a good example, right? Like I and I imagine met many people were sitting there talking to Claude or Codex or Gemini or Pi or whatever. And you’ve got, you know, seven different conversations going and it’s it feels really really productive. And then you find yourself kind of just like sitting there staring at the little dots, waiting for them to turn a color. So you can go and like engage one of them. And that like that mental state is not a good state. You’re like you’re stressed, you’re waiting on these things to return.

You like need to step away from the computer, you need to go for a walk, you need to come back fresh. And so this is like a standard, you know, mindfulness thing, is to not be too focused on this addictive behavior. But, you know, like you need to think about how your brain works. I think many programmers, many like, you know, top-tier programmers who are maintaining big open source libraries, they think very

carefully about how their brain works. They like have diets that, like, you know, I don’t eat carbs before noon. I do only protein and fat at this time. Find if I have just like a half a cup of coffee, then I’m really sharp. Right. This is like a common thing to hear from a programmer. They like treat their brains the way athletes treat their bodies. I think the same is true in you know agentic programming, just like there’s a different sport that we’re playing now, and there’s a different set of challenges. I think that it’s it’s more addictive.

And so there’s a different set of outcomes. You can become pretty dehumanized. I know if there’s a hard problem, like if it, you if Claude spouts out, you know, pages of expla explanation to me. There’s a strong temptation to just say, like, sure, whatever you think, go ahead. And you know, sometimes that’s fine and sometimes that’s not fine. And sometimes you gotta step back and say, No, I gotta go and relax. Or this is a great problem for me to think about tomorrow morning, when I’m a bit more fresh. So I think it’s an interesting mental game.

Right. I think the sport has changed and that’s cool. It’s we’ve been playing the same sport for the last twenty, thirty years. It’s cool to play a new sport.

It’s mostly just taking more walks. Yeah. I don’t like I still have seven Claude sessions open. I still, you know, there’s still a highly parallel workflow. I try to set up feedback systems and context so that agents can run for a long period of time. So like, you know, thirty minute turn is pretty normal.

Sometimes I will take a phone with me. I like actually I like Codex like remote that remote system is pretty good. So sometimes I’ll do that if I go for a walk. Sometimes I don’t. but yeah, I don’t know. I think this is just like a general mindfulness thing. You gotta step away, tell yourself it’s okay. Yeah, it’s not I don’t have like a a trick on how to maintain a good mental state.

Meditate? I don’t know. I don’t, I don’t like all millennials, I aspire to meditation and don’t. Right. But you know, probably I should do that more.

Yep.

Most of the day I’m in my apartment and I’ve got I’ve got a little like a hook at the end of whenever a turn finishes, it makes a little sound. So it’s like, an agent needs me to intervene. Cool. I walk on my computer, I read something, I write something, I send it. Then I go off and do whatever I’m doing, right? I read a book or I’m cooking in the kitchen or whatever. So I think my day has agents sort of interspersed within it.

yeah, I and then if I go for a walk, I’ll come back and it’s like exciting to come back. I got, you know, seven blue dots or yellow dots all waiting for me that I can go and I can go and take a look at. but yeah, no, that’s most of my days interspersed. How about you? What do you do?

Yeah, I had built my own harness around Claude for a while, which had sort of multi-agent support. And then when the desktop apps got like a nice multi-agent sort of, you have lots of different sessions that are running in parallel, I switched over those and I’m very happy. Claud or Codex are kind of the same to me in terms of the UI. But yeah, it’s been good. Like I I was talking to a colleague of mine and he still uses the CLI. And I was like, why?

I think you just like hadn’t really taken a look at the desktop applications, which is interesting, right? Like things things happen fast enough and we don’t talk that much anymore, as I mentioned. I think there’s actually interesting speciation among workflow development in different people, which is probably why podcasts like yours exist.

One of the reasons I started the podcast.

How do we keep connections to folks if all the information that we need is just out there with LLMs I mean, even in within companies, if you start if you have everything available, do you do you need to talk to talk to other people?

Yeah, no, maybe we don’t. Maybe we’re our connections are no longer based around work. Maybe we join sports clubs. Right? Like I think I spend more time talking to people and hanging out with people. I spend less time hanging out with work people, which to be honest, I’m okay with.

Yeah, it means our like craft gets a little less good, but maybe that’s okay. Maybe the agents help us share, right? I think like, you know, they might have terminology that we use.

They they are getting to know us very well.

One thing I’ve thought about a lot is how does it feel to be managing agents versus writing the code yourself? Do you find it more less tiring similarly?

in general I like it. I think it’s good. I think it’s a lot of leverage. I think it can be tiring, certainly. But if I had tried to write the same amount of code myself, I would also be tired. I think that there’s

As I mentioned before, programming has a couple different mental activities. There’s like the inspiration and perspiration part. I don’t mind losing the perspiration part. I’m fine with that. I now do have to like there’s now like, you know, a much much more time we can spend being inspired. And like we just run into just like limits, like human limits. You just can’t be inspirational constantly throughout a day. So that’s tiring. But like

If only did the ten percent I used to do and then like spent the rest of the time at the beach, I don’t think it’d I think it’d be fine. So yeah, I’m I don’t miss the way things used to be. I’m pretty happy.

It’s interesting, like building new software rather than reusing old software. So I think that we used to use a lot of sort of community soft I used to be a lot of community software, right? And that was because perspiring, writing code was really hard. We actually needed people to work on things with us, right? And we don’t anymore. And so there’s an interesting sort of speciation that’ll happen in software as well. Right. Like I’m working on new projects, other people working on similar projects. Cool. We’re gonna have like many projects that all do the same thing.

And that’s fine. That’ll be good. I think we’ll probably start losing a lot of community standards, we’ll start losing a lot of sort of you know, foundational software. It’ll be interesting to see what survives and what gets speciated. that’ll be fun to play with. but again, I don’t miss that, right? Like we had community open source software because it was really hard to build software. If everyone could build their own custom software, sure I’ll miss the community stuff.

But I think overall that’s good. Like the pragmatism is is excellent.

Think that’s reasonable. it’s not the choice that I would make. but I think a lot of open source projects are also like pretty

They have to be stable, right? There’s like a stability, volatility, and change trade-off you have to make as an open source maintainer. And a lot of error written code is more volatile. It’s happening quickly, it’s causing a lot of change. So I don’t I don’t think that like refusing AI generally is a good policy, but I think that it’s reasonable to sort of reject a highly volatile system into software that’s expected to be pretty stable.

For example, in Dask, I made a bunch of things in Dask recently that are just like, I’m not even pushing into the mainline Dask. I’m just making forks of things. And I’m publishing them in my own my own or my own GitHub organization. and that’s I think good because there are like, you know, banks and healthcare agencies that run on Dask. And like the stuff I’m building is probably buggy. it’s fixing a ton of things. Like it’s I think the overall it’s gonna be fantastic, but.

There’s a value to stability, right? so yeah, I think it’s fine. I would expect traditional open source to be pretty conservative. I would expect social open source to be supplanted by a bunch of other weird projects. I would expect to be a bunch of problems with those weird projects, and then for people to sort of like sort themselves based on how stable or unstable they want it to be.

maybe over time AI gets much better and like we all just learn to trust it more.

And for the projects that you’ve had on your personal GitHub, is this like I saw you’ve been working on a Rust rewrite of part of Dask, for example?

How is that compared to the Python implementation of Dask?

Yeah, it’s it’s fantastic and terrible at the same time. I released it like two days ago. So it’s it’s like a hundred times faster. It talks to agents natively, like there’s whole like dashboard for agents, which is really fun. it I just like hand it workflows and it says like, this is slow, let me go and fix it. And it like figures out why it’s slow, it figures out how to fix it. It’s like a little computational brain. It’s very fun. I I’m having a lot of fun with it. It also breaks all the time.

Right. Like it’s it deadlocks, it doesn’t produce wrong results, but like it definitely has lots of issues. I would not put it in the hands of a customer today, but you know, like in a week probably. yeah, and then like I mean the I would expect it to become a replacement for like early stage users in a whole weeks.

And then, you know, it’ll evolve through normal software things and in a couple months it’ll be a good replacement for a desk generally. will people switch? Maybe, maybe not. Maybe they’ll build their own thing, right? Like it wasn’t that hard for me to build this. Other people could have built this too. yeah.

Yeah, I think software doesn’t have intrinsic value, right? Like curing a disease has value, or helping someone find a home has value. Software is a tool. if we can find other tools that do the same thing, I’m all for that. now it’s sad as you and me, we’ve spent, you know, most of our lives building a craft, and that craft is now obsolete mostly, or you know, it’s 80% obsolete. we still know how to solve problems, and that’s useful.

But I would say that the value we provide is no longer in the writing of software. It’s in the building of automated solutions or something. And so letting go of the craft that we had built or the the thing that we had built is, I think, a useful thing for us to do. I think we’ll find there’s lots of value we can provide. what I tell people who ask about this, you know, people who are outside of software, I say that.

You know, 80% of my skill set has gone obsolete, but the 20% that isn’t obsolete is now like 50 times more valuable. And so I feel like I’m able to do a lot more, as long as I’m willing to really embrace that 20% and really reshape the way that I think, the way that I work around that 20%. When I see when I I think people who will thrive, people who are comfortable changing how they think of themselves and how they work.

pretty quickly. I think that we’re entering an age where dynamism is gonna be really important.

I don’t I don’t know. I don’t work with people anymore. It’s the challenge.

I mean, mostly the things that I see are probably people who are talking about it online. So there’s a pretty strong bias there, right? Because they’re also like pretty talkative people. But you don’t know. I think you need to be curious. You need to explore. I think when we were all like 14, we like got a new piece of computer equipment. I’d be like very excited to put it together and like try something.

Right. When I like first got a computer, it was like, ooh, DOS. And I was like making directories and seeing what I could do with that. There was like some like curiosity, some like childlike curiosity. Right. and we all had that experience when like Python came around, right? We were like suddenly very excited about like, what are all the cool things that I can play with? Then as we get older, we get sort of better at doing our jobs, we get into more of like a grind state, and we’re like able to pump out productivity that feels good. I think we need to get back into the childlike state again.

Right, we need to like be curious, we need to play around. I think I was in a good state because I like able to play around when like in December, January, to able to spend like full time just like building weird personal projects and developing some sense of taste, some sense of experience with these tools. That was really helpful. so yeah, dunno, play, an appreciation of play might be a good indicator.

a willingness to s to see one’s own ego be crushed, probably also a good indicator, because again, the thing that you were good at doesn’t matter anymore. Yeah, we’re all children again. And that’s a thing we have to accept, which is hard to accept, but I think it’s probably an important thing to accept.

there’s a there’s bunch of projects, but I would say as a theme, I actually really enjoy front end. I like I’ve always been like you know the back end of back end. I like you know matri make multipli I make matrix multipliers go fast, right? The furthest thing away from front end. But I also just like visual design. I like, you know, in working running the company, I like got introduced to Figma and I was like, you know, designing things and I’m not especially good at it, but it was like really pleasant to see things.

come through and sort of think about you know user workflows and like how you know accentuating something or bringing something to the forefront allowing other things to fade would really help make an experience better. And so when when AI came around, I don’t know in like Sonnet three five days, if you were active then like the first thing I did was I picked up TypeScript because it was just like, wait, the things that I want to have happen in the application, I can do now directly.

And it was that was very fun. And some of the first personal projects I did were like a new Gmail client. Cause I hate the design. That’s like the visual design of Gmail is like way cluttered for me. And like I use keyboard shortcuts every time. And it’s it it’s not to my taste. and so I was like, cool, I’m gonna go build a Gmail client. That was like the thing that I was excited to do, not go build, you know, Dask and Rust. so yeah, front end. I think front end is really cool.

People should write more front end things. I see people doing like TUEs now, like text user interfaces. That’s cool. I love people doing that. But like TypeScript and React can give like really good user experiences. And honestly, AIs are better at them. Like we should all be building websites, rich client applications, at native apps, all the all the stuff.

I I don’t have strong opinions. I think it’s I think there’s a level above that stack that’s more interesting about, you know, minimal design, color palette choices, you know, keyboard shortcut patterns that I think if I look if look at my personal projects, there’s like a definite theme that’s been established. But I think I use whatever

smart front end person I talked to said last, which was TypeScript, React, Tailwind, shadcn, vite and whatever. but I honestly don’t have enough experience across front-end frameworks to have an opinion. I think my opinion is that it mostly doesn’t matter. I think that agents are fine with whatever. And the real question we have to ask is like, okay, what is what is the user experience we want here?

And are we actually guiding users into that experience? How are we drawing their eyes around the application? Is it obvious what they should do next? These are the things that I would encourage people to think about. I think the choice of front-end frameworks is like now, I don’t know, do you use clang or do you use GCC? Doesn’t really matter when you’re writing C code, right? It comes out the same. I don’t really care about the frameworks anymore.

If you like put a gun to my head, I could definitely not write a Hello World application in front of it. Right.

Yeah, that’s

Do you think generally with languages today that is now other than choosing for the trade-offs of your problem, like Rust is gonna be high performance systems level, that it doesn’t really matter somewhat?

Yeah. I

Yeah, I think there are there are characteristics of languages that important. You mentioned like Rust versus Python. There’s some obvious differences around performance. Right. I think readability is useful. Like I do still sometimes read code. I find that I personally just like I I find that I will often write if I’m if I’m writing in Rust, which is my sort of default backend language today. I will often make sure that it has some Python bindings just so I can write tests in Python sometimes, because I just like PyTest.

Just from like decades of use. Like I just find pytest very natural. and so like I will often keep Python or project just for that framework. so readability is important. language coaches generally, I mean Rust has good, you know, memory safety things. Go produces nice binaries, the like compilations processing go a little nicer. I don’t really I don’t really care though. Like again, I think this is not a it doesn’t really matter. I was

Doing a consulting project recently. I was like, hey, I use I’m thinking but I wanted to switch to Go. And I was like, I don’t, I don’t care. Like, I’m not gonna write a line of this code anyway. Do whatever, do whatever you like. Yeah. I makes me kind of curious about what languages might be like. I wonder if there’s something in between human language and computer languages we might think about. Or like where we did a lot of creative work in the programming languages space.

All that’s moderately obsolete now. I mean it’s useful for agents, but not useful for us. Is there some higher level thing that we think about as programmers? Or is are we just done like is English the terminal language? or or will there be some innovation in languages that’s maybe higher level than historical programming languages, maybe lower than English, maybe above English in some interesting way?

Programmers, computer scientists are quite good at identifying repeated patterns, and then building systems to leverage those patterns. And maybe we’ve reached a point where we’re like, it’s no longer gonna be patterned. There’s no structure anymore, in which case we’re done. But maybe we’re gonna find that there are patterns again. And if so, what what are those patterns and what are the structures one can build in this new era? I think that’s a very sort of like

philosophical computer science question to ask right now.

Yeah, maybe you think about, you know, I when I was working on this sort of new task and Rust thing called Frisky, there’s there’s a state machine, right? So the problem is you have had a bunch of workers and they, you know, they all are doing tasks and they’re going down, they’re coming up, they’re sending communications to each other. So it’s a big, massive distributed state machine that has to be correct. Right. And so the way that I work is I establish that correctness by just running it.

And having various checks on it and then just like stressing it. And if it breaks, I can find out why this breaks, and I go fix that thing, and I just run it again. So a very sort of iterative process. A different approach would be to build a provably correct state machine, right? And you know, you I suspect you know, Dan, like there are languages designed around provability, around proving theorems. Right. And so I can imagine that’s a whole world where I can imagine that agents we could do quite well with theorem-proving languages.

Maybe we want to write code that’s provably correct. Maybe agents actually quite skilled at doing that. And that would solve the sort of like lack of trust we have around them. Cool. That’s a whole research area that I think could have a lot more value today than it had before, when like humans had to go learn how these sort of esoteric languages worked and like, you know, prove kind of silly things with them.

So

Yeah, that is something I’ve talked with a few people about on the podcast, verification being the name of the game in however you define that. So the more you can make deterministic, generally the better you will be. But from the formal verification side of things, no one I’ve spoken with yet as

talked about going into that, into their agentic workflows, but I think there is an interest in can we do that? Where where can we bring that closer to what more people are doing today?

Yeah. And clear, I’m not doing this at all. I am not that ambitious. Maybe that brings up a good point. I think a thing that I tell people that I find to be very helpful, and this can be a little silly, but is to drink more. like and when I say drink more, I mean like alcohol or some intoxicating substance. And what I what I really mean though is like we need to be more ambitious. We need to have fewer inhibitions. I think that

Most like when I when I do go out with friends and I have some drinks and I come I’m like, wait, no, I should totally do this thing. I should take this project and this project and mash them together. It’ll all just work out fine. And like usually it doesn’t work out fine, but it’s a good idea. And when I’m not drunk afterwards, then I can build that thing. but I think there’s a lot to be said for reducing inhibition and becoming much more ambitious. We are all trained to say no to wacky ideas.

Because they’re expensive and we are the ones who bear have historically borne the brunt of implementing things. But I think now that’s not the case. Now it’s actually quite cheap to try things. And I think agents do a surprisingly good job at things. We’re often surprised by how good they can be. I was working with someone recently, and he was working at using agents to write code the same way you would have done maybe back in February. Like, okay, cool, I want this feature built. Here’s here’s my plan on it. Or this thing is wrong, go go go look at these logs and tell me what they say, then go look at these things.

They say. And I was like, no, no, no. Just tell it the problem exists and like where the data sources are and and trust it to go figure that out. And that like changed the way he thought about things. He was working with agents that were good at sort of a small scale task when actually they had in the background grown to be able good at to solve larger scale tasks. I think that we are sort of constantly at a risk of making the same mistake of

not being ambitious enough. Now obviously we shouldn’t trust agents, but we should give them the opportunities to try bigger things more often, I think. Yeah, in sorry, go ahead.

Yeah. so my like global AGENTS.md file has a few phases. And the first one is to set up feedback systems. So those could be tests, it could be looking at number of lines of code, it could be benchmarks. But you know, I and the agent figure out what the feedback system is to make sure that it it knows what it’s doing. Then it implements on its own and it, you know, does a good job at that usually. then it you know uses the feedback system.

And then there’s always a review. So I ask it to review itself and I ask for an external agent to review it, sort of without the existing context. Then if I’ve got some concerns, I will ask some questions of it. So, like, hey, you know, I’m concerned about this particular fail case I was thinking about, you know, demonstrate to me that that’s handled. Or, you know, if right in front end thing, show this thing to me. Make sure that I’m aware that this is what’s going on.

And so I’m almost never looking directly at the code, but I am thinking actively about the problem, and I’m asking the agents to demonstrate to me that my concerns are met. And that works pretty well. I’m happy with that. I think often people will say, like, well, how do you know it’s not doing this? And I’ll say, Great, that’s a great question. Agent, how do I know we’re not doing this? And the agent then has the work of demonstrating that to me.

No, I think the bottleneck for me is

In making decisions. I think that the agents do a good job of building feedback systems, they do a good job of implementing code, do a pretty good job of checking themselves for things that are we’ve talked about. I think that they also do a good job of presenting questions. Hey, I’ve reached a fork on the road, there’s this choice or this choice. Which choice do you think we should make? and then I’ve gotta read and I’ve gotta think and I’ve gotta

Make a decision. And then ten minutes later, I’m presented with a different question on a different thread. And I’ve got to read and think and make a decision on that completely different topic. And so I think for me, the bottleneck is my own brain’s ability to think actively about different topics for decisions that are not clear or obvious. And that’s hard.

and that’s where I think like taking a walk is a good is a good activity. so yeah, I think I’m I think I’m I think if you’re doing things right, you are always the bottleneck to your own workflow. if you’re like if the agents have trouble verifying themselves, you should be investing in a better agent verification system. If the agents are going too slow or you don’t have enough of them, you should figure out how to get more of them or have them go faster.

And I think in my personal workflow, I have been sensitive enough to those issues. I’ve gotten to a point where I think I’m bound on my own ability to think.

Which is why I’m gonna keep on for a second, which is why I sometimes like prefer Codex over Claude, because I find that like Codex like gives slightly simpler explanations of things. And Claude is slightly more verbose. And so I find myself like gravitating to Codex because I find it a little less tiring.

And

Have to switch to caveman mode.

Ha ha ha

You’ve talked about building feedback systems for the agents. What are the most or you wanna talk about that a little bit?

Sure, yeah. so my belief is that the th like I don’t I no longer care about agent intelligence. Like I would I would happily take like an on-device, you know, Qwen 3 model runs my on my laptop over Mythos. I would I I’m fine with a a not very good agent as long as it has good context, as long as it has a good feedback system. I think that those things will cons the context will give it a direction and the feedback system constrains it to good behavior.

And they tend to react well, agents. If you like tell them something, they react well to that, they think about it, they do a good job. And so a lot of my work is now about just establishing good feedback cycles for them. So I mentioned, for example, I’m working this thing Frisky, the sort of Dask clone and Rust. And we just record a ton of telemetry. Every action at like the microsecond level is recorded. And then there’s a ton of code that like summarizes that.

for whoever there’s apis for that and so in Frisky in particular it runs a workload and it can it can very easily see what’s wrong even down to the point where just like it’s just downloading a structured log of spans that it like goes through and runs arbitrary python code on that does a good job. maybe another other such situation sometimes I’m asked to like help out someone’s Python code.

Like, hey, my Python code slow. Why is that? It’s running. I don’t know what’s going on. And so I built like a little Python profiler tool called Wiretap. It’s actually it’s publicly available. I’m the only user. You go to wiretap.justplane.dev, it’s there. And it like, you know, logs it like does it samples their system as Py PySpy would hand sample it, sends it up to a database.

A CLI can then like download what’s going on in that system. It summarizes traces of samples and like it gives me a nice gives an agent a nice understanding of what’s going on right now, historically. And so I find that when I just like hand an agent that and a part person’s Python program that’s running, the agent can figure out what’s going on that’s wrong. Like I think that most of the problems we’re asked to solve today are actually pretty easy. They’re pretty obvious.

And the challenge is really in just like figuring out what’s going on. You know, as maybe you know, Dan, like an open source, when a user came up with an issue, the problem is creating a reproducible issue and then like understanding what’s broken. Then actually fixing the thing is kind of trivial. Right. So really the thing that we care about more is about understanding what’s going on and about sort of clearly explaining that to something.

We can do that as humans helping agents, but I find that work to be somewhat dehumanizing. And so I prefer to write systems that can tell an agent what’s going on. So I I so there’s a thing I want to build right now. I haven’t thought about it, but I want at the end of every turn to just like feedback to the agent the number of lines added and deleted. And just ask it, cool, for the thing that you just built, you added 500 lines deleted 30. Is that an appropriate change for?

this for the feature that you built, right? And like I suspect that if you did that on every turn, agency like, no, actually that’s that’s too verbose. Let me go and work on simplifying that. Right. I find myself saying that to agents often, like, hey, is this is this too complex? And they respond, well, cool, how do we automate that? And that’s like the the code complexity AI slop feedback system that we can address. So I care a lot about just sort of automating things that I would say to the agent.

and giving that to them in a way that they can get it directly without me having to get engaged. And that’s how I think I avoid a lot of the sort of dehumanizing, tiring work and spend more time taking walks.

I yeah, I recommend people do, especially if you have like a I was talking to guys at NVIDIA recently, and they had like no idea you could run models on a MacBook. because they just like, you know, have infinite GPUs there. but yeah, if you’ve got a MacBook with like, I don’t know, 32 gigabytes of RAM at least, like you should ask your normal agent to set up an on-device agent for you. They’re fun. I think my first experience with them was I found that they were quite intelligent, but didn’t know that much.

I would ask the same question of like Gemma 4 and of Claude. And Gemma 4 was like, a good question. Let me go ask the internet. And Claude was like, cool, here’s the answer. And here’s like, you know, a PhD level treatise on the answer. but I’ve heard that they’re like, they actually do pretty well on programming benchmarks, which makes me think about the separation between knowledge and intelligence. And I think the thing I’ve come to is that they’re actually pretty intelligent, but just don’t have that much knowledge baked in.

And then recently I was working on a project where I had to learn about a code base. it wasn’t a greenfield project, it was sort of a helping an existing code base. And I found that working with an on-device model, it was a lot faster. Like the iteration cycle was much faster than with Claude or Codex. but it also required a little more active driving from me. And so I was a little bit more engaged in the process, which I think I liked.

and so I f if I today if I want to learn about something, I will switch to an on-device open source model. it’s kind of like driving in a city. You like turn on the GPS and have it navigate you. Or if you really want to like learn the city, you know, you like you turn off your GPS and like try to navigate there, you know, manually. I feel it’s kinda like driving without GPS. it’s still quite good, but like you have to be a little bit more aware. It’s I think

An efficient way to learn something.

is being written to a much more precise reader, maybe, which makes us, I think, actually think a lot more about what we write. I wouldn’t be surprised if we’re all if our writing skills maybe improve, but certainly change as a result of this.

I mean, going back maybe to like the art of programming, I think what we liked about the art of programming is it allowed us give us a a very precise language to convey an idea. Right? Like it’s it’s a it’s it’s extremely clear, there’s no ambiguity, it’s checked by a compiler, right? We can convey something that’s quite interesting and complex. I think that now we have all we all have these brains that are used to conveying ideas.

And we’re now conveying those ideas in English. And that’s interesting. It actually forces us to use that same part of our brain that cares a lot about precision and a lot about quality of idea, about the choice of words. And it kind of forces that same part of our brain into natural language, which is, I think, interesting. Right? We are encouraged every 10 minutes as we’re talking to these agents.

To write very precisely the idea that we want. And I wouldn’t be surprised if we become much better at communicating generally as we speak to our family, to our friends. We shouldn’t talk to them like we talk to agents, but I think we might get a lot better at.

Communicating without, for example, like I can tell Claude had a bad idea. I don’t get angry at Claude. When it screws up, I don’t yell at it. It’s not useful.

Nah, just go build just go

I’m building my thing. You should go build your things. Everyone should go build their own stuff. I rebuilt Dask and Rust in like three or four weeks. People other people can do similar things. Have a good time with it. We should all be having a lot more fun today.