How AI Agents Change the Work of an ML Engineer

I’m Dan Gerlanc and welcome to Agents and Engineers, the podcast about agentic AI and software development. Today our guest is Niels Bantilan Niels is the Chief Machine Learning Engineer at Union, a core maintainer of Flyte an open source AI orchestration platform.

The creator of Pandera, a data validation and testing tool for data frames. His mission is to help ML and AI practitioners be more productive.

He has a master’s in public health informatics, and prior to that a background in developmental biology and immunology. His research interests include reinforcement learning, NLP, ML in creative applications, and fairness, accountability, and transparency in automated system. Niels thanks for joining us today.

Yeah.

Yeah. I guess I’ll I’ll start with Flyte. I am as you said, the maintainer of Flyte and and Pandera. Two very different flavors of open source. Flyte, just for context, is kind of MLOps. It grew up in the ML ops orchestration space. We’re kind of pivoting, we’re kind of changing like how we talk about ourselves and talking more.

AI runtime and what that means, but

Effectively, it solves the problem of how do you build systems in production that have a machine learning component and with agents and AI increasingly what is the I guess cognitive architecture, if you want to call it that, around the the code that supports the code and the other pieces in the stack that supports the typically LLM that helps you build very flexible, powerful systems.

So Flyte is that type of project similar in the problem spaces like Airflow or other orchestrators out there with a few, or I would say a couple of core assumptions that that make it particularly well suited for machine learning and AI workloads at its like primitive level.

So that’s Flyte. And then Pandera is a data frame validation library that supports all sorts of data frame formats now. I’ll start with Flyte just because it’s there’s a very interesting kind of balance that you may relate to as well around open source and commercial. So whereas Pandera is very much a pure open source project that

I have not made any moves to commercialize at this point. That that may change at some point in the future. But right now it’s it’s like a labor of love. I started it as a side project, kind of merged it into Union as an open source project after I joined. and so that has some different properties, but I think for Flyte

And for both, right? But in particular, Flyte Agents has really, I think, helped me and and the Union team, who is like the main maintainer and shepherd of of that project. Flyte is also it’s owned by the Linux Foundation. but I mean I can start at the front line, right? So a lot of our developers are now just using agents like to implement

features and do bug fixes. I I would say if you already have a pretty well structured and opinionated project that has linters and type type linting, type checking, like a s kind of a style guide, so to speak.

and fairly mature in this the structure of how modular modules are laid out, how the unit tests are all structured. Actually agents at this point in June 2026 are are really fantastic.

I use kind of a mix. I generally use Claude or Cursor. I’m slowly migrating over to Pi. I’ve dabbled with OpenCode but yeah, I’m just trying out the whole local sovereign AI story and it’s you know, it’s it’s not as good, I would say, off the bat, as the commercial offerings out there, but I think I’m slowly getting there. a lot of it I think just has to do with

the prompting and like sort of the there’s a little bit more work you need to do in terms of like the skills and the harness you have to attach to these open source agent harnesses.

Do you have a Mac with a

lot of RAM?

So I I host I’m currently hosting Qwen 80 Billion Coder Next on it. And yeah, I think the the class of tasks in my job that the local AI does really well are mostly bug fixes and very small scoped changes. Like, hey, I need to add a flag to the CLI.

And make sure all the changes proc propagate through all the relevant parts of the code base does that super well. if it’s a longer running task that might take an hour to complete. I found that I can give a pretty sizable like part of a PRD pro product requirements doc over to like Claude and it will kind of

For the most part, hit it out of the park.

And as I yeah, as I mentioned earlier to you before we started recording, it’s like a lot of reading. So it’s sort of like you have a you have a coworker and they can spit out thousands, hundreds of thousands of lines of code. So you just you wanna be careful because you’re not you don’t want to review like a PR that’s massive, just like with irregular, like pre-AI agent workflows.

So the

Is that something

you have fairly strict requirements around that like things still go through a PR workflow into very well-defined changes?

For the most part in the project, yes. our team is has an engineering background. Most of us at at Union AI are backend engineers. I am a MLE data scientist by training. but I have I would say I do have more of an engineering mindset than like a researcher mindset. I do get a little,

more particular about sort of like style and and formatting and and things like that. I mean we have linters that for that now to like abstract that away.

So yeah, I mean we we still do PRs, we still do code review. most of it I would suspect. I I haven’t like asked this of my team, but I suspect most of us are have a fairly high level of trust of these systems now, and we’re mainly looking for code smells, security-related things, performance-related things.

that just immediately kinda stand out to us. but our velocity is like I would say much faster.

I mean I I wouldn’t say it’s 10x I would say it’s like

At least three.

I mean as I said, I it’s it’s been a while for me that I’ve got into the weeds and gone into like the into the traditional debug like REPL loop of running the debugger on like VS Code or Cursor. And like my debug loop is literally the agent making those changes. I’ll like add I’ll add tests.

via prompting. It was like, hey, you didn’t catch this, you know, like, hey, test this part more and you know, maybe do I mean we don’t use hypothesis like the the property-based testing thing quite yet. but you know just add more test cases to cover this like part of the execution path. and then yeah for the most part if if my prompt is is small enough in scope it’ll like do that very well as well.

when I debug stuff also, so Union and Flyte, it’s like a cluster, it’s a Kubernetes cluster on some cloud somewhere. I literally point we have a Flyte MCP now, so I just point

My agent to the config file that that will authenticate through my browser and then open up a session and it’ll just like run workloads and the MCP exposes logs and errors that happen on the Kubernetes cluster. So my agent basically has access to Kubernetes arbitrary computes within limits to test test out what I’m building.

about that implement do you feel like in this case?

There was a strong reason to go with MCP because you’re then hooking into a Kubernetes cluster.

Yeah, I would say kind of the latter. So I could have created a skill that like the answer is auth, basically. Like I could have created a a s a skill that describes the kubectl commands to like get the logs from the pods that are erring out. But Flyte also is like an abstraction over Kubernetes, so I

Like I don’t want to deal with Kubernetes as much as possible. that’s just you know.

An

an explicit decision that yes, that I’ve made. I’m not a platform person. I don’t ever really want to interact with it as much as possible. so

The Flyte MCP that we have just I I provide my like Flyte API key, and that just you know it has access to whatever parts of the Kubernetes API surface we surface to users through the Flyte API. so it’s a it’s a much smaller constrained surface. It gives me all the logs and errors and stuff. That’s mostly what I need for a proper like agent debugging loop. and it’s nice too because the agent can figure, hey,

this was an out of memory error, so let me just like slightly change my Flyte configuration to ask ask for a little bit more memory. it can even like potentially profile the data set that I’m working with and analyze it a little bit and kind of estimate like okay now I need to provision maybe four more gigs. so yeah it’s I I don’t know I feel very spoiled that like I have

Access to agents in general for development, but then as an ML person, ML is like the the core diff difference between a software engineer and an ML engineer is like generally compute. Like a software engineer, you can like unit test stuff, right? You can like mock out things, which you could do to a certain extent in machine learning, but

For anyone out there who’s like trained a model, like the magic only really happens at a certain scale. You have to wait for a while for like the loss to go down enough, for things to start kind of snapping into place.

I’ve I started off as a data scientist.

Because I I mean I enjoyed the visualization part, the understanding of the data part, the modeling of it. at the outset when I sort of like came in from grad school and and started learning about the space. soon after though, when I started productionizing stuff at my first startup job,

That’s sort of that’s when I learned about like build systems. I I wasn’t I’m not like a CS or classically trained software engineer. I guess software engineers generally like you learn on the job actually anyway. So all the stuff around CI testing, unit testing, integration testing, all that good stuff. That that part actually was fun for me also. And so I my sh my

Mindset kind of shifted to okay, it’s cool to have like stuff in notebooks and like analysis that provide insights and may help you make decisions, but it’s like when you start serving models in production, there’s like a whole set of other concerns that you have to learn about on top of like the all the other skills that I think it’s is good to pick up as a data scientist. and so now with AI engineering, my my

kind of reasoning by analogy is inference has taken up the r this the oxygen in the room. Like ML was all about training.

AI engineering is mostly about inference time things you can do with the model, which before was like all I do is predict a thing, right? And it’s just like a scalar, like for the the regression or a classification thing. Now with LLMs and like the multimodal stuff, it’s whole documents and whole videos and images, right? So

the the space of applications has just exploded on the inference side. And you’re kind of now seeing a little bit of a return with RL and like a j it’s an overloaded term, right? You have RL agents and then you have AI agents and there’s an overlap, obviously. and so a little bit more emphasis coming back to RL and like training.

And it’s a little bit it’s a hybrid thing because with RL there are rollouts, so you need like an inference server somewhere generating rollouts for whatever your latest checkpoint is or your latest policy is. So we’re starting to, you know, we’re we’re because we’re a platform company, Union AI is, we

We have to serve our existing customer base and as trends shift and as people’s use cases change, we turn our attention to those things. And so RL is something I have the the RL textbook here, you know, so I’ve kind of like picked it back up

Yeah, we’re exploring this. I mean, it’s because Union is general enough, you can like build everything yourself from scratch. But you know, the a big part of my job is just like ergonomics and developer experience. and like even though humans maybe take less and less of the coding time, agents still need, I think, good APIs, you know, for them to understand stuff.

in scare quotes and also humans for for reading the APIs and and sometimes like fixing the code manually so I mean at the platform level there’s there’s many things to do. the this textbook is more algorithmic so this is more just like what are the core pieces you need, right? You need an environment which could be CPU-bound maybe the generate frames in the whatever world you’re in.

could be GPU bound also. Your model is GPU bound, your your your like the training, the model that the is updating weights is GPU bound, the inference servers be GPU bound. So Flyte allows you to kind of like orchestrate all these pieces somewhat seamless seamlessly so that you can feel like you’re just programming locally, but you’re actually

Like when you do a for loop over your environment, you’re actually hitting an a a separate pod from the one that’s in your training loop. And the data is just being transferred super quickly through like a Rust powered object store so that you know it’s the data transfer is has a little overhead, but you know, it’s like within some like tolerance that’s acceptable. so they’re they’re like those infrastructure pieces that we’re we’re starting to look at.

Yeah, so I guess the the broader point I’ll make here is that when you’re building a platform that serves not just one company’s requirements but like a lot of people’s requirements, it’s hard to really optimize for everything, especially if it’s like open source. If it’s a closed source platform.

Everything is hidden, right? So it’s like the the thing you can really focus on DevEx and just making the the the user-facing SDK super nice and making the experience magical.

And so if if your MLE, ML researcher, AI and is your main audience, that’s like your main focus. Our challenge is we have to serve that end user, but also a platform engineer who has to maintain this on a cloud somewhere. so we have to make very careful decisions on what technology choices we make. All that’s to say on the data transfer and like all that like optimization.

level stuff. We haven’t invested a ton on sort of like the GPU stack. We we we often rely on open source technology for that. I mean the Rust ecosystem has been amazing for us in a general sense because it’s just way more performant than Python. our team loves like programming in it

I don’t know the latest on if agents are very good at it. I think my my vibe my vibe sense is that they’re like okay, but not as good as Python or JavaScript. but in any case, the we’ve

Yeah, anecdotally

I’ve heard that they don’t always use the type system to its full advantage.

I see. but yeah, to your original question just around like data transfer and and GPU optimization, a couple of things we do there around

yeah, using Rust powered libraries for general data transfer. So in Flyte, like you have tasks and you have composed multiple tasks to for a larger application. Each task runs on its own pod and its own container. So you need a way to like take the output of that one pod and save it to blob store and then load it up into the next pod to do the next job. and so that with that we used RustFS

To quickly transfer the data. I think our bench internal benchmarks, I’m not sure if we publish them, but it’s like I think it’s the 500 gigs and four seconds in terms of download time, which is which is pretty good, I would say.

Is that writing to blobs

to something like S3 or is it writing well wow?

Yeah. Yeah.

S3, it’s any S3 compatible

object store. I forget the upload times. I think they’re a little yeah, I forget the the the exact numbers there. But in terms of GPUs, we invested a little bit and it’s using the same kind of RustFS backend, but we have a system that will automatically load weights from S3 into the GPU without having to download all the weights to disk first. And there’s a

So that will use like the direct

to GPU memory way of doing it. Yep.

Yeah, yeah,

exactly. one fun anecdote around there with agents was that there was an off by one error in the like so you you save a bunch of like tensor chunks to object store, right? And in object store also you keep like a metadata file, like you think of a JSON file that just is sort of an index of you know which

Tensor chunk maps onto what part of the the model architecture when you when you load it up into memory. there’s like this nasty off by one error that I just like could not find. And this was pre me being agent-pilled. So I was just like still sifting through the code and like trying to debug it and

The symptom was basically the LLM was just outputting like garbage, garbage, like symbols and random stuff, right? so there’s clearly something wrong.

Just a hard thing to debug,

right? It’s like what’s what’s going on here? Yeah.

Yeah.

so then I I think at the time I forget what I was using, but I just gave it the context, I gave it as much of the logs as I as I had access to, and then, you know, within five minutes it was like, yeah, here’s here’s the like the minus one you forgot to add to this like one part of the code. so yeah, that I think that was the beginning of my agent pilling story or journey.

I think that’s the thing that worries me is I’ve I’ve stopped I’ve I’ve started just like skimming what the agent’s findings are. Like ‘cause it’ll output a report. It’s like, this this is all what happened, right?

And yeah, I’ve I’ve started getting to the habit of just like not really trying to fully understand what went wrong. And that I feel like that that worries me ‘cause I think I’ve already lost the the hard earned developer skill of just like finding a needle in the haystack. and now it’s like I’m already slipping in terms of my understanding of like what actually went wrong.

So yeah, it’s I don’t know. It’s it’s a push and pull. I I don’t know if it’s yeah, I feel bad about it, but at the same time it’s like I have like so many other things to do. context switching, ‘cause sort of like wear multiple hats at at Union that just like I’m just incentivized to move on.

I I don’t know. I feel like I am if my only job was to be an engineer, I feel like I would still want to deeply understand the issue even even if if I’m not the one to find or even fix fix the bug. but yeah. It my role is just so much context switching that it it

Like I feel like more of a technical PM who had a coding background. who still does it for fun. And I I still like sometimes roll up my sleeve and like write from scratch. for like fun stuff. But yeah, it’s it’s getting harder and harder and

I I would just like for anyone listening, I would just say, you know, hold on to that as much as as you can, ‘cause you still need to understand stuff. Like I think that’s the last bastion of of what we can do as humans.

Yeah, I think in personally I know on systems or code where I’m more familiar with the area. So I’ll look at what it’s doing sometimes and be able to say, hey, from a structural or understanding point, here is a better way to to do this. Because I’d understand it better if you had this abstraction or not, which

I feel like is an area that’s still LLM’s, AI agents aren’t they’re not lazy, so they they don’t care as much about creating

Yeah. yeah, I mean when I do I mean I I I do still do a lot of code review and reading of what it’s generating, right? So

it it does all sorts of dumb stuff enough so that I’m I still don’t like hundred percent trust it. like an example is it just I I had a variable called like memory key and like the original script that I had called it memory key underscore something else. And it kept the it kept that indirection. So it just like reassigned the constant

To memory key or like with the old value just because it it was like too lazy to refactor the places in the code base where it it had the old variable name or the old constant name. and so you just had memory key underscore something equals memory key, and and it’s just like why why’d you do that? but you know, the the code worked, so as far as it was concerned, it it was fine.

we have in some of our repos we do have GitHub Copilot enabled. It catches some good stuff. we haven’t set up bespoke things or integrated with other third-party services. and that’s I think mostly like we just haven’t felt the the strong pull to do so. our internal agentic systems are

Are a lot around like internal operations. we have one that’s like the the inspiration of for the mascot we have internally is like the Butter robot Rick and Morty. So it all it does is change node pool configurations. So it only has access to one repo. It can only

read and write like a specific set of directories in there like YAML files of like customer node pool configurations. So customer asks for hey I need to bump my limits for this instance type and then so Nody is its name and so it it will make a PR that we have to review it. I think

As of today it’s been online for like half a year and it’s like closed couple like hundred plus PRs for these node pool configurations and you know they’re they’re customer facing, so it’s it’s critical and we do code reviews for that and make sure everything passes our like internal tests before merging it. it’ll get things wrong. Our tests will usually catch it and you know we’ll fix it and

That kind of like makes it into the context, its context, because it has access to GitHub and, you know, the Git repo itself. So it can kind of like see the cases where the PRs were were edited by us.

So you can activate

it through s you have it tied into Slack.

Yeah,

yeah, yeah. so we just reply to the thread, the customer thread where they made the request and we just at Nody and then it’ll kinda wake up and do its its thing. we also have like an official node change form that our customers have access to. so that form will directly kind of trigger a webhook on Flyte. So the Flyte agent then wakes up and and does its thing.

we have another one. We our naming convention, I I guess I I am to blame or take credit for it. so we have another one called Doxy, which syncs up our docs with code changes from various repos. so PR comes in from our SDK, and then Doxy will go and see is this a meaningful change that we need to document? and if so, it will

Write the page and then make another PR to the docs repo.

Yeah. I mean a a ton of other areas. It’s really just like a matter of time investment to the initial like hump of setting a thing up, and a little creativity of like, okay, what what kinds of problems map onto agent capabilities? it’s a it’s a big space, right? So I have or

We have internally like a PRDs repo where we work on our PRDs. We have a go-to-market repo for like the technical side of the go-to-market team to take in all the context, write blogs, write code examples that support the blogs. we I ha I have like a

local skill that just like has a laundry list of all of my previous writings. So I just say, you know, hey, like, assume my tone and voice. And so it it it gets gets it part of the way, and then I go and edit on Notion. and so there are like parts of it that it I can outsource pretty effectively to get

like writer’s block out of the way and all that stuff. I mean this is not literature, right? So it’s not like I’m

I do have fun in the kind of creative writing process, but again, it’s sort of like a race against time. So it’s like given the choice between facing a blank page and just like writing the outline and going through my whole writing process, it’s like easier to get the initial thing and I’ll make substantive changes actually. I’ve there are some times where I’ve just like

effectively rewritten large chunks of it. But the core the core flow and the story and like narrative it gets right. It’s just like wordsmithing and like the taste part of it, I would say, that I I still put my my hands, my fingers in the pie.

But it’s it’s pretty widespread. Like I would say, at least for my role, it’s like it goes from coding to like customer customer solutions, customer success, and and marketing.

Yeah, so Nody is a Flyte powered agent and I think that was that was like our first that that was born out of a hackathon actually and then it we just kind of productionized it and it’s sticking around. we’re in the process of making the Doxy agent I guess what we’ll call a background agent or a long long horizon agent that kind of just wakes up when it’s needed.

I I would say our flow so far has been someone has a problem and they will start off with Claude or whatever local agent harness they they’re working with. And I have to hand it to the those projects, right? It’s very easy to integrate all the sting all the things with it. So you have like Notion, Linear, Slack, GitHub, all the stuff, and it’s like a point and click

kind of OAuth experience to connect your like local system with it. And so you can get really s started really quickly and prove pro prove the concept or like prove the value of the agent. Like somewhat this is like the manual version of of agent agentic development, I guess. You start local, make sure it works, you get some value out of it. And

We don’t have yet like a good heuristic of like, okay, when do we graduate this, or should we even port this over to Flyte? generally the shape of things that we productionize in Flyte are roughly speaking, like what activates the agent. If I really do need

like a human opening up a terminal and like giving the thing context for a particular task. I think that’s it’s a it’s a good time to just keep it local at that point. When you have things like GitHub events, if if you have like external programmatic events that should wake it up, that’s generally good signal to put it into Flyte. So if like a Slack event

wakes it up and based on that event you can just like

gather all the context needed to complete the task. or it’s a GitHub PR, and same thing. Then it’s it’s easier to set up, or I think conceptually it it should be going to Flyte. I think the challenge is the whole integration piece. And so I mean that’s kind of signal for me that Flyte should have a story for that to make it like really nice to

either use a third party service or build it internally where you know, it’s a whole thing. Like you need to make a Slackbot or you need to get make a GitHub app and you know it’s it’s not too bad, but it’s still enough friction for it to for me to just defer to my terminal and be like, okay, well let me just do this on my terminal because it’ll be kind of quicker on the sh in the short term.

Yeah.

So I I see the value of all sorts of like MCP services and and other types of services that support the the agent harness. it’s it’s like, you know, it’s for convenience. It’s like what all of us in tech kind of know well like the trade offs of of that.

engineering workflows, which can have very different shape and time frames or much bigger compute needs and like some agentic.

workflows, it’s the compute is all going to the to the model versus what you’re doing locally is not a lot.

Yeah, I think we’re I think we’re getting to the place with agent agentic applications where we have a pretty good profile of like I would say, you know, just hand wavy like eighty percent of the use cases is like

Most of the compute is outsourced to an LLM provider. If you’re lucky enough to self-host, like congratulations. You can like host an open weights model that’s good enough to serve your needs. you know, here GLM 5.2 is great and I’ve played around with it a little bit on one of our instances. Very slow, just because we don’t have a very big one.

And the the compute that the agent itself needs a separate from the LLM is like very, very light. So I think that’s like I would say maybe eighty percent of the the mass of the applications right now. where I think Flyte is differentiated and

We’ve see started to see some rumblings of people at least expressing demand for this. Is the agent becomes the workflow orchestrator for your traditional ETL pipeline or your traditional ML pipeline, your your hyperparameter optimization workflow, right? It’s no longer hard-coded.

and you give the agent a bunch of tools and the tools are the point solutions in or like the the nodes in your traditional DAG. this time the agent can like compose them arbitrarily.

the benefit of Flyte is it it’s type aware and it uses types in Python to understand like how to connect the pieces. And so the agent is given all of that information to compose the tasks. We have a thing called code mode. We use actually Pydantic Monty, which is an awesome project. folks haven’t seen it, it’s really nice. But it’s like a Rust-based

sandbox, which I know is an overloaded term, but it it’s a way for you to run Python code. It’s like a Python interpreter in Rust for like a strict subset of Python that prevents you from doing network calls or doing any IO

And we’ve Yeah, and

we’ve hooked it up ‘cause cause Flyte has a notion of files and directories. This is how you express like writing arbitrary bytes from that one container to the next, as I said earlier. also with data frames, so like parquet files, other data frame formats. And so the the power of like Pydantic Monty plus Flyte is that

You can actually do IO indirectly because Pydantic Monty gives you the customization utilities to be like, okay, these are like allowed symbols and these are like allowed functions that you can write and

you can actually compose this tasks that run on a separate pod, right? So the actual like control flow logic is still happening in the Pydantic Monty sandbox. It’s just that the compute is happening elsewhere and the I/O is indirectly happening mediated by Flyte. So like Monty doesn’t know things are happening in terms of I/O, but Flyte is just handling that under the hood. so you can actually

It just knows here’s

your types, input output types and that that’s cool.

Yeah. Mm-hmm. Here’s like the variable

that is like basically a pointer to this thing in S3, right? and so yeah, code code mode I’m I’m having a fun a lot of fun playing around with ‘cause it’s it’s a a nice kind of abstraction where tools are perfect.

To to say, here are your building blocks, here are your nodes in the DAG. but there is no DAG anymore. Here’s the agent. The agent’s gonna come up with an execution graph to compose the the tools. And if you have any like prescriptions about how the tools should fit together, here’s like some skills. And skills are just like text that specify how to use the tools in some sequence or in some way. so

these are all like built into the Flyte SDK now and and are open source as well. so I think to circle back to your question, I think if an agent needs like disaggregated or heterogeneous compute where the the loop itself runs on a CPU is like not very memory or compute intensive, but the tools are

It’s like, hey, build me this model and do HPO on it, and you know, here’s your metric that you need to optimize. this is basically auto research that I think Andre Karpathy popularized, which is like fancy hyperparameter optimization where like the search space is text over the code that it’s writing. you can do everything in between. You can do like the very structured, config-based.

Like here’s like a model or set of models and here’s like the JSON config for like the hyperparams for it, all the way to like hey, here’s like an initial seed training file, edit it until you know you make the number go down to like a certain certain number.

Yeah, I think that’s it’s cool and it’s it’s interesting because like what Flyte will take into account for is and for ML kind of problems, which is always a big challenge is where is your data? How does it get from one place to another? Like if you’re on AWS you can’t be sending

gigs and gigs of data over the internet or I mean you can but it’s gonna be expensive

Yeah. I mean that’s how we’re kind of conceptualizing this notion of AI runtime. I think we’ve seen peop other people talk about it in the market. we agree for the most part. Our specific take on it is basically durable workflows, like all the tasks and durability in this case means

You have a 100-step pipeline, step ninety-eight fails. There the system understands that all the previous steps succeeded, and so you’re not gonna have to recompute all that. You just start from the where you left off. that’s like one piece of the durability puzzle. the second part is serving, so it’s like you’re not you’re not only wanting to

Yeah, so take the RL case, right? You’re not only having this kind of like long-running compute-intensive training job. At every checkpoint, you also want it to spin up a separate inference server to do rollouts, right? So even in the model training case today with RL, like you’d still need online serving. and obviously the the the majority cases like here’s a model endpoint that you like hit or your customers hit.

and then the last piece is multi-silicon. So it’s like that means multiple clouds, the GPUs and TPUs and whatever other processing units come up in the future. So like having a system like Kubernetes that like mediates the like container c container and compute orchestration behind all of it.

so that’s how we like think of this like AI runtime component. It’s yes, you it it is agents as well. but I think moving into the future, agents are gonna be also compute bound at the tool level as well.

Yeah. Yeah,

yeah. Yeah, exactly. so yeah, it’s it’s been it’s a fun journey so far. Like we started in around twenty I think the team the Flyte team spun out of Lyft in like twenty nineteen and I joined in twenty twenty one. Or twenty twenty and then I joined in twenty twenty one. so it’s been five years and less like the evolution in this space has just been like

pretty nuts. like several just like existential moments for me, at least personally, of like, what the heck are we doing now? You know? yeah.

What

what do you mean like with agentic software engineering or just with L LMs generally and our role as software engineers

Yeah, exactly. That the latter the latter point. It’s just like if if one gets caught up in the narratives, the Frontier Lab nar narratives too much, it it sort of just like takes you takes you for a loop and like just like questioning what what is the point of like having built all these skills up. But then when you start putting things into practice

it becomes clear like, there’s there’s a role for me here actually. And you know, so that’s I’ve I’ve settled to the point where it’s it’s it’s clear to me that there’s still like a role for a human ML engineer, AI engineer to build systems and build them like with assistance. but yeah like

Creating a a auto research Ralph loop is it’s it’s not gonna get us to AGI. Like there’s a there’s a reason like Anthropic and OpenAI are still recruiting people. as much as they they talk about the other stuff around like recursive self-improvement. to to get that system you still need humans, right? So

Yeah, yeah.

So yeah, I mean I I the the experience with Flyte has been actually quite different compared to Pandera because Pandera has just been a pure pure win in terms of agents, just because generally like you know, I have a two-year-old now, family and it’s it’s like time is very scarce.

And so like the late nights I would spend till three in the morning, like building out a thing in Pandera, fixing bugs, maintaining the project, just just becomes a lot more tractable. and there isn’t like a huge commercial pressure behind it. So it’s purely just like Yeah, it’s like my exercise of like, hey, I’m just giving this purely back to the community.

People are benefiting from it, people are using it. It’s useful and kind of boring in a sense. Like it’s not it’s not gonna get 10,000 plus stars. You know, it’s it’s like it’s not like an agentic thing. So it’s I’m glad it is where it is now. And I’m like like happy at the level it’s it’s gotten to. And I’m still excited. I think I have an

angle that I want to work on that’s goes a little bit more in the agent space, so like vector databases and like validating things that are not that are like quasi-tabular in the sense that it’s a table, but there’s like a vector inside of that or like images inside of that. so there are projects like DAFT and a few others out there that provide like a data frame like structure for multimodal data.

Obviously I mentioned vector data databases and RAG use cases that I think are like interesting areas to get into for the project.

And if not for adoption, maybe just as a intellectual exercise to just see how far I can take the kind of abstraction and like the framework that I’ve built. So and and to do it in the in a way that doesn’t like bloat the dependencies and all that stuff. So I think there’s like still a interesting engineering challenge there that still motivates me.

so actually a contributor just shipped the Narwhals backend. for I guess folks who are not super familiar, Narwhals is sort of like it’s like that xkcd comic of like the standards. It’s like but I think it’s done a really good job. So I think it is actually interoperable

data frame library that kind of like subsumes is like subsuming various of the other frameworks and wrapping it into like a really nice Polars-like API. and so we just released 0.32.0 for Pandera and

That will add support for I think the Ibis, Polars, and PySpark schemas so that you can like use a narwhal’s backend to validate those schemas. and so we’ll you know we’ll play some catch-up to like add pandas to that just because Pandera’s pandas functionality is just like way more since it’s more mature. so there’s a little catching up to do on that front first.

before getting to, you know, to throw out a few ideas out there of like LanceDB is a cool like Rust-based vector database. a few others out there.

Part part of me like maybe is like, will Narwhals add support for?

LanceDB or whatever, right? I think with LanceDB you c you can export it to Polars and then validate that. So there’s some there may be some quick wins to be had there with some trade offs. But yeah, overall I think I’m curious about sort of the the place Pandera can play in like the Agentic AI stack.

as part of

your data validation and it’s going to the agent can

Yeah, yeah, exactly.

I yeah, that’s the my decision to not make HTML report Pandera error reports is I’ll do it eventually, I think. I think it’s kind of unavoidable to have like a nice, pretty artifact. so there’s a project called Pointblank, the great validation library that I think was ported from R into Python. there

Main differentiator is is differentiator is part besides the API, like you express validation rules differently, is you have beautiful like HTML reports, which I think, you know, I think is definitely coming from the R DNA of like visualization. Whereas Pandera just like spits out of plain text, like it’s it’s not beautiful, but definitely it’s agent ready. I mean it’s a very concise

blob of text that says which columns failed and like what were the failure cases, what were the types that failed. so yeah, it’s you know, it’s I think

I haven’t had a finger on my pulse recently of how agents are starting to be used in data science. I know Eric Ma’s work, he I mean he’s like running a tutorial in SciPy this year about it. and so yeah, I I wanna like make sure Pandera is kind of ready when that adoption starts to take off so that like you know it’s it’s you know

whatever th it means to be agent compatible, like Pandera is is sort of that soon.

Yeah, yeah, exactly.

Yep, yep. Just use Jinja, write some HTML, whatever like templating it needs, it just uses Jinja. So yeah, it’s I think markdown is still good for like the human AI interface. but I think from a display perspective, like HTML, you know, you just put whatever interactive elements in there you want. it’s it’s pretty pretty nice.

like your diffs your your diffs start to have a lot of stuff in there you might not care about, at least related to the content, things like that. So I’m

still kind of in the markdown camp as well.