Episode 025 · November 17, 2026 · 01:09:27

From a $20 Contract to Browser Automation at Scale

Dan Gerlanc
Dan Gerlanc
Podcast Host
Suchintan Singh
Suchintan Singh
Co-Founder, Skyvern

Skyvern founder Suchintan Singh joins Dan to discuss building browser agents for people who do not code and rewriting those agents as models improve. They also examine benchmark limitations, AI code review, and the risks and opportunities of wider agent adoption.

0:00 −01:09:27

Suchintan Singh is a founder of Skyvern, which uses AI agents and computer vision to automate browser workflows. He previously built machine learning platforms at Faire and Gopuff.

Dan and Suchintan trace Skyvern’s origins to a failed marketplace search product and a search for better browser automation. Suchintan cold emailed 100 founders, found four pilot customers, and built the first version in a weekend. One early customer paid $20 a month for insurance quote automation that cost roughly $2,000 a month to serve, yet even an incomplete run saved its operations team work. Skyvern later open sourced its product to reach users and build trust with enterprises.

Suchintan describes browser agents as increasingly similar to coding agents, with a crucial difference in audience. The people who understand an operations workflow may not know how to program, so they need an interface for building and monitoring automations themselves. Running browsers and responding to website changes also create costs, while anti-bot defenses raise questions about which kinds of automation a company should support.

The team has repeatedly rewritten its agent as models improved. In the latest approach Suchintan describes, the model chooses when to inspect a screenshot or HTML rather than receiving a preprocessed bundle of page context on every turn. He says the change increased token usage but improved cache hits, cutting cost and increasing speed. He also discusses Rustwright, a browser automation project used in production at Skyvern but still in alpha and short of Playwright’s full feature set.

Dan and Suchintan discuss agent-assisted development, including Skyvern’s multiple AI review personas and the danger of reviewers generating endless low-value findings. Suchintan is skeptical of public benchmark scores after watching products perform well on tests yet fail in practice. Skyvern helped build WebBench with more websites and write-heavy tasks, though he says its usefulness as a differentiator diminished as scores rose.

The conversation turns to giving non-engineers agents, the difficulty of keeping a custom model current, and the different views of AI adoption Suchintan encounters in Toronto and San Francisco. He also worries that AI could misrepresent minority histories when source material has not been digitized, and that persuasive answers can be hard to verify. He remains hopeful that agents will save people time, while stressing the need to address cybersecurity and other risks.

We have nothing. We have no product. I saw you have this problem. Can we help you solve it?

The people who actually do that work and understand it very well don't have a coding agent yet.

Our token usage went up, not down, with this approach. But the cache hit rates were so high with it that the actual cost for us was like fifty percent cheaper.

If the data it needs to train on is not digitized, how can it ever know the truth about a particular culture?

Skyvern
Open source browser automation project founded by Suchintan.
Rustwright
Alpha browser automation engine used by Skyvern in production.
WebBench
Browser agent benchmark covering websites and varied tasks.
WebVoyager
Earlier web agent benchmark discussed in the evaluation critique.
Pi
Minimalist coding agent mentioned in the architecture discussion.
Transcript

Skyvern’s first customer paid $20 a month for an agentic automation that often got stuck halfway through an insurance form. Even so, it saved the customer time. On today’s show, our guest and I discuss what it took to transform these early experiments into an agentic browser automation platform.

Welcome to Agents and Engineers. I’m your host, Dan Gerlanc. Today’s guest is Suchintan Singh.

Suchintan is the co-founder of Skyvern, which uses AI agents and computer vision to automate browser workflows. Previously, he built machine learning platforms at Faire and Gopuff. In this episode, we discuss how early customer pilots shaped Skyvern and why the team decided to open source it, why Skyvern rewrote its browser agent as models improved, and how it reduced the cost of running automations.

And what browser agent benchmarks can tell us, and why strong scores don’t always translate into useful products. Enjoy the show.

Suchintan, great to have you on today. Welcome to the show.
Yeah, nice to meet you, Dan.
So you are the CEO and co-founder of Skyvern, and you guys were part of the YC 2023 summer batch. For folks who aren’t familiar with Skyvern, can you briefly describe the kind of problem you solve?

Yeah. So Skyvern is a open source browser automation tool. Um we actually today help lot of non technical people like vibe code browser automations, as if they’re programmers. And it’s um one of the most rewarding things I’ve done.

basically companies big companies will take our product and give it to their operations team and um the the people who often do the work manually who can’t program, they would use our product to actually um create their own automation so that’s a as as you would as a programmer with like playwright or something else.

And how did you originally get into developing this?
Yeah, the story is very long, but um it was not our first idea. Or
Hm.

the one we worked on during YC. So um you mentioned we did YC YC Y Combinator um in summer ‘23. Um we actually started a startup almost a year before that and went through maybe five pivots before we were working on the idea that got into YC and which also didn’t work out, and then we afterwards we pivoted uh into into Skyvern, which is working out.

And and it’s you know, you hear stories about startups pivoting a lot. We definitely lived lived that. Um lived that in our history.

So did you start with the open source and then build commercial around that?

Yeah. Uh actually no we didn’t. So um when we were going through YC, we were building a tool to help marketplaces optimize their search. It it it was based on something I’d built at two companies I worked at before. Made them a lot of money, we’re trying to bring it to other marketplaces. And um we’re having a lot of trouble selling that product. So I was looking for browser automation tooling to help us kind of make the demo process easier for that. If you could like go to e commerce websites and scrape all their products and show how much better the search could be on the first call, it would like I like,

people will buy it more. Ended up not being true. But but that was

Ha ha ha.

the that was the hypothesis. And while we were looking for that tooling, I talked to a bunch of startups back in twenty twenty three working in the space and nobody actually solved was solving the problem well. And so we did something that like I read about doing but never actually tried. I um

Cold emailed 100 founders who talked about some browser automation problem. Uh I I I can share with you, Dan, after what the email was, template was. But I sent

the email on, and the email was literally like Yeah, it

Yeah, that’s I can uh put it in the show notes.
was literally like, we have nothing. We have no product. I saw you have this problem. Can we help you solve it? Can we help you solve it? And and I emailed about 100 founders and about 30 actually emailed me back, which is crazy high.

Wow,

that’s yeah, that’s a crazy high response rate.

crazy high. Pre maybe

pre AI, so it’s okay. Um, but

Right.
thirty emailed me back and four ended up signing pilots with us, knowing we didn’t have a product.
And was this like on LinkedIn you reached out to them or just email direct? Yeah.

Yeah, you l mostly email actually. I just cold emailed

them. Yeah. and four ended up signing pilots with us, knowing we didn’t have product, and we kind of um started with those four. We lost two right away within the next like month and uh we ended up working pretty closely with two of them for the for the next few months, which um created the seed for Skyvern and and then a few months later we were like

Okay, this is like a a lot of startups are starting to do the same thing as us. And we’re having trouble like selling to bigger enterprises. And so we’re like, okay, like how does a startup actually like sol solve this problem? And you know, I’m a big programmer, my co founder is a big programmer and and a huge nerd, and so we’re like, what if we open source it? Uh what what if we open source it and and become like the first like open source AI browser automation tool uh in the world? Um and um

And we wanted to do it for a few reasons. One was like we we wanted to like sell it to enterprises. Two, if you look at history, all the all the popular browser automation tools in history were open source. Selenium, Playwright, Puppeteer, all open source, very popular. uh and and and three was we we just uh thought that it would be like a really good way to kind of get our product in the hands of a of people.

So we did. And we launched in Hacker News, um, I don’t know, like a week after we open sourced it and we like three thousand GitHub stars in the first week. And I was like, uh for that I got like one star like in all my in all my projects. So it was like a big

Nice. Yeah. Yeah.
it was a big wave and uh wave wave of excitement that came from it. Yeah.

So when

did when did you open source the repo?

This is in March twenty twenty four. Yeah. And today we’re at twenty two thousand. three hundred GitHub stars. Yeah. So it’s I I stopped counting after we had five thousand. A after five thousand,
Who’s count who’s counting? Yeah.
every enterprise is like, Yeah, it’s a legit project. We’re like this
Yeah.
is legit. It’s not like some some shovelware, you know. So yeah.
And how how long did it take to build the initial product in the when you were doing the pilots to get to something where you were actually doing work that you could charge money for?
So the first version we built in a weekend we c technically
Nice.
we charged money for it, so it kinda worked out. Technically we did
all right.

it wasn’t very useful though. the first our first customer wanted they what they did was they resold like insurance quotes. Um and so they wanted us to go to Geico.com and basically fill out their like thirty page flow to generate an insurance quote. And I think they paid us twenty bucks a month.

And

I think it cost us two thousand dollars a month to serve their volume in like tokens back then.

Mm-hmm.

and it and it also just didn’t work most of the time. Like it like it would send the prompt, it would like go to the website and like start filling it out, and then we get stuck halfway through. And something I didn’t realize when we first signed them was even that was useful. Like in that in that f in that form, they could send the prompt to fill it out, and at at the point it would get stuck, they could actually have their like

operations team pick up from there. Which meant that they didn’t have to do the the work up to the point it got stuck at and see see value from the product. Which was like uh unexpected for me when we first sold it and and and and very, very cool. And they were our first customer. I th they’re no longer our customer today, unfortunately, but they were our first and like, you know, y y you you all have a very, very special place in my heart for it. Yeah.

Right. It’s still

like your first love. there you go. That’s I mean

Yeah. Yeah, paid us twenty bucks a month for the entire time. You know, I wrote us for six months and made

like hundred and eighty bucks or hundred and twenty bucks. It’s awesome. Yeah.

Oh,

it’s I mean the the line between paying and not paying is large, especially when you’re first starting out, right?

It’s big. Yeah.

Yeah.

And what makes building so mean when you first started doing this coding agents or when you released it coding agents weren’t yet really a thing, right?

No. And and this is uh a funny topic ‘cause if you actually look at our Hacker News launch, we got like we were like top top one on uh on for that day. And the top comment was, Can I have this write code for me? Which kind of like is funny. We didn’t listen to that feedback by the way, w which is a probably a mistake in our startup, but we didn’t listen to that feedback. But basically developers at the time were asking for coding agents. And this is a year before like Claude code was even released.

Like before it was a product. This is like when tab completion was like the cursor with like tab completion was like state of the art. Or or g GitHub Copilot. Um but the top comment was can I just like have this write the Playwright code for me so that I can deploy it and use it? Yeah.

And were like, Yeah, no, that’ll never happen. Well, we well
It’s
we we hope. Maybe in the magical future.
Yeah. Yep. And it did just, you know, a year later. Well, maybe two years later, depending on who ask. Yeah.
And so from the beginning were you using eight using LLMs to help navigate the pages?

Yeah. Yeah. So that was that was the where we saw we were adding the most value from the start was LLMs even back then were really good at ambiguity.

And

so where we got lot of use, um, was websites where the where what you were doing was different every time. And so insurance quotes are like a really good example of that where the questions they ask you are highly dependent on the person that you’re filling it out for. Like, you know, your home might be in New York or it might be in California and they might have like local regulations for like flood or whatever. And the insurance forms will actually ask you those questions depending on where you’re where you’re filling it out from, which made it incredibly difficult to automate with code because you couldn’t

anticipate the world of questions they could ask, but it was like really easy for LMs to just read the page and answer the question, if that makes sense.

And how did the open source project differ from what you offered from the commercial side? How did you distinguish those?

So we actually open source almost everything, even today. Um almost everything is out out there for people to use through the open source. Um and and you know, all the previous generation of browser automation tooling Playwright Selenium, they like weren’t monetized products. But they’re actually really easy to monetize because um running browsers is really hard. Each each one is like, you know, a couple of gigs of RAM.

And you have to manage a fleet of them for the automation itself. So it’s it is pretty hard, um, even for developers to do. Um there’s like a huge crop of companies that sprung up to help company help developers do that. And then also like when you automate things on the webs what on the web, you have to deal with like the adversarial part of it. Like, you know, personal agents are taking off now and none of them can order on Amazon because Amazon has blocked it. And so um there’s a lot of value to be created of kind of circumventing that.

There’s a lot of ethical questions about whether that’s a good idea or not, or like which things should be circumvented. Like are the blocks like in good good good faith? Or are they trying to prevent evil? Or are or is evading it gonna like create more evil in the world? Like for our product, like our number one use case was for a while people kept asking us to do like TikTok spam or like create like a million Reddit accounts so they could go spam threads. Like those are obviously not things you wanna support as a business and not kind of behavior you wanna condone. Um

And so uh we didn’t open source any of the antibot stuff that we work on to help people access like, you know, Claude or blocked government websites so that people could like pay their submit their taxes automatically. Right? That’s like pretty pretty um like well Yeah. Uh yeah, maybe um

Probably not a great idea. Right. For them for them to do either. I guess. Uh-huh.
Like that stuff’s like generally okay, but like people do want to do a lot of nefarious stuff. Um and so we didn’t want to open source that part of the part of the product. Like so none of the anti-bot stuff is open source. Um, which makes it o monetization a lot easier for a company like ours.
Did you have exper how did you learn to build that side of it? Did you have experience with that from previous work or was that something you kind of learned and built de novo?
man. we definitely learned a lot of stuff about this uh on the on the job. But when I was a kid, I used to play this game called RuneScape a lot.
RuneScape, yeah.

Yeah. You know RuneScape. And um back in I think 2009 to like 2013 it like the heyday of RuneScape where they basically weren’t enforcing any rules. And so I used to write a lot of bots for RuneScape. I I I remember I wrote this one uh that like ran this maze over and over again. And it had like, I don’t know, ten thousand users and uh

you know, my player got really rich, it was awesome. Um but

I know.

but you had to like deal with a lot of anti-bot stuff that RuneScape had put into place back then. So like um for example, they used to like detect how rhythmic movements were as you were like playing the character. Um people follow

a very specific pattern, but it’s not rhythmic. It’s like not precise like computer’s are. It’s like a little bit arrhythmic, but it’s still pattern. Um or w the way you move your mouse, like, you know, the obvious one is straight line. But you can’t even take like any elliptical curves. Like the way people move it follows like this thing called Bezier curves to actually go take the action.

And so I learned all that stuff when I was building

Right
building my bot and you know, we were like doing like OCR to figure out like where o OCR back in the day, like where like image recognition to figure out like where where the prayers are so you turn them on so you don’t die in the particular part of the maze and stuff. ah so I was doing all that stuff as a kid for fun, which I guess is how I learned out a program, uh kind of kind of accidentally. And i Yeah, and then like a decade later it’s actually relevant like you know, to like what I’m doing for for for my startup. Yeah. When you

Right. Yeah. It’s it’s funny how things turn up again when you don’t expect it.

And if I if I remember correctly, you had also worked in data science or ML uh in previous roles, is that correct?

Yeah, that’s correct. So before Skyvern we’re building this tool to help marketplaces do better search. Um and before starting a company I worked at two really big marketplaces, Faire and Gopuff. And they built their search engines from ground up. replacing a vendor they were using t with like an in house system that for those companies like increased their conversion by ten percent. Which was a lot of money. Like which is a lot of impact, yeah.

So I I I was using like these ML and models and LLMs before they were cool. And kinda back back when they were still kinda janky. Yeah.

Yeah, yeah, I I mean I worked in ML

as well and yeah, my first jobs were in quant finance, so I learned uh what I would always tell folks who were gonna work in data or ML is make sure that you work somewhere that has a large enough problem that it’s worth solving. Uh

I mean there were tech there were every every company has a d ML problem.

They just didn’t realize

Right, yeah.
the the way it would be deployed would be with LLMs not like specialized machine learning, right? Um but w w one thing I remember oh, go ahead.

Right, exactly. Yeah. And at low cost. Yeah.

Yeah, and at low cost too, right? Like you can build infinite flexible classifiers with LLMs these days and with yeah.

they were hard well, you I don’t

know. Um I don’t know if you ever use BERT models back like twenty twenty one. But they’re really they were like remarkable back then too. Um just not as I guess not as remarkable as like the Jev model is, for example, now. But

Right.
but they were I remember we’re doing some query classification with it and I was like I was like how did it figure out how did I like it figure this this the fact that this person’s searching with like five typos and this actually me they actually mean this, you know, and yeah, I remember
Yeah, well I r I know when Jev came out some of the Bert and other folks who were like, Hey, we’ve had this for a while now, like so
Yeah.
isn’t it’sn’t new. I mean
Yeah.
maybe tuning it better is an improvement, sure.

but you gotta be careful

uh you gotta be careful of those guys ‘cause even I remember uh in twenty twenty one I was working on like a lot of search stuff and there used to like two camps of people, one would be like pro neural nets and one would like anti neural nets. And they were like all like statistical models this statistical models that.

Yeah.
And implicitly they’re all betting against this like AI movement happening.
Yeah.
Um and I uh I think a lot of company I w i now I guess it mostly doesn’t matter, but I remember like they were like not keeping up with the BERT style model improvements because of that kind of mindset ah
Yeah.
of classical machine learning, which is super interesting to think about w with the benefit of hindsight

Yeah, because before

it was all I mean

like random forest and decision trees, yeah,

Decision trees, this decision Yeah.
yeah. Or I mean regression, like tradition linear type models of some kind. or decision trees, like one of the regular regularized regression or tree models mostly.
Yeah, and then you need like an arcane Jupyter notebook to actually like hit train one and hit go. Yeah.

Yeah, exa yeah, exactly.

yeah, I I think I r I I guess I pr probably in like twenty the end of twenty seventeen when uh Jeremy Howard had the fast AI course. That was I think when I was first being like, Yeah, this is gonna start

to be something, at least on the machine vision side. And then yeah, the using embeddings on the language side. And then people started using embeddings for tabular data problems. And around that time it was like, yeah, this is gonna be in a lot of cases the most cost efficient way to make predictions or

Do ML.

Yeah.

Or or or for us it was for search it like laten latency efficient. Like we ran this like really interesting experiment when I worked at Faire that measured the imp impact of latency on the u users like beh conversion behavior. And we found that we could basically take five hundred milliseconds and the person would buy no matter what. I it was probably different for every marketplace, but for them it five hundred milliseconds. Which meant that like using a fast search engine that

Like too fast. It didn’t actually matter, if that makes sense. And so you’d like this like, you know, four hundred milliseconds where you could like put pump some good ML in and really like give them good recommendations back or give like good search results back.

And a lot of it just came down to like how much how how mo how big of a model can you run in like four hundred milliseconds and still serve serve it to the user. And so it created like both this ML problem where like can how big of a model can you train? As well as like a uh engineering problem, which is like how can you serve the inputs to that model fast. Yeah.

Do you think that’s changed today with agents and LLMs? Or has it not changed? It’s just.

I mean it’s bas it’s basically the same, right? There’s cases where

speed matters, there’s cases it doesn’t. Um in general speed is good. But, you know, if you’re doing like risk stuff, which I think you mentioned you might you were doing p in the past, like speed was important but not like the most important thing is like accuracy was way more important at that point in time in y for those kinds of use cases.

W whereas if you’re doing like real time work, like um, you know, you’re trying to predict what the person’s trying to type while they’re typing on their phone, then speed definitely matters, right? And and and both. uh yeah, okay. Yeah. Yeah. It’s like uh you’re like measuring nanosecond or microseconds, right? So yeah.

Yeah, when I worked in quant finance our holding period was like four months, right? So it was or like one of it wasn’t high frequency, so no, yeah.

So that was kind of starting to become a thing, I think, r around when I started. Um but yeah. Yeah.

Compared to quant, I think I was doing easy ML with my with the

millisecond optimizations, you know. Yeah.

Yeah, but I think

the highest frequency we had was multiple times a day, maybe. Uh but yeah, it was I mean, everything we built was in R, which was very slow. So it was

Mm.
not uh not high speed by any means.
Yeah. I yeah. I think for me, like I remember spending a lot of time looking up like compression libraries and um encoding libraries to like squeeze the juice out of the data. So i if you could like encode and decode data over the wire faster, that means you could transmit more data in the same amount time. And similar same same with compression, right? So if you do like compression if you use G zip it’s like super slow. but has a high compression ratio. Which which is fine if you’re not

yeah, GZIP is terribly slow. Yeah, that’s like a hundred megabytes a second.

Decre

Yeah, which is fine if you’re like

if you have that tolerance, right? But there’s other libraries. Um we use this one called LZ4. I benchmarked a lot of them. Um, which are really like well tuned for speed and you could and and compression ratio and and then you could squeeze you more juice out of the ML models ‘cause then could ferret more data to the model, you know, in the same amount of time so it could make a better decision. Um pretty fun stuff.

Yeah, it’s I I always tell people today you’re you’re either using yeah, LZ4 or Z standard compression, depending on uh what your trade-offs are.

Or gzip if you want general compatibility.

Yeah. Yeah.
What makes building browser agents most different from coding agents?

I I don’t think they are anymore. Yeah. They they used to be quite different. But um as coding agents evolved to be what they are today, they’re basically the same thing. Um basically the same thing with a couple caveats. Um one is

The u the primary user of coding agents is still programmers. But like you and I don’t actually know the work to automate that well. Like if you’re working at a bank and you need to do some like know your customer flow or you need to go to some like to some like background check website, like I actually don’t know what to search up there well or like what edge cases to handle. And so coding agents st are still made for like you and I to use to build code. But the people who actually do that work and understand it very well don’t have a coding agent yet. That that

w wherein like the code actually doesn’t shouldn’t matter. And so we spent a lot of effort in like our UX to see how we can make coding agents accessible for those people who can build and monitor automations at at a scale that you know you and I are used to as programmers. Um so the UX side of it definitely is like a big different delta than than standard coding agents. Uh the second the second part and this one is still pretty hard, is um

For an automation to run well, you actually need a coding agent to be running alongside it. Um if the website changes, like how do you identify like what the change is and whether it’s like good keep going or like bad, flag it to the person because maybe information’s missing. So you need an agent to run alongside that. And um because you need an agent to run alongside it, any high vol any high scale thing, it’s gonna be expensive.

You know, I mentioned our first customer, they’re paying us twenty bucks a month and it costs us two thousand to serve them. That was like a really extreme example. But it basically means you need to run cheap LLMs alongside these so you can like make the cost profile be like acceptable to a business. And so um it I would say that

What

we are building is basically a coding agent. It’s just packaged differently and um uses different models under the hood to like op optimize for cost rather than for accuracy, which a lot of people care about coding agents. Yeah.

And as time has gone on, have you found that you can do better with less expensive models or using less tokens with more expensive models?

What’s interesting is we’ve had to rewrite our agent three times in the last in the last thir in the last two years. We’ve actually had to do like a full incinerate and start again from scratch three times. Um and it’s because uh over time like your harness the the thing around the model ends up like understanding the details of the model. So the first version we wrote was for GPT-4 and it hallucinated all the time and it couldn’t like think at all. And so we had to build a lot of logic into our like our

tool chain to kind of deal with that, which ended up not mattering at all when the when they released reasoning models. Um and then more recently, um, when OpenAI released GPT 6 Luna and GPT 5.6 Luna, like the cost was cut by like, I don’t know, sixty percent or seventy percent. But it also required an agent rewrite um to work with that kind of model.

And and so the answer is yes, but it’s not like a plug in p plug in and and see see the rewards. we’re expecting to do one more rewrite in the next twelve months for for the Jev style models. the accuracy’s just not high enough today for it to be better than something like Luna, but but when it will be. We’re definitely switching to that, which requires again a complete rewrite of our agent.

Is is there anything that really competes today with like the Luna or Haiku class models for this kind of work?

Deep S DeepSeek’s DeepSeek

definitely competes with it. It’s like a maybe thirty or forty percent cheaper, but um

Like the latest DeekSeek Flash models,
Yeah.
yeah.
Yeah. I think it now has image processing as well, so it definitely competes with it. Um um but you know LLM cost is no longer the main worry with the text to Luna.

And you work directly at

the page code level or do you use some combination of converting to images? What does if you’re if an agent your agent is coming in looking at the page, what does that look like to the agent?

So this was part of one of our rewrites. So I’m gonna I’d love to explain like what we used to do and what we do now. Um we actually did like a big overhaul a few weeks ago where we increased our agent sp agent speed by 2.3x and cut our cost by 50%. Um and

it’s pr it was

Pretty good.

pretty cool. Uh better than I expected, to be honest. Um and um with coding agents now, particularly ones l like Pi.

Um which I don’t know if you guys have heard of, but Pi is uh

Yeah.

a very minimalist coding agent where you can add add functionality as needed. You can you can start doing this thing called progressive disclosure where you don’t actually give the model anything and you let it pull information it needs. So uh the way it works today is uh the model gets a task and then it it has access to grab the screenshot, it has access to scan the HTML.

It can has the act ability to interact. It c it has the ability to do a bunch of extra stuff like solve 2FA and so on. And it can decide what to do based on the feedback it receives from the website or based on feedback it receives from the LLM on what to do. And this is quite different than the Yeah.

So these you’ll provide as tools, or these things you’ll provide as tools

or skills to Pi, essentially.

Exactly. And so what we found was

actually like our token usage went up like sev like I don’t know, seventy percent. Uh it went up, not down, with this approach. But um the cache hit hit rates were so high with it that the actual cost for us was like fifty percent cheaper. And

the input on the read token cache.

Exactly. Yeah.

Um and and um with this approach also um it was way faster because each turn just had less context passed into it. So what the architecture was like before this with the older models, which weren’t good enough to do stuff like this, didn’t follow instructions l like long enough to do stuff like this.

We basically have to do rag on a website. So we’d like take a screenshot, we’d like annotate it, um, we’d grab the HTML, we’d like run it through like an algorithm that would like distill it down to the important parts using the accessibility tree and a few other things. And we’d shove all that context to an LLM and be like, What should do? Right.

And this this approach was really good because um it made it so the older models can actually operate well on these websites, but it’s expensive because you’re often passing in context it didn’t care about. Like if you go to Amazon you’d like pass in the footer of Amazon. Like does it need to know what’s on the footer if it’s asked to like search something on on Amazon?

You you’d have

like a vector database of all the components, something uh or like in memory database

Yeah. Yeah. uh
and be like which of these are relevant to the next step? Something like that.

Exactly. Yeah.

And so um just inciner incinerating that approach, like quite literally, which w was essential for old models but not at all required for the new ones, led to like this dramatic increase in performance.

So now you essentially can have one thread, one main context thread, and then
Yeah.
the model iterates it. You have it in
Exactly.
it’s cached, so it’s like one hundredth or ten percent of the cost, or magnitude less cost per model turn.

Yeah.

Um and I expect with the Jev style models that approach won’t work won’t work either. So it’s kind of like kinda funny to like know what’s coming in the future, if that makes sense. because they’re they’re classifiers that are not coding models. And so you wanna basically give them the same context of what’s on the website and ask them to classify what action to take. on the website. Yeah.

So tell me a little bit more about the most recent rewrite. I think I read that you also created a new version of or engine for playwright.

That was that was a side project. Uh yeah. Okay, so one big imp one important part of running a browser automation company is the browsers themselves are a big line item. Um if you’re not careful, each one can occupy like two gig gigs of memory and usually one or two V CPU. And if you’re running millions of them, that ends up being quite expensive. you spend a lot of money with AWS to to host it. And um we’re trying to figure out how to like

lower that profile. And one of the things I learned about play playwright is um when you actually use playwright it runs as like subprocess in Node to go interact with the website for you. It’s like kinda like a proxy. And that thing like two hundred megabytes of memory that it was like using for for nothing. Um and so we Yeah.

Just ‘cause it’s running a node VM

and loading whatever libraries there it needs.

Yeah. And so I was like, okay, like what if we rewrote this in Rust, where you kinda get rid of the middleman there? And so you have bindings to every language that needs to use it, like, you know, Python, JavaScript, whatever.

Even Rust itself doesn’t have a good browser automation tool. Um and uh you make it do direct connections to the browser instead of going through like a proxy, um, which is what Rustwright ended up being. And when we benchmarked it, the impact was like insane. You basically deleted that overhead and the memory usage like shrank pretty substantially. Um I forget the exact numbers now, but it might have been like s 70% less memory usage for a particular run.

So should everyone just be using Rustwright?
And if you find bugs, report them please. It’s still ex it’s
Ha ha ha
it’s still it works for us. I will say it works for us and many other people. But it is like an alpha product still. So if you you should g definitely give it a try, especially if you’re running stuff at scale. Um the Yeah.

Are are you running it in production or

experimentally?

We’re running in production, yeah.

So so

it’s it’s it it’s good enough for you at scale. Right.

It

i it’s good enough. the

reason I’m cautious in saying that is because playwright actually like a lot of surface area to cover and not all of it is ported over to Rust Right out of the box. Um and we don’t use all of that functionality at Skyvern, so

depending on what you need in playwright. Your your mileage may vary.

Yeah. Like Playwright lets you connect to like Firefox.

It lets you like um do all kinds of like w screen size interactions and all like record video in a bunch of wacky ways and I I don’t even know if people use that functionality still. But they support it. Right? So to make a claim it

Feel like people are

using it to record what their agents or how they’re interacting with websites and then proving that the functionality worked these days, something like that. I feel like that’s a new popular thing.

Yeah. Have you given that a try?
Yes, actually, uh we have uh I’ve used it it sometimes I’ll be like, hey, go implement this and then record a demo for me. Uh and show me that is working. And it’s actually a pretty effective way in some cases. Like even with terminal things I found it works sometimes.
Well, if I find it l lets them catch their own mistakes quite a lot of time. So the the the video you end up getting is like way higher in quality than than it would be if you hadn’t asked it to do that.
Yeah, it’s all a lot of agenc is creating that verifi verification feedback loop.

One one crazy thing we’ve had a lot of success with is um in our MCP we added this like feedback tool um where uh agent or or um agent or person using our MCP, the agent can now give feedback to our product.

directly. So if there’s ever like a missing feature or like a missing requirement it need it needs and it can’t do something, it can give feedback. And then we set up like a coding agent that listens to that and like fix fixes it autonomously. So so we can we can kind of like automate like s like small scale bug fixes, if makes sense.

And are you still I know this is kind of the topic du jour, but are you still using standard PRs, CI, or like the traditional software development life cycle or are you working in a different way?

So um we have PRs for every change that goes in. It has to have an approval. Um one one of the things that a lot of people don’t know about is like uh to sell like B2B software, you need to be SOC2 compliant. It’s like a virus, like any company with SOC2 compliant basically can only buy software from other SOC2 compliant companies. And one of the controls for it is you have to have human code reviews. Otherwise you violate SOC2. And so we still have code reviews, unfortunately. Unfortunately.

Yeah.

Which it can be s someone reviewing an AI code review.
The AI c quality of AI code reviews I think dramatically exceeds the quality of human code reviews now. Um outside of architecture, I would say, like or like big like bigger picture stuff. It it catches small stuff way better than people can.
And you do you use any kind of code review tool or is agent a code review tool or you have kind of rolled it yourself with a prompt that

Yeah, we we rolled it ourselves, yeah. Yeah, we run a c we run Claude code in like a a box that goes and reviews the code. Yeah. We found that it was really easy to tune like our our architecture and like have like a high level guidance on what the product sh should be like with with that approach.

And then we actually run multiple so that I can give different perspectives of feedback. Like we’ll have like one that’s like a staff like a staff engineer style. Another one will be like a designer style. Another one will be a security reviewer. So we have a few personas that run.

So you’ll r you’ll run those in separate contexts so they don’t

Yeah.

It’s a little bit dangerous though because c like a s I don’t know what happened, but s since March, all these coding agents basically can’t like say they found no problems. So they all

Mm, yeah.
particularly the security one, there’s always a security problem. So you gotta be like really careful about how much you wanna listen to that thing’s output. Otherwise like if you let it be fully autonomous, it’s just like it’ll keep finding issues and then keep fixing them. And it’ll turn into it’s like
Yeah.
token mess like you’ll basically burn like hundreds of dollars doing nothing. Yeah.
Yeah, I’ve seen some people say that what they then do is if there is a finding, they have an another agent decide whether that finding is relevant or not and then stop.

Yeah. Yeah.

We we have the we have it stack rank the findings into like priority and then we basically just tell it ignore anything else like not high. Yeah. uh

Yeah,

I know ‘cause I’ve like with personal projects I’ve had I’ve sometim I’ve built like a service where I’m running it initially off my local file system, right? Or something like that. And it’s like, hey, someone could do path traversal here. I’m like, yeah, I’m not gonna do path traversal on myself because I don’t think this is a problem here.

Yep. Security man, it’s interesting to me.
This is a slightly different topic, but related to agents and the work. Is that I believe that you would help create WebBench. Is that correct?
Yeah. Yeah.
So what are some of the problems with current browser agent benchmarks?

Man, don’t give me started on benchmarks.

Like I largely think benchmarks are useless. Like I largely like don’t think like they’re useful to know that you’re like on the right track, but they are largely useless from the public perspective. The reason we created WebBench was because every startup was like uh the before WebBench, the best benchmark for browser agents was this one called WebVoyager.

And every startup was like, I got eighty-nine percent, I got ninety percent, I got ninety-one percent. And they were like increasing about half a percentage to be like, I got the best. Like a maybe, I don’t know, go fundraiser, like have marketing material or something. And it was really uh difficult because you’d actually use the products and they wouldn’t work. Like you know. so so um that then we looked into WebBench deep pretty deeply, and we found that it had like fifty tasks. Okay.

Like you as a pr like I think it was like sixty tasks or something like that. So you as a programmer could write sixty scripts and get n ninety percent, you know, all on this benchmark that was like supposedly very hard and uh meant to exemplify like what it’s

What you what you mean that

you could just solve get ninety percent?

You could l literally sit down for like two days and like write the code yourself and then get ninety
Okay. Okay.
percent on the benchmark, right? It wasn’t like substantial of any any any any any dimension. Yeah, it was uh yeah, it was just a bunch of
Okay, so it was yeah. Easily solvable, yeah.

Yeah. It was a useful research exercise, which is what what it was originally meant to be, but then it turned into this marketing thing, which is not what it is meant to be. So we we built WebBench. and then and actually stopped maintaining it shortly after that. But we built WebBench, which basically ten X ten X that number.

Our hypothesis was uh Yeah, number of tasks, num number

In terms of n in terms of number of tests that needed to be solved.
of websites. Um I think it was like four hundred and fifty websites the webbench went to with like around five thousand tasks. Um and it expanded the types of tasks that were in the in the benchmark. So it would be like the original one had like nothing related to logging in or solving 2FA
Yeah.

or anything like that, which is obviously a huge part of the internet. Right.

Um so we added

Yeah.
a bunch of like write heavy tasks where you had to like fill out really complicated forms or like go and log into the website and then carry out some operation afterwards. And benchmarked all agents and they all sucked. Like
Yeah.

like for the for the ones who had to fill up forms, like the highest pass rate was like, I don’t know, like forty percent. It was like it was like crazy. Um and then and then people f got the benchmark and then they you know, they did the same thing again and eventually all the agents were like eighty percent. And so benchmarks in general are just like

this

thing that is useful for that point in time that it’s released and after that you should ignore it. Uh the most recent one that happened too was this one called Odyssey’s benchmark, which another um I think CMU or something published, which is another br browser one. And then we ran an R agent on it. And now it’s ninety percent. It’s like okay, n now that it now that we’ve passed eighty, you can ignore this benchmark and any results you get from it. Which which is true for every benchmark I think out there. Once once the number passes eighty, you can basically ignore it.

So the answer just

If something’s less than

eighty, you should definitely look at that ‘cause that’s bad. But anything higher than eighty you can ignore it. It’s like that’s all the same. Yeah.

So is the answer just keep creating new benchmarks that push the limits of what the agents can do, solve it, then move on.

I mean that’s that’s a whole category

of startups. That’s the whole category of startups now. Like when we built WebBench, we partnered with this company called Halluminate and they basically became that company. But that’s like one of the fastest growing startup spaces right now, where you b make benchmarks for big labs to train their models on. And the value props basically that you can make a benchmark that hasn’t been solved yet. You get you get them data, you make a benchmark. That’s like net novel net novel. And as long as that

As long as you can create that and and and have that exist then that it’s valuable to the labs ‘cause then their models will get better. Which means their A AI are will be able to code faster. Right? But it’s a very fast growing space for that reason. Um the value of public benchmarks is like questionable. Yeah.

When you see that the latest models are all improving on these public benchmarks, you’re like,
You uh you should always be skeptical. I mean if you look to the benchmarks you you you would learn that GPT 6 Astra is the best model. When Fable is obviously better. Like you use it, it’s obviously better. You can like know it right away. Right? And so the benchmarks don’t always tell you the full str full truth.
Yeah, I think it was interesting that GPT 6.1 Terra came out recently and like if you look on one benchmark they showed it’s like, well, you should only ever use GPT Terra because it’s cheaper and better than any of the other models.
Yeah, that’s hard. We we’ve been doing a lot of benchmarks lately and Sonnet five point five is actually very good as a model. We’ve we’ve been really liking it for our our specific set of tasks.
So you’ve written about r running parts of your company with agents or using agents to help him. You talked a little bit about it with hooking them up to MCP to be able to report bugs. What does that look like in other ways for you?

Yeah,

so um after open claw came out, um, I built a version of it for our company where somebody interacting with the agent gets their own like i personal instance to work with and and set up like monitoring in place to make sure the agent wasn’t doing anything bad. I set up a version of that and and deployed it across our company. And I looked at the stats of who uses it, and like our engineers didn’t use it at all. And our non-engineers

Mm.

like use like 24/7

U and so I don’t know if this is actually your question, but um I think the most valuable part of agents and the mo untapped part of agents still is like helping non technical people actually use them and be productive with them and and um

Accelerate. Yeah. Like our sales our sales team like c creates like slide decks before a sales call on the fly with agents based on like the customer profile and stuff, right? And and it’s it’s it’s great for the sales team because the the salesperson has to spend less time doing it. It’s great for the buyer because they get a personalized sales experience.

They say how do you support non-engineers in using agents? Is that something you have people focus on or

You just gotta I mean this

is our mission with Skyvern, right? We gotta help non technical people create aut automations. You just gotta give them code let them write code without knowing they’re writing code. Like that

Mm-hmm.

like you know, like if you can package coding and coding agents in a way where it’s basically writing code and doing what the person asked them to ask it to do and they don’t know what that’s happening, that’s like the perfect interface for non programmers. So like Facebook’s Muse product is basically that.

Right. Instinct is basically that. Um chat GPT dot dots I forget the what Claude released, the the name of a duet, I think. Claude Duet is basically that as well. It’s it’s like coding agents, but you don’t know it’s that actually a coding agent.

Do you think Meta will have an advantage in this area because they’re so consumer focused in the first place?

I

think that they’re determined. They’re determined. Like an anthropic is doing so well because they’re so determined and focused, right? Oh, I imagine that Facebook will do well as well. Because th they’re determined and focused. The other part of is like um um

It’s probable that c personal agents will be as big as the internet is today. And the internet’s like very big. Right? It’s used by everybody in the world. All comer commerce flows through it. Like a like a pretty high percentage of commerce flows through it. Like um a lot of jobs exist because of it. Like uh probably coding agents will be as big as that, so um creates a lot of incentive for Facebook to go after that space.

Yeah.

With personal agents. I mean, I guess some of this depends on who’s providing it, if it’s Facebook, Meta, it’s gonna be through their models, but do you see a future where people are y using personal agents and they’re all through Frontier Lab models, or this is moving to people running it on more open kind of models or or locally?

To me that’s a question like asking like do people care whether their servers run on AWS or Azure or G C P? Like the end user actually doesn’t care. I don’t know if that makes sense. Like the end end user doesn’t really care if it’s like they might care in the short term, uh because one party might have a better model than the other. But at the end of the day, they don’t really care where it’s running or how it works as long as they have access to it. Um

They’ll they’ll care about the brand. Like people care if they have an iPhone versus an Android phone. Right? They’ll care about the the user experience. They’ll care about whether it can do the task they ask it to do. But I don’t I don’t think they they’ll care about the model under the hood. The providers care though about the model under the hood because um it’s possible they’ll be they’ll be able to make a better agent if they have access to the data.

of the user interactions and so on. it’s possible they’ll they’ll be able to have higher margins if they c own the model. It might be able to serve larger scale of users for the same cost or something like that. I don’t know. Yeah.

Have you invested at Skyvern in training your own models or found it’s just not worth it with the continual improvement of uh

frontier models?

We we tried it a year and a half ago. We

tried a year and a half ago. Um and basically the model we trained back then became obsolete in like two months. So then we’re like, okay, like how much money do we need to raise to keep this up? And the answer was it’s not worth it. Like it was distracting and not worth it for us. Um I think that pendulum was switched swing back. Um before we open source I wrote this like thing.

I was like my pre-mortem on po open sourcing, which is like if this goes poorly, like what are what are some reasons or if this goes well like what would happen?

Mm-hmm.

And one of the things I wrote in there uh in twenty twenty four was that I think eventually like every company will um deploy their own LLM on prem. Um, either through a a partner like Azure or actually have their own like GPUs running with their own model. Um and and my thought my thought was basically in that world like open source tooling will matter the most.

‘Cause you won’t be tied to a specific model. You you’ll be able to pick whichever provider you need to pick. Um and so I I I do think we’ll train our own model in the future. That’s definitely a part of our roadmap and vision but it’ll be more like post-training than actual training. And it’ll be like specializing for our use case, not so much a general

Yeah, yeah, yeah.
LLM. And and the goal of it will be to deploy it on prem for our customers so that they can like our a lot of our customers work with like healthcare data so so they can keep
Yeah.

people’s private data.

To themselves.

Yeah, my sense is similar that at least at the moment, unless you’re fine tuning like a very small model for some specific use case or because you have a hard requirement that

does not make sense to be doing training ‘cause the cost is too high relative to the pace of the frontier. Like you’ll never recoup that cost relative to just switching to the next frontier model. Unless your model is super small and some specific use case that is Yeah. Yeah. Exactly.

Yeah, un unless you’re anthropic. I think I don’t know if you read their S1.

we’re we’re talking right after the S one was published the day

Yeah, yeah.
after. I don’t know if you read it, but they they basically haven’t spent any money in the company’s history.

Yeah.

It’s it’s and it’s interesting to uh to get all that that info after like all this time of what are they what are they spending all the money on? So yeah. Yeah.

They’re they’re not. It’s like Yeah.

It’s crazy. It’s actually like insane. It’s like I I don’t know. Never expected that. Like is there an AI bubble? I think the answer is decidedly no, based on that

Yeah.
information. It’s like definitely no. Yeah.
So you have you’re you’re you’re based out of Toronto. How has it been building an AI company outside of the Bay Area?

Okay, so uh the Bay Area is very AI forward, right? And um sometimes I wonder if they’re too AI forward. Like, I’ll tell you a funny story. Um last time I I visited SF, uh I go every month, so I go pretty often. And last time I visited people only thing people could talk about is like, oh my god, it’s all I’m gonna like are we even gonna have

like work in two years. Like what are we even doing? We’re gonna join the we’re all gonna be part of the permanent under class. And like a couple months before that I played bas I played basketball here and I worked I played with this guy who used to run an environmental consulting company. And he saw Claude and he’s like, Shit, this thing’s awesome.

Mm-hmm.

I can’t bill as many hours as I used to, I’m gonna sell it to a bigger European company. And he sold it to them.

And the European and and then he was trying to bring Claude into the European company. And he couldn’t do

Yeah.

it because they had to do they had to do like some water usage assessment of AI before they’re allowed to bring in AI like bring in the tool. And so and so I I feel like I’m between these two worlds where like I get to see the future in SF that everybody’s like I think over corrected for there. And then I get to like talk to normal normal people where like

Like, you they’re like, I got to use GitHub Copilot at work, I’m a coding beast. You know, it’s like it it’s like two different worlds. And most startups live between them, right? So if you are only subject to SF opinions, you’re gonna miss a ton of great ideas because you you think it’s already too late.

On the flip side, if you live in this world, you don’t know what’s coming, right? So living between them is probably uh is like an advantage I would say. And and I have I have the the the benefit of that advantage I guess in a way. But it’s like crazy to see both happening, if that makes sense.

Yeah, I think it it’s like the saying the future is here, it’s just not evenly distributed.
Yeah.
Always gotta be cognizant of where you are in that distribution and where

I I

I mean a startup like a startup’s literal job is to like find a market inefficiency and solve that, right? And like this is inefficiency. This is what it looks like in practice, right? It’s like the like our job as a startup is to make it evenly distributed or as close to that as possible. Um, if we’re successful. Like

And is your is your team all in Toronto or you distributed?

No, it’s all North America.

Yeah. It’s pretty well distributed. I mean people in SF and ev everywhere else. Yeah, we’re all remote. Yeah.

So you’re mostly you’re mostly remote. Okay.
Remote’s tricky, but it it can work with a certain kind of person.
Do you all meet up every so number of months or something like that?

Yeah, we we

we do like company off sites every couple months. Um I find remote can work, but it requires a lot of like personal discipline to make it work. And so when we hire people we we have to screen for that. Like we we’ve had, you know, interns in the past that just like didn’t work for a week. You know, they

Yeah.
just like go wouldn’t log on and like nobody like there’s no office to like police people.
Yeah.
to actually working, if that makes sense. And so it requires a lot of personal maturity to be effective in that role.
Yeah, I would always say about with working remote with people, like if people if you have a really good team, it doesn’t matter if you’re remote mostly, but if if your team is any l any less than that anyway, it’s gonna be better to be in person just because
Yeah.
there’s additional there’s that free accountability, but you you can work with it, but you have to
You you can make fewer you can make fewer mistakes, to be honest. Like
That’s uh yeah. Yeah.
the cost of communication is way higher. So you have to be good at it. You have to like
Yeah. Yeah.
you can’t wait a day to communicate ‘cause I could be fatal in a startup, so it’s definitely harder. Uh yeah.
Yeah. Yeah.
But the benefit is, you know, you get a lot more personal freedom. If you
Yeah.
don’t want to be bothered, you can just mute Slack and you can grind on stuff, right?
Yeah, yeah. Yeah.

Um

you can hire people of different talents who are what you wouldn’t otherwise be able to find

Yeah.
in a c in a particular location. yeah. oh
So somewhere between the San Fran and rest of the world thoughts of AI because what are you worried about? What the world looks like with AI or what what are some things that what’s what are what do you see as the good and the the not so good going forward?

Yeah,

so uh good good question. So there’s a couple things that are interesting here. But I I’m I’m part of a minor minority uh group called Sikhism and um I went to like a really interesting talk a few weeks ago about how minorities are affected by AI. Um so i in my culture it’s like not a very high percentage of population, like less than like one percent of people in the world come from my culture. And um even in even in India where we’re from, it’s like a very small minority. And a lot

of our history is written in like old manuscripts. Um it’s not digitized at all. It’s like in books that are like not translated and the translations are very have to be done very carefully because you know OCR isn’t perfect with like handwriting. And um the talk I went to talked a lot about how uh AI can be used um like AI is trained on digitized data.

And if the data it needs to train on is not digitized, like how can it ever know the truth about a particular c particular culture? And it’s relevant for any any minority in the world that’s not like part of the digital age, like, you know, the monks, our culture, really every every small culture in the world. And and the problem with AI is it’ll just hallucinate or tell you close to the truth, but maybe not the truth. And so

the point of the talk was really to like get people to start thinking about it and like organized to help really digitize those manuscripts so that they could be ingested by AI so like the future models can be trained on it. You can create benchmarks that detect detect that kind of stuff. But one of the things that I wor worry and think about a lot is um, you know, um

I I trust the answers AI gives me to a a lot of degree. If it sounds reasonable, I’ll just like believe believe it. And you know, that comes a lot of risk, especially if you’re not mindful and thoughtful about it. And so in the future it’s also possible for minorities to like fully like for the history to be wiped, for example. If

the d digit digitization is not done properly. Or it’s very possible for governments to like use LMs to like spread their truth about the world to other parties. Um and so that part about LLM is pretty scary and it’s stuff I wasn’t really thinking about before attending that talk. It’s like, you know like we just had a local election here. I used LLM to help collect the research for me. Like can I trust the output it gave me of the candidates that are running.

Right, how much how much effort am I gonna go through fact checking that? And if if they wanted to if you know anthropic wanted to influence a particular way, I’m not saying they do, but they could very easily be like this candidate means this and I’d be like, I like that, I’m gonna vote for that. And it might not be the truth, who knows? Yeah. It’s pretty scary.

Yeah, I think yeah, I mean I think that

side of things is like a big part of the question of in the future open versus closed source models because the closed source side, you don’t someone can control it and

a more nefarious way in theory, not that they can’t do that with the open source side, right? Like unless you go and check it for every fact, it’s uh you might not know, but I mean at least you you can hash the the weights, you know you know what you got.

Yeah. I don’t know. I I over time I’ve really appreciated like the Dario’s like CEO of Anthropics, like ramblings. You know? ‘Cause these we I think you started talking about the dangers of AI earlier than everybody else did and started sounding the alarms. And the more I think about it, I think the more likely it is that the that stuff is like actually worth thinking about and ver is very important. Um even if the method might not be yeah.

Yeah, there is uh

Book by uh Neal Stephenson the guy who wrote Snow Crash origin and I I don’t I don’t know if this is his most recent book now, but he had one uh called uh Fall. And uh in the AI future of that novel, basically

the elites have better filters for news, basically. That’s the thing that distinguishes them is to be able to distinguish what’s true from what’s not true.

But that’s the hard part about AI. It’s like the ben the benefits like do they outweigh the risks? That’s kind of the question. It’s like, you know, the elites today have their own executive assistants. They have their own private chefs. They have like somebody that schedules to buy stuff for them and like um schedules meetings for them, et cetera. And we can all have that now. You know? We can all have that, which is really cool. But so the b do do the benefits outweigh uh the risks? I don’t know. It’s kind of

Well, and even I mean, even if the information is out there today and not through an L L are people gonna go and find it otherwise? I think that’s like the alternative might be just having less information. So or

Well, I usually try to keep around an hour, so it’s probably I feel like that’s a good stopping point as well too. Uh so let me they can do

Sure. Yeah. uh Kind of kind of a heavy note to end on. That was the only downside that. Yeah.

Well, yeah, that’s that’s that is true. We could uh yeah, maybe let maybe let’s get hopeful last. So hopeful last. it’s like well

Also th that that’s definitely the darker side, I guess. What what are you most hopeful about?

I

I think I think I th I really do think the positive benefits are gonna outweigh the negatives. Like there will be negatives we’ll have to solve them. I think we will solve them. Like the cybersecurity stuff going on. But I do think the positives will like greatly outweigh it. I mean just the like just the idea that, um, you know, if I’m having a problem with my home server I can ask an agent to go fix it instead of fixing it myself, saved me hours. Like that’s awesome. Um, you know, I can

We’re having this conversation thanks to AI. You know, my agent booking it. It saved me hours. Like it’s like that’s amazing. And and I hope the conversation is also interesting back for you. So I think the net net net positive will be extremely good good for the world. And

Yeah, I yeah, no no it was it was
We just need to solve the neg we’ll just need to solve the negatives. Easy peasy.

great. It was it was great to have you on. Yeah, and I I agree. I mean I s I sometimes I mean I’ve said this before, but I mean I think a lot about

AI LMs, it’s it’s kind of like automobiles in a sense. Like, are there negative extern externalities with automobiles? Like, yes, but like how do most things how do you get your Amazon delivery or like would we have anything we have today if people couldn’t have moved more quickly before? Probably not. So

So it’s like any new technology, there’s going to be downsides to it, but we have we ha we we have the choice to try to use it in the best ways possible and like it it comes down to individuals and governments and all of us. Yeah.

Governments need to move fast. They need to move

fast. The cybersecurity stuff is so real. It’s like the internet might need to get shut down if everything gets hacked. But you know, it’s yeah, the governments need to be a little bit quicker here. And they’re not used to that ‘cause most times

Yeah, that’s why I’m like
are longer. Yeah.
Is that stressful running a s SaaS company these days?
I mean for a SAS company, like you get one data breach, right? And then you have to disclose that for every future prospect. It kind of kind of is killer. Like in a bad way. Um it kinda kills a company. So um it is very tough of mind. Like it is like we work with a lot of companies. We want to make sure they’re safe. Like, it’s definitely tough of mind. But even personally, like I I don’t want my bank account to get hacked. You know? Yeah.

Yeah, yeah, I know

it’s I tell I tell everyone, like, use get a password manager. It’s like enable two FA that’s not just your phone, like get a UB key. I I like UB keys so I don’t have to look at Google or authy in case as long as it supports it at least. So

Or we’ll go old school. I’ll get a checkbook and write checks for everything.

There you there

you though check like checks like appar in the US apparently like check theft by mail is a huge thing. actually like I had to so I had to f file like a tax or like a amended tax return one year and

I got to get a refund and they approved it, but like when you were filing an amend amended return, you could not get it electronically. The only way you could get it was to get a check. And actually someone stole the check out of the mail and cashed it. Yeah. and

wow. Wow. That’s crazy.
And so yeah, so I was like, Where’s my check coming back? And when I called the IRS they were like, Yeah, someone cashed it. So I hadn’t it was a whole thing of having to be like, Yeah, we never got it and then they sent us, of course, another check, like then they didn’t like there’s no other way to get it, but uh
man. It’s kind of or like the credit card. Like the credit card stuff always like surprised me ‘cause in the old days, right, you’d like give your credit card to the person taking your order and they’d like go to the back like swipe it and then come back, you know, and that’s like how all the theft happened. Like Yeah.

yeah. Yeah, I know exactly. Yeah, yeah. It’s kind of

it’s kind of crazy that uh but yeah, always trade offs in all things. Well.

Yep. Anyways,

thank thank you for time.

Yeah, well thank thanks for joining. It’s great to have you. And uh everyone you’ll be able to find, I’ll have all of the info on the s uh episode notes. So be sure to check on and we’ll catch you next time. Thanks.