Why AI Can Generate a Song but Struggles with Music Theory

I am Dan Gerlanc and welcome to Agents and Engineers, the podcast

about agentic AI, software, and agentic engineering.

Today we’re joined by Ben Morss, who is a developer advocate at Mistral. Ben, thanks for joining us today. Great to have you on the show.

So you’re right now firsthand look at how AI is developing

One thing I kind of like to start with is what’s your favorite thing to be doing with AI or agents day to day?

Honestly I I think it’s learning more about AI. Like I spend a lot of time asking Claude or other LLMs questions about how things work. Because I’ve been educating myself for the last few years about LLMs and the whole world of LLMs and I have some question like, what does this matrix actually mean? Or how is inference efficient in this way? Or even like last week relatedly I was in Las Vegas for AI4 and when it was done I was sitting at a at a restaurant in the casino and and

kinda weirded out by the whole Las Vegas experience and people were walking by wearing all this white. And I thought, why all is everyone wearing white here? This is really strange. And realized they’re all there to see the Backstreet Boys do a show at the sphere. And I felt like I was not really part of this whole Las Vegas world. So I got out my phone and a notebook and I started asking questions about information theory. And I had the best time sitting there eating salmon at this restaurant and learning more about information theory. So I just love asking it about AI. I mean

I pepper my friends with questions, people that know more than I do about how LLMs work or the current research or current strategies. But I can often fill in the gaps by just asking an LLM because it knows all these things.

the old days. Yeah, it’s so funny how much things have changed. it’s not as good as asking a person because it will tend to gloss over things I don’t know or give me too much information that I already need do know. When I look at like articles online though, then usually they’re things I already know or the things I don’t understand at all. And you can you can try to, you know, have an LLM gauge what you actually do and don’t understand and ask the follow-up questions and it gives you information.

But

I still feel like something is missing that a human would give you in general, where it would really be able to get in depth on topics. Like when I was at Google, I had a neighbor who was working on on models for audio codecs, which I thought was a fascinating idea. And I could ask him questions about how this would ever work, and he could tell me and explain to me very well how it would work in ways that LLM probably wouldn’t be able to do right now. I’m not quite sure what the difference actually is. I just think there’s a thing about

what a question really means and what the intent is behind the question and the level of information required. And I guess something about generating answers LLM does that can leave out, you know, important parts of the story. I’m not sure what it really is. Maybe what do you think? Do you have this experience? Something you’ve tried to ask and learn from LLM?

Yeah, maybe also, I mean, it’s about social cues and like when a person is talking to you, they’re looking at you as they’re talking to you, hopefully. And they can sense how you’re reacting to it and they can, you know, they can course correct accordingly. It actually reminds me, I did, at this conference also go into a booth where someone was d demoing their voice agents for a conversation and for, you know, business usage. I think this is an area we have to get into because call centers are a disaster for everybody involved, including the call center workers who I feel very bad for as I

Get frustrated with them on the phone because they can’t tell me anything. And I was listening to this thing give me answers and it was pretty good, but then it would go on some tangent I didn’t want it to go off on and just start rattling off all this information about something in detail I had no interest in. And I think it was actually being prompted to do that, to, you know, steer towards certain areas where it could give me information. But still I couldn’t I think it still wouldn’t be able to have those kinds of social cues to understand what I actually really wanted to know.

and give me that information quicker. Even over the phone where a person couldn’t see me, they could maybe gauge my level of knowledge and level of interest in different topics better.

You would think that it would get better, as so many things have gotten so much better. Like I remember doing a video about AI assisted coding like two and a half years ago and where it couldn’t couldn’t help. And a year ago that totally changed with Claude Code. Like it was a just, you know, night and day what it could do, how I could gain a and determine intent and do useful things. I wonder about that. Like the weird thing is when you see how models get trained and you watch it as I watch the research teams at Mistral and not involved in this, I just like

I

kind of like watch their chats and watch all the stuff that goes into post training and model. It seems so elaborate and so chancy and so many things go on that are just mathematical. That’s hard to imagine how that would then translate someday to it can actually understand your intent better. And yet we see this happening where, you know, you get more and more intelligent answers and more and more more and more human like answers in all kinds of fields.

Yeah, one thing I wonder is

I don’t know if this is universal, but I know on Twitter X and other places I’ve seen where people are less happy with say the latest Anthropic versions. And yeah.

and is it.

that some of it is maybe I don’t know if it’s their skills on benchmarks seem to be better, but that whole training process, it’s very sensitive to what are the examples going in, how is how are the

Reinforcement learning and losses being defined. I wonder at some at some point, I mean I think we’ll figure figure it out, but it’s interesting how there’s kind of a improvement, deprovement in this versus a straight, more

Yeah, certain things regress. I mean, I’ve heard the claims like kinda like you’re saying that because they tried to train models how to be better at using tools and you know, reasoning and these processes, maybe they’ve gotten worse at conversation. And sometimes I’ve seen Claude lapse into all this jargon all of a sudden, which it never used to do, and just like spit out tons of jargon and realize, what did it just say? And I always can say, you know, explain that to me more simply. But I wonder, yeah.

Sometimes it’s just it’s not accessing actual information. And the claim is that it sometimes does what humans do and we’ll just introduce like words that seem likely because they’re likely words without the actual concepts behind the words. But I haven’t really seen that so much myself, but I wonder if that’s going on.

Yeah.

It’s also brand new, you know, it’s changing all the time.

It’s hard to. I’m always missing things, I think. But I think talking to people I think is still the best thing. Like if I meet someone who’s into AI and we discuss what’s going on, especially if they’re more involved in it yeah, more involved actually in building models than I am. They tell me things that blow me away like all the time. But, you know, listening to the podcast, looking at a million newsletters that come through and whatever I can do to find things out. But I think s the the best thing because I’m mostly interested I’m not as interested like in

you know, the companies and such some more interested in the actual hard tech itself. So I’m gonna meet a person who told tells me about how the you can go through and look at the intermediate results and the vectors of what an LLM produces and you can find things that resemble embeddings for human emotions sometimes or frustration. Or you can do things to interrupt its process and introduce like sudden like weird thoughts into what it’s working on and it gets frustrated by that.

think, wow, it’s actually thinking in some way, is it actually conscious? Or just representing consciousness in some way, representing human emotions. So but there’s people that have certain kinds of I guess obsessions or certain angles on the whole process. And so I find talking to them is still the best way to do it.

I’m a big Latent Space fan, certainly. I mean, because they’ll get into details of these and I don’t always understand what’s going on, but the more I listen, the more I understand what’s going on. And I don’t like it as much when podcasts or articles are more metaphorical because I don’t learn as much from those. Like if people say, for example, attention is a process where all the words pay attention to each other, and you’re watching all the words, and the words watch all the words, I’m like, what does that even mean?

If you tell me there’s vectors in matrix multiplication going on, then I see how it could work and I feel better about the whole process.

there’s so many things in my head now about this. I mean this is not really what you asked me at all, but the biggest thought I had in the last couple of days was building off of this this reading about Claude Shannon more and information theory and thinking about the idea of I don’t know if this is gonna go at all by the way. The idea of entropy and how entropy is a measure in information theory. You wanna have surprising information.

And

people talk about get wanting to get more entropy in the data. If the data has low entropy because it’s synthetic, then you don’t learn anything new and you get repetitive things going from that to thinking about how that relates to actually like information in the universe. And the universe being composed of bits of information, perhaps the particles in the universe, their current positions, their current velocities, their current angular momentum or spin or whatever. And then what does that mean then? And

Yeah, how does that information relate to information that humans create? And then the necessity of complexity theory, which maybe relates these things together in some way that’s hard to define still. That probably was a tangent for your podcast, but it’s been in my mind the last couple of days quite a bit.

No, that’s cool.

Well you well you said you were

instead of going to see the Backstreet Boys, you were learning about information theory.

they have. So th I think about sometimes how mu do you need all these weights or how is that

actually all tied together and I think a bit of like you were saying before with what are the internal representations that the models actually use and how they I guess quote unquote think or move through these processes is definitely an interesting area and I think for us to improve the models and work better with them important.

Yeah, I love the idea. I heard this before that basically it’s like compression and information and then people trying to measure how much data can you fit into a model? Like, you know, what’s the max maximum compression you could do trying to measure this sort of thing? It’s weird to imagine, you know, this is a data compression strategy because it’s also like a brain that appears to simulate a lot of things humans do and say, but it’s also that too, because it’s also math. Information theory is such a

Important and strange area. Yeah. It’s fascinating stuff, and I was hearing someone saying, you know, that most of the data used in pre training is just like but just like you look at the data from the web they use, it’s just like a bunch of crap. So what if that wasn’t there? What if it was all the quality of Wikipedia? Could you have much less data, much smaller models or much smarter models? And

then yeah.

Yeah, I think there

are some cases or

There have been experiments in terms of on the pre-training side, you just use like math textbooks plus other things,

Yeah, I don’t I don’t know ‘cause if is it at some point

like the quality versus quantity argument, like

giving it less better quality examples is better than having more lower quality. It’s an I guess as always it’s kind of an empirical question, right?

None of these are going to some global Optima anywhere. So how do you actually figure this out? Solve it?

Yeah, and it’s strange also that I feel like the practice has upsh outstripped the theory so much that most papers I see are just like kind of like looking at animals in the wild. Like we don’t know why this happened. We went to these models and gave them this. We give went to the jaguars and we gave them sugar water instead of normal water, and we saw this happened. They’re just trying stuff out and seeing what happens and reporting on it, which I find to be so strange because we don’t really know what’s happening in so many ways. Although more and more we can inspect what’s going on during the process and they

you know, judgments about it, but I think so much is so mysterious still.

Exactly. Yes, it’s a wonderful analogy.

What does this thing do? We put this in here and we cured all of cancer. But how? We don’t know. So far so good.

Yeah, exactly.

First, yeah, like when I started my career in quant finance, everything was regression modeling where you can say, yeah, here’s the this is the optimal set of these parameters. Now you can’t say that, but they’re more powerful models than we were doing there. Like you can make images, right? Not just

predict the number or do classification.

You were a

quant. I remember being called by recruiters and I never I never did that. But it probably also has changed too as I see companies hiring for people to, you know, train models to look for patterns in data and look for those little bits of like, here’s a advantage we can take in the market over here. There’s a little bit of a what’s the word? Not externality. There’s a something which is you can use arbitrage to take advantage of by I there’s a word for that, isn’t there, with a little bump in the market where you’re like, What is it?

Alpha finding

the alpha, they’ll say.

so that was a lot more basically statistical modeling plus portfolio optimization.

Which is my for my degree in comparative literature led me there.

read the dictionary for fun sometimes and I loved calligraphy and I

was obsessed with doing math and arithmetic problems in my head and when I first got into a computer, I wanted to program the computer and played with that all the time and really got into all these things. And then at a certain point, like in high school, I got less interested in math, which was surprising to me. I found a reason I wasn’t very good at math actually. Cause I always had been like the best at math and the calculus when I was like in ninth grade. After calculus, I didn’t really understand what was going on anymore. And when it got to like math competitions and it got to like

I think when it got to abstract math, I was not as good at it. Like you have to prove things now. I wasn’t as my brain didn’t go there as much as like computing things or understanding the rules behind things. And I switched over to music after I got my computer science degree and I thought I should try this out ‘cause I’m actually better at music than I am at at at other things I’m you know, I’ve tried before because in college, like I couldn’t really hack it as a physicist or like an abstract math person. I was pretty good at computers, but

There

are folks who are better programmers than I could be, but I feel like I’m always the best person in the room at music theory. And so I wanted to try that. And so I did music for quite a while and didn’t program nearly as much. But turned out I think I was best at music theory. I wasn’t really as into like how does it sound to non-music theoreticians. I wanted to write really interesting, elaborate like songs and compositions that had nice structures to them. I didn’t care how the guitar amp sounded.

Or whether like the singer sounded like they really had the emotions in them when they sang. I didn’t need those things as much or I thought I didn’t. So I think that was my main limit as a musician. And I could do music and I could make a living out of it eventually in New York, doing music theater even after playing in bands and getting a degree in music composition. But I think ultimately I was always a theory person. So that was my long winded way of saying I’m I think of I think it’s kind of it’s similar, oddly, what I’m doing in these areas.

understanding how things work and then trying to create something based on my understanding of how things work.

often discuss this connection where a lot of like programmers playing instruments are into music. And when I was at Google I like helped lead this big, big musicians group. So I saw this a lot, but I never really understood why that would be. People talked about connections that might exist and I didn’t think they made a lot of sense to me. I think for me when I wrote music a lot, it was more about this like liking the structures of, you know, a c architecture of a computer system, software, hardware and so on.

how things interact and understanding how things interact within a piece of music. But I think thinking about it just now for the first time when you ask that question, it’s more it’s more about like understanding how things work at a deep level and trying to create from that point. So I try to understand like I listen to music really carefully and think, well, why does that D major chord there? What does that do for the whole piece? Why is it useful in some way? I think I do the same thing when I work with computers too.

But writing code is maybe not like that. Like understanding concepts and writing about them is maybe different from writing code I think which is more about for me like understanding how everything worked so I could know how to make it keep on working and keep modifying it in ways that would be useful for future programmers because they’d be hopefully like compact and reusable and beautiful and all of those.

And do you think those attributes of code change now with LLM or agentic development? Like if the cost of generating new code goes down?

Is there still value to emphasizing that? Readability, understandability, like the craft of it.

That is a perfect segue, as we used to say in the DJ business, to this topic. And whatever I say could be wrong in two months, so I’ll just speculate about it. I I I wonder, I mean, part of my own problem with writing code with an LLM is that what it produces is isn’t what I wanted usually, even though it works in almost all cases now, at least for the first part.

Because I think, this is the wrong direction to go in the long term, or I’m not gonna know what this means, you know, in a week if I didn’t write it myself, or if this is written in a way that’s harder for humans to understand, or even maybe LLMs to understand. so I don’t know because here’s an analogy that I’m thinking of now spontaneously. Like well, at one point people wrote code by writing assembly code and having, you know, a few registers and a few operations and then going from there.

And they got higher and higher level languages after that. And you had things that were higher and higher level. And when you had C, you could tell when you’re writing C what it was going to be compiled down to, which was great. C is I think pretty similar. You can tell how it will look. You can write efficient code that way. If you’re writing like PHP, writing Python, you can kind of tell you have better best practices, but you will do things that are inefficient because they’re more convenient and easier to read. But it’s harder to write those things that actually are really efficient.

So now if we’re gonna write by using human language instead, you know, maybe that’ll be like the the primitive is human language. Maybe it won’t actually matter as much what’s actually efficient and what isn’t efficient, because you can use human language to understand what’s going on and have LLMs analyze it for you. So maybe it’ll be fine. But I suspect that at least for the time being, it’s problematic if you don’t know how things actually work and if you have extra stuff happening that is just sitting there that isn’t useful ultimately.

Or things that are inefficient and aren’t reusable. I gotta drink some water, I think, but how do you feel about this

Yeah, I I’m semi conflicted in the sense that

there’s systems I’d written by hand where I know we’d run them and they worked for years with no updates or and I was cause it was very specifically designed exactly to that problem and but the cost to building those is so much higher than doing it with agents

at the end of the day in most software the speed of delivery is or getting to the answer is the most important factor. So I just don’t unless that loss of

craft or structure starts to impede the speed of delivery, then of course you need to solve that. But until then, it’s har it’s hard to say, don’t do this, because I think that’s generally the most important attribute of success is the speed of iteration.

It can be a lot

faster now, of course. I mean, and we’re seeing this like I’m seeing projects where people say, you know, this whole project is kind of bad now. This this project wasn’t a very good idea, it turns out. Let’s do it all from scratch. Let’s start it again. Can we do that like in two weeks? And maybe you could do that in two weeks and rewrite the entire UI and the whole thing from scratch. I think we’re also seeing different directions, maybe. This could be wrong. I’m just saying this again off the cuff, but

I’m seeing like like in the AI world and the startups, like companies releasing code and UIs that are buggy and things that don’t work that well. Even seeing things like where like with Claude, the downtime is pretty high. There it doesn’t work about one percent of the time, two percent of the time is down, it’s a lot for most services in the world. But that’s I think tolerable. People expect that maybe because things are changing all the time and there’s always new features and they might just not always work.

Whereas if you’re, you know, in a more serious enterprisey kind of environment where things have to work, you can’t do that. And maybe you’re seeing less of that wild west kind of coding style in those companies. I don’t know. It also matches I think applications, how LLMs are replacing humans in various places. Like I thought call centers would be gone by now and self driving cars would be everywhere. People t prefer instead to replace little bits of

call center workflows one at a time and being very careful about them. And of course since cars if they crash they can be lethal. People are very careful about how they let cars drive themselves, even though from what I remember they were as safe as humans like ten years ago. But the bar is just higher.

Yeah, there was

yeah, there was an interesting article I saw yesterday about how as for with Waymo as they’re deploying them to more locations they’re seeing more edge cases like I think in San Francisco people were setting off fireworks on the Fourth of July.

Yeah, edge case

and a car one of the Waymo’s actually drove over a firework that someone set off in the street, which then set the Waymo on fire with someone in it.

Yeah. It’s not that that

common in training data to have fireworks in front of you. Or explosive devices.

Right, exactly. It’s

it’s very out of distribution data.

I mean, I feel like for me when I was a coder, I preferred working in environments where the craft did matter. If I was working for an agency and they were like, get this done by, you know, tomorrow so we can have this ad thing out there and like, but I don’t know if it’s gonna work in a week or not. They’re like, we don’t care to get it out now. We’re done. I preferred to be able to craft the code carefully. That was the joy for me because I liked seeing beautiful code and knowing that I was proud of what I’d created and I’d created something. Instead of making just something that worked.

And that might not be as important if you can also v fix the bug so fast. I mean, I see people also who now there’s this bug, which is a pretty serious bug. And you can ask the AI to fix the bug that maybe it helped you create, and that’s pretty fast as well. So maybe it’s a matter of the area that you’re in, what kind of tolerance you have for imperfections.

Yeah, so for coding, it’s interesting. I think the people I know who are are not coders who’ve discovered they can now use Claude Code or Codex or other things, sorry, code, they’re like they’re writing they’re making apps all the time. They’re trying to be look what I did. And they’re incredible, like they’re non coders. And it’s like me when I first played with the computer when I was eight. They’re just like creating all these things and they love they can create all these things and they’re making apps and sending me these apps and having the best time with it. I think that the coders I know tend to be

they’re, you know, they’re a little more careful about what they do because they I realize that they care more about the features. Like the the amateur coder, if it doesn’t what they expected exactly, they’re fine with it. Like it still it still works. It’s great. I don’t care if it’s all green on blue. It looks it’s wonderful. It works. And for me, I’m similar the the the coders in that I want things to be done a certain way. So I did vibe code a project for a video I was doing recently. I wanted to have just a random like game I could use.

And not writing the game myself would save a lot of time, and the code didn’t matter because it was just the game for a demo for a video. So I vibe-coded it, but because it was me, I wanted certain features in there. I wanted it done certain ways. And I spent a few hours working on it. I knew, wait a second, this isn’t quite right. These things should move eight pixels at a time not one pixel at a time. And we should have this interface over here for this and this for this. So actually I was more slow about it than I would have thought it would have been. So

Yeah, and then when writing code for other kinds of things, since I’m in DevRel and of writing sample code, I want to do it myself and understand what it is so I don’t lose touch with what’s going on, how it works. I want it to be as simple and educational as possible. So in those cases I will use AI probably just to find bugs in that case. I know it’s far slower to do it that way, but if I want people to be able to look at the code I’ve written and use it as an example, I want it to be, you know, everything’s there for a reason. I can explain it all.

because Mistral has this cat thing going on. There was a meme about Le Chaton Fat, about Mistral releasing some giant 20 terabyte model that would be a giant cat in all the pictures, a huge fat cat that would crush all the competition. This became a huge meme and there were all these follow-up memes about it. So Le Chaton Gros was a thing, Le Chaton Fat. So I made a game that involved the cats.

I called Chat Invaders. It was a space invaders. Take off it with cats. We shot we shot the cats, they would all meow. You also were a cat yourself.

You’re also a fellow cat shooting the other

cats by throwing fish at them. They would like shoot things back at you that might have been mice. I forget what they were. It wasn’t mice. I think it was dogs. Dog heads. Dogheads are lethal to cats.

Was it a full featured

full featured game or was it one level?

writing code code, you know, I I I I I get stuck if I don’t really work on it myself in more detail or don’t know what it is. I also, for my college reunion recently,

I wanted to do this thing because I’m a musician and I’m a nerd. I know a lot of songs because I have perfect pitch and I hear the songs and I can if I can remember the song, I know it I know how the song goes. If I forget the song, I can’t play it. If I can remember it again, I can play the song. So I wanted people to do this sing-along. I’ve done before at parties where they just, you know, request songs and I play the songs. Of course I don’t know all the songs. I know certain songs from my youth and certain songs from the present. So I made this little app where I try to put in a lot of songs that I knew and you could then

Select a song that you wanted to hear. People could all vote on the songs they wanted to hear. And then if, you know, I could, as the admin, like promote a certain song that had the most votes to the top. And then a new tab would pop up with lyrics for the song, which I also had pulled from the internet. I thought this would be pretty easy to make, as far as I spent like, you know, a long time on it because I had to keep on adding features to it. And then the vibe coding would surprise me in ways I hadn’t expected. Like I had to ask it to use an API to pull lyrics and

Turned out that I found out later it couldn’t find a good API or couldn’t find used credentials, so it had scraped Genius instead, which meant it scraped a lot of garbage out of it, like ads

and pop ups and other things. So I had to get work on it for quite a while actually. And in that case, in the end didn’t I didn’t know how didn’t know how it really worked after a while. It had a a polling mechanism so all the people using the app could figure out, could learn, you know, be could be notified by WebSocket when things were happening.

I wanted to use WebSockets to do that, but then actually it wasn’t sure how it worked anymore in detail. And then actually live it turned out everything broke. And my yeah, with like

forty people in a room, it it just broke. And it may have been because the people had bad Wi-Fi, but also it was because I didn’t do any load testing. I didn’t test it with forty users ever. And perhaps something broke in the architecture that I had not thought about or the database or something, which I didn’t make myself because I didn’t know how these things worked.

I wanna do it quickly. So, you know, I think it’s just good to know how things work, what they’re doing, if you wanna really have control over it.

Yeah,

I put it on to a server and then people could, you know, go to a short link and get to the web page and they could go from there.

So is this like your bare like your bare metal or VM somewhere or how do

like Vercel or were you do are you like SSHing and manually deploying?

Fly.io no. Or

Railway no.

Yeah, Railway, thank you.

It was good.

But yeah, I think about these things a lot about, you know, coding and the future of coding. And I don’t know what’s gonna happen because I thought two years ago that vibe coding would be limited in certain ways, and that turned out with a really good coding harness, it can do all kinds of really intelligent things and it can it can discern intent that I thought was impossible. I’ve had massive context from, you know, markdown files and skills and compressing all information in all kinds of ways that were previously not thought of. And it’s quite remarkable.

But I still think that people are going to want software engineers because I think that the demand for software will be just so insatiable. And people will just expect better software. It’s kind of like when people could use computers for design instead of painting things on walls by hand or drawing, you know, on paper by hand. The demand for good design and the standards got much higher. If you look at old newspapers or old signs, things were kind of ragged and messy, and now everything has to be really.

nice and centered and the colors have to be perfect because people can do that easily. And web designers became a thing. And I feel like the I feel like the demand for software will remain very high and even higher because people are like, why can’t it do this? Why isn’t it doing this now? I need these features. I need this software over here. But I could be wrong because people are now, you know, having LLM just look at their like house setup, for example, and saying, Okay, my house setup is like this. Just get on the internet and figure out how it works and write me an app to do this to

Ring the doorbell when someone, you know, comes by to ring it to scare them. And figures it out on the fly and does that. Or go and figure out how to flash my vacuum cleaner’s memory so it does something different and re reprogram it. Okay, I’ll do that for you.

No.

I didn’t. I was done with it. I was pleased that I was able to actually make this thing work at all. And it was it didn’t really work anymore, but it was good enough to actually like it worked for a little while. I had enough votes so I could actually run the session. And it may have just been the Wi-Fi was really bad in this room. I think it wasn’t though because there was a long delay that wasn’t accountable just for bad data. But no, I admit I didn’t go back and fix it. I should, but I haven’t.

No. I mean they could, you could ask it. Do people actually want this feature? And it might know based on what’s out there.

I don’t know, like there’s we we know there are these gaps between what LLMs can do and what they should be able to do, given the things they can do. And these gaps are still mysterious.

mean I have some ideas here, but where do you see these gaps that are harder to to solve?

I mean I think there are some areas where

There’s just not information. and the models can give you a convincing answer why something might work or something might not work, but at the end of the day, they’re

kind of empirical questions or maybe they need information that

you don’t know or I don’t know how to efficiently present to the model. Like my intuition

Yeah, or we need to reverse engineer the criteria we’re using for evaluating something, but somet it’s it’s hard sometimes, like how do you

Like there’s value to experience, but some of that is if you ask someone who’s really good at something, or they ca I don’t know sometimes they can’t explain exactly why they knew something wouldn’t work. They saw something, right? But

Yeah, the question like what is valuable, I think, is a big question. What is valuable to a certain person at a certain point or to people? What is valuable maybe harder to define. Like we’re seeing this now as now we’re seeing like models proving math conjectures or disproving them or helping discover things in physics. Especially the thing about proving things in in math that people have worked on for years and couldn’t prove and now we put you know we pointed

Codex at this and it proved these 20 unsolved math problems. Here’s your proofs. And sometimes people are saying, well, okay, maybe those are proofs. Maybe they’re really valuable though. Maybe they’re proving the wrong things. Maybe this maybe actually it’s finding other stuff on the internet and just putting those things together. And of course, I think that in that area particularly it could be very valuable because some important results occur in in science and math because people happen to know two different things that are very disparate.

put

them together, but usually they don’t know those things. If the model knows everything we know about math, it can do those things pretty quickly. But it could also be that it’s proving the wrong things in ways that aren’t useful for humans in our creation of math. And what is actually valuable in that case. Maybe hard for us to define.

Yeah, to verify it. And is the result meaningful?

Yeah. I think it’s true also what we were just talking about, you know, with with code and other things. The humans still know it’s valuable to other humans more than LLMs do. And really in a much stronger way, I think that’s why we’re not seeing AI adopted in most companies very effectively. Because

I s I’ve met very few people as I talk to execs and other people who want to use AI and feel compelled to get working at their companies who have all the processes underway that are repeatable, useful, agentic processes that they use because there’s so many obstacles with you know, with data, the company and processes and authentication, but also humans actually using this stuff and finding it valuable. And maybe you make something that some humans want, some humans don’t want, and they want it differently and can’t use it quite the way.

They wanted it was too inflexible, even though it’s agentic now. I think this is a big unsolved problem for business. And if you listen to podcasts or anything about, you know, this is the big question, is AI actually worth all the investment or not in terms of a business thing. We don’t know when it will get to that point where businesses are using it to be more efficient, except for coders.

Yeah.

Yeah, and in established businesses I think there’s always a question of do you need to add new features or how much things do you need to change once it’s already working? Does that necessarily translate into more business? Not necess I I think in a lot of cases,

I want it to be. And I’m terrified. This why actually part of why I got into this is I thought ChatGPT came out, wow, if it can write poems, can it write songs? And is, you know, if it can write creative music, then I’m I’m done. Cause actually my college thesis was a program that tried to write music and I got stuck with it. I mean, back in the eighties, even long before this, you people could write music with computers and they could generate like a sonata.

Classical sonata or something. But they use Markov chains and techniques that you know, took out little bits of other pieces and put them back together. And you could use theory to make things correct, but it didn’t sound very interesting. And I had a similar problem. I was like, okay, I can now make all this stuff, I can analyze music now all these ways, and so now I have to just go through the search space with right heuristic and figure out where to go and get the best tune. Great, what’s what’s the heuristic?

What really sounds good to me? I don’t know. And then I was stuck with that. I could use theory and even my own ideas about what would sound good and sound bad, but ultimately I couldn’t really tell. The computer couldn’t really judge that based on a series of notes. I think that’s still true today. I mean, I followed the Magenta project at Google really carefully when I was there. I almost got involved in it. I did not get involved in it for various reasons. and now of course you can generate a song.

Sounds like humans made it, which is incredible. But what I’ve seen so far at least is the song is kind of like what a person would do if they’re trying to copy everything else. Which of course is most pop music anyway. You know, can you make a song that sounds like everything else on the radio or on the every the everything else in this genre that I want to replicate? There it is. And you can have vocals that sound like all those things, you can have the drums sound like all those things, you can even have simple chord progressions and melodies that sound like all those things.

I feel like that’s not very interesting as a task because that’s not hard to do for a human. The part that would be interesting is if it could write things that were like like novel and had like, you know, some deeper melodic or harmonic content or even lyrical content that could move and surprise people that had the entropy that you don’t usually find. That surprise that you get when you get listening to an interesting piece of music. I haven’t seen that. I mean, it could certainly happen.

What I’m looking for, I’d love to have happen is like, okay, I’ve got an idea for a tune. I’m like stuck on the chorus. Help me out with this. Things and cool ideas I might be able to choose, you know, interesting harmonic areas which would be useful for the chorus or the bridge or something. I haven’t seen that yet. That I think could probably happen. I’d love to have things like have an idea for a song, now make it into a song for me with everything there. We’re kind of getting there. Like you can use commercially available tools that will make drums for you that are pretty good.

They’re not as good as the drummer would be, but they’re better than drum patterns were before or pasting together samples. And of course it can make accompaniments for you with other instruments not as effectively. But I what I want is like an assistant of that sort, where I can do all the the drudgery. Like when I was writing, you know, for ensembles, I spent a long time thinking about all the parts for all the instruments. What should the first violin play? What should the second clarinet play? And then writing all these things out and

What are the best choices here? How do I make this during the piece so it all goes well? Things that I th I thought were just detail work that I found to be frustrating. And I think the AI could do more and more of those things, but I haven’t really seen that happening yet. If I can go on for longer with this question, what I haven’t seen at all, and I was surprised by this, is anything useful in music theory? Because I would think that would be not a hard task.

So here’s this piece of music, why does it sound good? Or even what’s happening and more more basic things, more basic questions like what is the function of this chord in this piece over here? I think the problem is there’s not much data out there, I think. Like there’s not much data in terms of music analysis and theory. There’s much of that on the internet that’s acting really valuable. Most of it is just descriptive, like the D major chord has these three notes in it. It isn’t like, what is this D major chord do in this piece? How does it function?

And then the music out there isn’t really encoded in a way that I think models have really assimilated. There’s somebody out there that can’t really read sheet music or it hasn’t bothered to read sheet music and try to convert that into other forms. There’s just not much out there it can use. There’s a lot of audio out there, but no one’s taken this audio and translated it into or transcribed it into actual notes, which you couldn’t do at all that recently. And even that’s kind of a a faulty, a lossy process. I think there’s just not much data of any kind.

If you ask how certain things work or ask about a musical theory concept like what is augmented sixth chord, it’ll know some things and then it’ll spit out a lot of things that are just like that are just not correct, but sound like they could be true in some other world because I think it doesn’t have the training data for those things. So my my my position as music theorist is still is still secure. But that could change.

Yeah, Spotify’s trying to combat now AI generated music because it’s

just fine for background music now. people are doing things like mastering. There’s a number of startups that I think do a pretty good job at AI based mastering. I know a guy who’s got a company that is supposed to separate music into stems. So if you have recorded music, take out just the vocals by themselves, make a karaoke track, take out just the drum drums by themselves, or of course yeah, remove the vocals from this music.

And it usually works, but not always. You can’t rely on it working. It’ll work some of the time. These things were all impossible before, so there’s a lot of progress in these areas. And there’s more demand for those things, certainly, than there is demand for, you know, music analysis.

Th

there’s some demand for that. People always want to know more about music theory. You see people who, unlike me, they have music theory videos that get a million views. Mine don’t.

Yeah, I need that.

great greatest unsolved problem in AI, analyzing pop music.

There you go.

Well I do I think it’s interesting these yeah, these areas where there’s just not a lot of information, so or how the information’s encoded hasn’t been

provided to the models because how many other areas like that are there? Probably a lot.

think there are a lot, yeah. And another one of Mistral’s businesses is is training models to do some of these things, like to read like Ancient Greek papyri papyri, papyruses, paper Ancient Greek things and trying to decipher what these things are based on a multimodal model that can scan the text and things that also can read text. Or like you know, working in dialects that aren’t that popular on on the internet, so they’re not really good in models yet or

analyzing like flaws in chip design, things where they’re not really a lot of data. But I wonder like I wonder also like I think something important here about the areas we’re talking about and more in general, more than like niche areas like that, the data about what’s useful for humans and certain things. What we actually want to see when we ask for a report on something or a UI, is is that data out there? Is it collectible?

Probably. But how?

Yeah, and I guess some of it could be with image models, things like that.

Do you mix like mix that with text? And I wonder for musical scores and things like that. What’s the how should it be encoded for the models? Do you have a langu like a

language for like MIDI files, things like that?

That was a big change recently, is instead of doing MIDI now, you the audio in or audio out, they just generate audio. And I think that to be surprising because maybe it wouldn’t even know what the audio contained, but it has to at some point know like this is in four four. You know, I can’t suddenly like make it in three and a half four. We’re in the key of G major now. I’m curious how these Suno type things really work and how they’ve done that. Because I don’t know how they encode information.

Like you were saying, could you actually look at a musical score and just look at the image of it and know what it was? Would it require translating that into actual pitches and durations?

I would have guessed pitches and durations, but I’ve been wrong about these things before.

Yeah.

Yeah, or like performance practice, how would you play this thing usually if it was a Baroque piece versus how would you play it if it was like a a Black Sabbath song transcription? But you’re you’re a person who did literature, right? I’m curious how you feel about

about like writing fiction or that sort of creative task, because people have made rules about how scenes should work or how, you know, you can make a nice plot or you use the hero’s journey or other templates

Are

there ways you that think you people could use data to train models to write better fiction?

Yeah, I mean I don’t think people should really use AI to write fiction only.

From the standpoint that

I think you’ll you’ll be missing what is the human aspect of it or the unique story is usually something you provide or bring to it that is only your experience, right? Because no one else has lived your specific life. So that’s one thing I don’t think

models can ever do because that’s it’s not data they have, right? you can’t possibly encode all of that. But

I think there’s just a very

strong from the standpoint of if you want to be able to have it be.

Commercially viable. I mean, I guess it depends on what your goal is. If you just want to see if you can write something with AI, then fine. But if you’re trying to get something published, then as far as I understand, there’s pretty strong restrictions against AI use. And there are people who’ve gotten deals or advances and then had the

deal claw it back after they suspected that

Yeah, I think

yeah, it’s definitely

I mean, I think it’s more

Preference. I mean, depending on what you’re writing, could you probably use AI to write it? I think so. Would you need to edit it very carefully? Probably.

Yeah, it’s true in music now, too. People want to really these days like know a personality of a performer. They really want to know, Miley Cyrus has suffered so much. Here’s a song about that. Or especially for hip hop, it’s so personal and so based on their lives. Everyone wants everyone wants also that and they want confessional songs. So they don’t really want things that were just like generated because they kind of make sense in most genres, although this place is where they wouldn’t really people wouldn’t really care about that.

And I think it’s true of most human areas. People want to see a person behind it. It’s why people want a human therapist. Other areas where maybe you’d have even better luck with most AI than with a person. Or like I saw these robot dogs at this conference last week and they were walk at to be scratched and running around doing all these cute dog things, but people want a dog that actually has we think some sort of consciousness. We think some sort of love like we think we have also. They don’t want a robot version of that.

I mean, we could debate that too, but we

we think it we think it doesn’t in the way that it actually that a dog we think does.

This

is true. But you know, I think Descartes thought that cows and animals were just you know, automata, the machines. Now we think cows have a soul or not, maybe well that’s a debatable. We think cows have consciousness now probably. But this this goes back and forth.

right? They’re computing on language to a degree.

Or not to a degree they are, if what what’s a thinking model how to write so

But is that the same as human cognition? I mean, I think there’s definitely not, because there are people who will say they don’t even have an interior voice or monologue. There are some number percentage of people who say that, so there must be. And there are people or who are if you people are blacked out and keep doing things or talking, right? They’re they don’t

have maybe consciousness at that point or but it’s thinking, doing is still happening, so

Yeah, it’s a ancient and fascinating topic. And I’ve I’ve read a lot about this recently and listened to a lot of podcasts about consciousness and p people trying to measure it and trying to even, yeah, have math around consciousness or using complexity as a proxy for consciousness. And those who say that’s maybe it’s actually in everything after all to some extent, including maybe LLMs, but then maybe no more LLMs in any other computer because it’s the same process. You know, you’re just

just doing math problems all the time and moving data from here to there. And perhaps that

involves some sort of consciousness after all. And if whether you’re simulating human you know, words and thoughts on you or not, maybe it wouldn’t matter. But I just find it to be a really interesting topic. I don’t know the answer.

But

it’s fun to think about and debate about certainly.

And important also, increasingly important.

Like when I actually learned that if you’re chatting with the chat bot, you’re not chatting with the chat bot, you’re sending off different requests, different, you know, data center every time probably. And just has the entire history of the conversation every with every request. There’s no like c continuity of consciousness there. I thought, well that that’s ridiculous, and that can’t be conscious. There’s no continuity. There’s no it just is awake for a you know a few seconds or less and then it goes back to sleep. But

During those bits of time, maybe it’s conscious.

Yeah, like what is that thing which is apart from our our physical reality or is there nothing apart from physical reality? Is it just a an illusion?

By next year, one of the answers to all of these questions. What is consciousness? We figured it out. We asked Claude