In this episode, we chat with Jacob Lauritzen, CTO of Legora, following the announcement of their $550M Series E at an $8.8B valuation a couple weeks ago. We talk about what it takes to build AI agents for lawyers and what other people building agents can learn from that work.
Legora started in a Stockholm basement with a simple product: ChatGPT, Swedish law, and a search tool. Jacob had already been through YC with his own startup when he helped Max Junestrand with the application for Leya (now Legora). He later joined as head of engineering and is now CTO.
Today the agent drafts and edits Word documents, reviews thousands of contracts at once, and coordinates work between lawyers, their clients, and other agents. The company passed $100 million in ARR in April, less than 18 months after launch, and now has more than 1,800 customers. It raised $550 million in March at a $5.55 billion valuation, then another $550 million in October at an $8.8 billion valuation. Its latest acquisition, Wexler, was its fifth just this year.
We asked Jacob how you evaluate agents when you can't see your customers' data, which tasks still need better models, how legal engineers and forward deployed engineers split the work, and why the billable hour as a business model is being upended.
What changed for the billable hour?
When Jacob first read Leya's YC application, he had one piece of feedback: people had tried building legal tech a million times, and it never worked. If lawyers bill by the hour, getting faster means earning less. Why would they buy a tool that does that?
What changed, he says, is the pressure on firms. In the US, routine work like small reviews is already shifting to flat project fees. Some firms do it for free to hold on to the client. The money moves to M&A, litigation, and time spent with clients.
We asked whether that means firms will stop needing paralegals and junior lawyers. Jacob says that's not what Legora is seeing. A good junior is worth more today than two years ago because they get so much more done, just as in engineering.
Evals without access to customer data
Most teams improve their agents by reading traces. Legora can't read any. Almost everything users upload is confidential, and much of it belongs to the firm's clients, not the firm.
So the evals team has to recreate the kinds of work customers do. Three engineers and three lawyers build RL environments to simulate the work their customers do. One example is a 10,000-file data room where the agent runs due diligence and has to produce a report that catches the right red flags. The team plots the results on a heat map by practice area and task. A new model often gets better at some kinds of legal work and worse at others.
The team draws on a pool of about 100 legal engineers: lawyers who spent three or four years in big law, often with a CS degree. In production, Legora classifies failures into 10 to 20 known failure modes and runs cheap LLM-based checks behind feature flags.
Those restrictions also determine which infrastructure Legora can use. Everything runs on Azure because when Legora started, law firms wouldn't accept AWS or GCP, and Microsoft was already an approved vendor through 365.
Where smarter models still matter
Legora resisted a model picker for a long time before giving in. Jacob's view is that a lawyer shouldn't have to decide whether a contract is a Sonnet-high or an Opus-mid task. It's Legora's job to find the best available tradeoff between quality, cost, latency, governance, and regional availability.
He estimates that models already handle about 60% of legal work really well. For those tasks it's a cost and latency problem now. Complex analysis and editing across million-token contracts still need more intelligence.
Legora has started post-training open-weight models for narrow tasks like legal research, with no plans to train its own frontier model. For drafting, Jacob finds Fable and Opus too verbose. Astra is more concise. To address hallucinations, everything the agent writes gets a citation that highlights a bounding box in the source PDF, multiple layers down.
A few building blocks, many products
Six months into the job, Jacob showed the team a slide of the 300 Spartans and declared that Legora would stay at 20 engineers forever. It now has 100, and he says that still feels too small. His job now is to build a system where the team can ship just a few things and customers feel like they're getting a lot.
The team builds around a few primitives: projects that hold files, the agent, Tabular Review, and lists. Lists are a flexible Notion-style database with triggers and automations. Agents can create and read lists, and changes to a list can start an agent. Intake is a Slack bot that turns a request into a ticket on a list.
Tabular Review is the one the industry copied, down to the name. Each row is a document, each column is a question. Drop in 10,000 employment contracts and ask for the termination period, the IP assignment clause, and the severance terms. It supports up to a million cells, but starts with just 10 rows so you can fix your prompts before running the rest.
Legal engineers configure new products on these primitives without writing code. Even Wexler's fact intelligence product runs on the same building blocks.
Legal engineers first, forward deployed engineers second
Legora hired its first forward deployed engineer about nine months ago in New York: Di Qi, a two-time YC founder who worked on search at Meta. Legal engineers configure the product the way a power user would and don't write code. Forward deployed engineers write code into production, build integrations into a firm's CRM or document management system, and push what they learn back into the core product. They still don't touch client data.
When a big, engaged client signs, Legora assigns legal engineers to work with them and adds an FDE if the project needs one. Within a year, Jacob expects to have a rotating group of FDEs who sit with a client for a couple of months, build what they need, and move on.
Where this goes
In two years, Jacob thinks big enterprises will use Legora through tools like Claude Cowork, while law firms work inside Legora directly. He doesn't expect the labs to cut Legora off the way they've done to coding companies. For a $400 million M&A deal, he thinks people will still want Legora.
Show notes
Timestamps
• 0:00 Cold open
• 0:40 Meet Jacob Lauritzen, CTO of Legora
• 1:05 Starting out as underdogs in a Stockholm basement
• 1:54 How Legora grew from ChatGPT with Swedish law into a legal agent
• 4:12 The switch from assistant to agent
• 5:23 Which models Legora uses and why it added a model picker
• 7:48 Are today’s models good enough for lawyers?
• 8:20 What agents couldn’t do nine months ago
• 9:18 Testing the agent in simulated worlds and a 10,000-file data room
• 10:07 The evals team and Legora’s 100 legal engineers
• 11:03 When evals didn’t seem worth it and one release took citations from 70% to 99%
• 12:18 What legal practice could look like in five to ten years
• 14:18 Will the law firm partnership model survive?
• 16:40 What happens to paralegals and junior lawyers?
• 18:00 The billable hour and why earlier legal tech fell short
• 19:39 Where the money goes now
• 22:44 How Legora used its own product for acquisition diligence
• 23:59 How Swedish and American law firms differ
• 26:12 Product metrics when you can’t read the traces
• 27:41 Paraphrasing traces and narrowing failures down to 10 or 20 types
• 28:42 Why data privacy led Legora to run everything on Azure
• 30:04 Open-weight models and model arbitrage
• 30:55 Transcription models and why Legora doesn’t use ElevenLabs
• 33:33 The agent, Tabular Review, projects, and the Word add-in
• 34:24 How a lawyer uses Tabular Review
• 35:51 Why a spreadsheet works and what you learn from the first 10 rows
• 37:53 How the legal intake bot works in Slack
• 39:11 How agents create lists and lists start agents
• 40:15 Building products on those same pieces and acquiring Wexler
• 42:37 What forward-deployed engineers do at Legora
• 43:52 How legal engineers and forward-deployed engineers onboard a big client
• 46:43 What’s next for the forward-deployed team over the coming year
• 47:24 Why Legora resisted building APIs
• 49:26 Can the agent be the API?
• 50:12 The 300 Spartans slide and the plan to stay at 20 engineers
• 51:32 The bets nobody asked Legora to make
• 53:05 Post-training open-weight models for legal research
• 54:05 Search, retrieval, and late interaction
• 54:51 What makes a good legal agent harness and why drafting depends on search
• 56:25 Which model writes the best draft?
• 57:03 Hallucinations and Legora’s citation system
• 57:41 Do lawyers check the citations?
• 58:19 MapReduce or one big context window for a million-token contract?
• 1:00:16 Getting anonymized examples and the idea of buying a law firm for its data
• 1:01:53 Why Jacob advises against default embeddings
• 1:02:44 Will lawyers work in Legora or through Claude?
• 1:03:26 Will the model labs cut Legora off?
• 1:04:24 Law schools and scholarships for computer science majors
• 1:05:28 Hiring in New York, Stockholm, and London
• 1:06:09 Why most work eventually becomes legal work
Transcript
Automatically transcribed from the revised edit; names and wording may contain errors.
0:00 People were asking us about Mythos and everything, even though I think realistically a lot of legal work does not require Mythos level intelligence. Paralegals, gone. Yep. Junior lawyers, gone. I don't need them. That's actually not what we're seeing, and we had one problem where, like, it wouldn't cite properly, and so we were, like, next model release solved. It went from, like, 70% to, like, 99. It's just, like, we did nothing and, like, great. At least you're doing a, you know, $400,000,000 M and A transaction. You're gonna want Legora. We're just gonna be 20 engineers for the rest of the company's history, and, like, we're gonna beat everyone who's much bigger. We're a 100 engineers now, it feels like we're way too small. But it's something that I'm trying with is, like, how much of our product can just be the agent versus all the other stuff. Okay. We're here in the studio with special guests, Jacob Lauritzen, CTO of Legora. Did I get it right? Lauritzen? That's fine. Yeah. That's okay. I'm not Spanish. How... It's Danish. Yeah. Exactly. Does it have a special history? It just... No. Is like son son of. Son of. It's like you have son and... Yeah. So son of Lauritz. That's my name. Nice. We've crossed paths here and there. You you you spoke at AIE a few times, and, obviously, Legora is one of the fastest growing legal tech in history. Yep. And you said something at the start in a pre show that I thought was really interesting. You said that your origin is very European Mhmm. Even though, obviously, you're expanding in New York and SF. What feels different about your culture? I think we are, just at the core, probably very humble about what we're doing because we've always been the underdog from the very, beginning. No one believed the small startup in a basement in Stockholm was gonna, you know, outcompete this huge behemoth that was, like, way ahead of us in the US. So I think we've always been underdogs, so we've always been very humble. We've always sort of had to fight for our existence. I think it's something we've had to learn as we've also become more and more... As we've entered the US and been in the US a lot more is to be a little bit more bold and, like, tell the world how great we are, because it's... It doesn't come naturally to, I think, some of us.
1:30 I kinda wanna start by going over your guys' product, because I feel like with a lot of these, like, legal products, you know, they're kinda like under wraps, like, you know Yeah. You're not supposed to like show it. Yeah, if you could just talk about, like, what does the product look like, and maybe, like, how did you guys get there? Like like, what have been the product iterations that got you to where you are today? I think that would be really interesting. Yeah. Absolutely. The way everything started was ChatGPT with Swedish laws Okay. Search tool in it. That was literally the entire everything. And so that's where we started. We started with legal research, which is like a huge market, basically, in the world where lawyers spend a lot of time figuring out what are previous cases that look like this, what's the legal precedent, like, is this, you know, what have previous people done in these situations? That's extremely important. And so that has always been part of our product. Basically, we started from, like, an an agent, you could ask questions and it would query laws. That was a very simple beginning. And then from that, it's expanded. And so it's expanded in, like, what can the agent do? It it will now, you know, draft a lot of stuff really well. It will edit Word files. It will, you know, move your files away, organize everything, run really complex analyses, and we've added just a bunch more features to it. So one of our features is called tabular view, which is now a name that's being used in industry, and we're very happy to see Claude for Legal using our terminology for what they're building. We gotta talk about that. We gotta talk about that. Which is essentially like a way to just bulk extract and bulk review many, many, many agreements, which you do in due diligence and when you raise money or whatever.
3:00 So that's part of it. It's like we help lawyers just do their work much more efficiently, much much better. And then outside of just doing the work, we're also now helping orchestrate the work a lot more than we were just even a year ago. So we have different sort of we have a a thing called lists, which is sort of like linear for us. And we have intake, which is a way to collaborate with clients, and we have portal, which is also sort of a way to collaborate, you know, lawyer to lawyer, client to lawyer, lawyer to agent, agent to agent, maybe. We're we're spending a lot of time there because there's just... Frankly, the way that you work with a lawyer ten years ago or maybe even today is is really painful, and there's loads of room to do that in a much better way, to be honest. And so those are, like, I think the two things. It's it's doing the work and it's also, like, orchestrating the work. When did you guys start doing or building the agent? So, I mean, we've all... We basically had this switch from the assistant to the agent Yeah. We call it. Right? I guess that the agent... I don't know. I wanna say a year ago, probably. Yeah. So this is a rough timeline for people who are understanding YC 2023. Yes. I know you are you are YC founder as '24.
4:30 YC twenty four. Yeah. I was in YC '23. Yeah. Legora was '24. Okay. And then... So so there is... This is, like, a little bit pre a... Pre, like, agents really taking off. Exactly. And then, like, using models obviously changed everything. Yes. I mean, you know, in one in one sense today, you know, we had a preview of the products. It does look... Well, preview. We had a private thing. Well, we will put up a video on on the YouTube. It's mostly like ChatGPT for lawyers. Yep. Right? Like, very, like, very fair to say that, but then the sidebar has a lot of nice custom tools like the tabular view, like the intake, which I really liked, and a bunch of other Exactly. That's exactly what it is. It's it's coworker or ChatGPT, just like really, really good for legal work. And the... Yeah. The the the workflow for, like, either, like, in house corporate lawyers, like, people, like, looking for a coding regulations Yeah. All these things. And we can talk about more of the the products there. Yeah. What models are you guys using underneath? Like, can you talk about that? I mean, we're using a bunch of different ones depending on tasks. Right? That's And then that's not available to the lawyer to see. You guys Or the agent? Yeah. Well, we've been back and forth on this. Yeah. Just like everyone else, I think. In the agent, there's a model selector for the primary model, but there are certain tools that we use different models in, and there's, like, tabular view, and, like, we we try... I think we found it most effective to subdivide the problem and and and route on, like, as we subdivide the task. But we do right now have a a picker in the agent. Mhmm. I'm not sure if that's gonna stay there always, to be honest. So... Yeah.
6:00 Yeah. I mean, so I bring a little bit of context to this because I'm an adviser for Cognition. Yep. And when I joined... You know, part of my my thesis for what an agent lab is, an agent lab is, you know, ideally a domain specific GPT wrapper. Yep. Right? Like, we we we run the... We are the domain experts. We will pay attention to you. We'll we'll serve your needs way, way more than the Frontier Labs, and we'll give you the best of the frontier labs according to our judgment. Yeah. So our job is to decide for you because you're not... Your job is not to Exactly. We'll always keep you on the Pareto frontier Yes. And make sure that it's it's great all the time, and you don't have to dive into all the details. Because, like, of the things that I've... Exactly to this point, if you go to a lawyer and you ask them, oh, you're gonna draft this contract based on this precedent, do you think that's a sonnet high or an opus mid task? Actually, they they won't know. Just give me the I've this model. Just, like, make sure it works. Our hourly rate is thousand dollars an hour. Exactly. Just like Make it work. So, yes, I originally came in with that. Was like, oh, yep. This is this is what Devin does differently. And now we have five things in the model picker, including Fable and Opus Yep. Astra. Yes. And people wanna experiment with the fancy new model. Yep. Do you... Is that the same? I I think so. I think it's prob... It's it's definitely way less in legal than it is in Coding? Engineering. Yeah. But there are definitely people that would just want table one right away, and, you know, people were asking us about Mythos and everything Yeah. Yeah. Even though I think, realistically, a lot of legal work does not require Mythos level intelligence. Yeah. There is
7:30 So exactly right. Like, I think, basically, you're like, okay. We'll give you the option. You you ask for this Exactly. But you're gonna run down your credits, like, 10 x faster than them. Yeah. I would have loved... We we fought introducing it for a very long time, and then in the end, we we just kinda caved in. We're like, okay. If you want this level of control Yes. You know, then we'll give it to you. Yeah. So you think the models are already, like, as good as lawyers need? Like, there's no need for them to get better? It's a... I think it's a power distribution. So, like, there's probably, like, 60% of the work Yeah. We can do really well right now, and you know, from now on, it's like a cost and latency thing. But then there's the rest of the the thing where, especially for, like, very complex analysis, we we still need more intelligence. And then I think there's also just the work expanse kind of. It's like that's the work that we have right now, but also the work with Fort Lauderdale, thus probably expands with intelligence Yeah. I imagine. Early on, like, when you guys first started, I guess, transitioning to the agents Mhmm. Or building the agents, like, what were... Like, what models were you guys using? I guess it would have been like Opus four six or something? That would have been... Yeah. Sonnet four six, Opus four six is probably right. What were the agents, like, not good at at that point? That they're much better at now even though it's only a little, like, nine months later? Sub agent delegation was one of them that we've gotten working really well now for, like, fan out type retrieval queries. Works extremely well for us now. That was difficult before. We basically track... So we have a team that's just doing evals, and so we track all of the model's performance in different practice areas, which is like subgroups of of legal, and different tasks across all of those. So we basically have, like, a big heat map, and we're like, and then we can see them move, and it's like, usually, they improve most stuff, but there's actually a lot of model releases where they improve some and and progress in others. Okay. And that actually happens a lot more than you'd think.
9:00 Like, what are some of those tasks? Like, are you familiar with how you guys are running a lot of the evals? What are those different sub segments, I guess, that you look at? Yeah. So, like, there's different just like framing stuff. So we have different, basically, like, worlds, like RL environments, essentially, which is like, here's a... Because because lawyers work in projects always, more or less, so it's like, here's 10,000 file data room, you're doing your due diligence, in the end, a GRIP report looks like this, like, it contains all these red flags and, like, it doesn't contain whatever weird stuff and, like, we want it to be concise, etcetera, etcetera. That's kind of how we've built it. So there's different tasks, which is, part of it is drafting a report. Like, does the draft look good? Does it contain all the right stuff? Is the wording sound? Is the analysis sound? There's also the part of finding the red flags, like the analysis before that, so we can kinda like subdivide it a little bit, And that's how we run the evals. Interesting. Wait. What does your evals team look like? Like, do you hire lawyers to, like, do that? So, yes, it's a mix. We have engineers, and then we have lawyers, and there's one team. So there's, like, three engineers and, like, three lawyers right now, and then we have a pool of what we call legal engineers, which is people that have worked in big law for like three or four years. Typically, maybe they have a CS degree and like they're very tech forward, and then they come to us and... Because they wanna like help change the industry.
10:30 Are they IT in big law? Or No. They're lawyers in big law. They're lawyers in big law. With CS degrees. Yes. My god. It's a Venn diagram that... Yeah, exactly. That's like our hardest person to hire for, but they're also such a force multiplier because we can pull on them for the eval stuff and for the product development, which is like you have a pool of a 100 experts that use the product every day and go and talk to clients every day, and also, like, they live this and have lived this, and we just pull on them to do eval stuff, and just like help us shape the product, and it's such a such an advantage. And curious, like, did you guys start that team? Evals grew out of the assistant agent team at some point. I think there was a point up until recently where it was like, honestly, it's just not really worth it, because all models just got better. So it was like, do we want to spend a lot of our time doing really complex evals, or do we just want to build... Because there was so much to be gained from building a better harness and sort of leaning into everything's gonna improve, and everything kind of just improved. Mhmm. But, yeah, like, every two, three months, you get a different model. Yeah. Exactly. Mean, we pass it through. Right? Exactly. I mean, we had one problem where, like, it wouldn't cite properly, and so it would be difficult to follow, and it was like, next model release solved, it went from, 70% to, like, 99. It's just like, we did nothing and, like, great. Yeah. Maybe the the the real work is, you know, keeping all the data private, making it taggable, sortable, adding new features like the the tabular view. Is that the, you know You mean over and above ChatGPT? Yes. But obviously, now you're also doing evals. I assume your your use of open models is also increasing. We can talk about that. So, yeah, I I don't know what I'm saying here apart from, like, you know, what is the job? The job is maybe not... In the early days, it was not so much to do do evals because it was mostly exposed model capability to lawyers.
12:00 Yep. The huge part of it is just figuring out what does the legal profession look like in five years, ten years with all of this stuff. Like, coding is maybe a year or two ahead, that we can kinda look to that a little bit, but, like, what are the primitives? How do people work? How do you collaborate if it's not a 200 thread email with, like, v five word files back and forth? Like, what does that look like? And that's, like, not even necessarily purely AI. That's just, like, build a much better product experience Yeah. And make sure you do it with, like, audit logs and data retention and all of those sort of really important things in regulated industry. And then there's the the AI part, which is, like, make the work faster and better and proactive agents that react to stuff when it comes in so you don't have to, you know, kick off everything. So I think a lot of the work is just figuring out what do we think it's gonna look like in five years. If, you know, model capabilities keep increasing, if price keeps coming down, what could we do? Can you log in and it's like command center where it's just like, you need to react to these five things that the agents have been doing lately and, like, review and check and improve the system? Maybe that's where we're going or it is where we're going. Yeah. Yeah. I was gonna ask, like, could you go into that in a little more depth? Like, yeah, what what is the, like, law profession gonna look like in a few years?
13:30 I should preface this by saying I don't know. I mean, who knows, really? But I think we can look to coding a lot. I think all lawyers are gonna be way more productive than they were before. They're gonna be able to do a lot more stuff at a higher quality for lower cost. I think the profession itself, the work itself, is gonna improve a lot. It's gonna be a lot more... I think lawyers are gonna spend more of their time doing the stuff that they became lawyers to do, you know, figuring out what's the right risk profile, what's manageable risk, what's like the... All of their judgment, strategic judgment, and like working with clients, they're gonna do that a lot more and much less of the, like, trolling through big documents to like find the smoking gun or like do all this menial super annoying work that a lot of lawyers have been doing. What do you think is gonna happen in the partnership model? Like, do you think that stays or, like, what happens? I gave this... I actually gave this a lot of thought just, like, last week. Yeah. Just the the partnership model... I don't wanna get into too much of the details, but, like, the partnership model kind of exists because the only type of capital that these companies have is human capital. So, like, it's not like, you know, us or like a software engineering firm or like a manufacturing or whatever. You you have physical assets or like some sort of intangible assets that that you accrue over time and and they pay dividends to the shareholders. But if you work in an industry like professional services, and it's the same for, like, consultancies and and whatever else, all of your value is in your people, and you can't really own human capital. You know, you you can't... You know, slavery is abolished. Can't own people, but you can rent them. You can pay them money. And so partnerships essentially is a structure where you make the human capital the owners of the company so that they stay. Yeah. Will that change? Like, I don't... It doesn't have to, but I think what we really want to do with the lawyers, we actually want lawyers to be able... Law firms to be able to build some sort of intangible capital, some durable capital, and, like, you can build a system that learns of how you work.
15:00 A lot of firms right now think about, like, well, there's the ex law firm way of doing things, and we can codify that, I think, and we can build a system, and whether that's like a self improving knowledge graph or whatever it is, and like precedent files and documents, where there's like a way of working that produces a really good output, and that can be their IP. So it's like a way to codify the human expertise and judgment into a system. That I think is... Now I'm just rambling completely, but like, that's something that we want to help empower. And does that change the partnership model then? Because incentives are different. In the partnership model, you just gotta bring more clients, and you don't wanna take away time from bringing in clients to codify your information, basically. Mhmm. Because, like, you get rewarded from, you know, eat what you kill or make this luck step, whatever. That needs to change a little. We need to figure out an incentive structure where partnerships... Partners are incentivized to codify things for... To build an asset for the company, but right now, there aren't because as soon as they leave the partnership, they don't get any dividends from the assets that they built.
16:30 Yeah. So there's like a... Yep. I think the the the whole business model is is... We'll we'll see. Yeah. Yeah. It's all it's all in question right now. Yeah. My my actual other take... You know, so on the partnership level, that's like the top end of the Mhmm. The industry. Actually, my take is more on the bottom end. Right? Yep. What happened there? Paralegal's gone. Yep. Right? Like, junior lawyer's gone. I don't need them. Like, it's That's actually not what we're seeing. So, yeah, it is... I think it's... I know. Would you would you say that we're seeing that in engineering? Yeah. So junior engineers are really kind of cooked. You kinda have to come out of... Drop out of university fully baked or else you're You're not gonna make it. I feel like I'm not... I'm being a bit dramatic for for a comedic purpose. But, like, yeah, I know a lot of lawyers. They started as paralegal. They go to law school, all these things. Right? And now they don't... They're not needed. They're they're seriously just not needed. Yep. Yep. Yeah. It is... It's interesting what happens to the partnership model, like the pyramid. I think generally, a lot of lawyers, you still need your... You need the next generation. And at least for engineering, the way I think about it is you have a good engineer today, they're worth a lot more even than they were two years ago because they can produce so much more stuff, like, they're good. And I think it's the same thing in legal, where they can actually do a lot of really good work if they're good. So I think it's more that really good lawyers or junior lawyers are worth a lot more. Yeah. It's always been the case. Right? Yeah. Seems like even more But it's, yeah, it's polarizing even more so.
18:00 Yeah. I'm actually curious about that. So, like, the whole, like, law business is basically, like, charging per hour. Mhmm. So Yep. You know, if each project now can be done Yeah. A lot faster is it going to be, like, way cheaper? Like This was actually... So fun story. I was I was reading Max's YC application and, like, helping him getting into YC. Leya, it was called back then, is YC. And this was my one thing. I was like And and curious, Leya is a Danish thing for something? No. Swedish. Swedish? No. Also, it means nothing. There's this Okay. Okay. There's this really funny story in... I think it was a Spanish newspaper that wrote about us in the early days, Leya, and then they they explained that the abbreviation was legally efficient Yoda assistant. Why Yoda? Yeah. I don't know. So I was like, why why is it See, hallucinations are not just from LLMs. No. I think journalists hallucinate. But, no, it doesn't mean anything. So he wrote the application. He he wrote... Yeah. And my feedback to him was, you know, people have tried this a million times. Legal tech has never been a thing because, like, there's no incentive for lawyers to to... Why would they be more efficient? They earn less money. Like, it doesn't make any sense. But I think the reason that lawyers do wanna adopt this is that they understand the gravity of the situation. They know that, like, the business model and everything is under pressure, and it's like they need to change because the competitive pressure is so high. The billing model itself, I think, is also gonna change. We've... We're already seeing that, particularly in the US, that a bunch of stuff is actually being priced on, like, a project fee and not on an hours fee, particularly these, like, sort of lower end stuff on the spectrum, like, whatever it is, like, reviews and whatever. Some of it, honestly, is being done for free, and some of it is just being priced on a flat rate. So where is
19:30 the money being made then? Is it... Because In the strategic work. Yeah. I feel like a lot of what you guys talk about is M and A specifically, like, that's where all the money is a very easy example. Oh, okay. Okay. Litigation is also. The big work still is there, and it's very valuable. And so they'll they'll spend... Lawyers will spend more of their time there. Client relationships, also super important. Like a CRM? No. Just like spending more time with the client. Like, a lot of law firms will, or historically have basically done stuff for free just to have their relationship with the client. Yeah. Yeah. And so now they can do that, but they don't have to lose a lot of money on doing it. I thought I thought I just... I have to pay the lawyer money just to be my friend. Like, in this Yeah. Isn't that the retainer thing? Yeah. Yeah. The... That's You you told me I can do it for free? Like Yeah. You just have to find the right lawyers. Yeah. It's alright. Cool. Or Latent Space has to grow much bigger. I don't know. Yeah. Yeah. We're trying. You know? Yeah. So... Okay. So so so you're saying, you know, obviously, the the strategic work and all these high value add is still there, but our legal fees are not going down. But you also said paralegals are not affected or, like, they're they're just shifting. And and this is something that we also did with... Worry about with Cognition. Like Yep. People worry about the phrasing of, like, we will replace your junior engineer because nobody likes that kind of statement. But in actual fact, yes, like, some some part of it does happen. Other junior engineers embrace it and they become more productive, which is exactly like you just said. So I guess, like, is is there a new term for this role or this this job that is taking... That is having traction with your customers?
21:00 You mean the the Super AI poll junior legal? Nothing I found yet. Maybe we should coin one. Isn't it legal legal That's our... And and also now an industry thing. Yeah. Yeah. CSJDs. Exactly. Yeah. Yeah. Yeah. Yeah. And maybe that's the... There are those other people that are best positioned to do this kind of stuff. I think the the the derogatory term of meat proxy, where you just, like, sit in front of the machine and type in something that someone else tells you to type, and then you repeat what the machine tells you to say. Yep. But, like, I do think a very good meat proxy is just like... Well, you know agents very well, and you you know Legora very well Yep. And you know how to, like like, this is my toolkit, and, you know, every industry has a toolkit Yep. And I can use my tools to do the thing that you need me to do. But I I guess then you're not really a proxy. Right? You're actually just doing meaningful work. Yeah. It's it's just like I no longer... Like, you know, well, I'll talk about tabular review because that... That's that's a thing that happens a lot. I have hundreds, thousands, millions, thou... And thousands. We do up to a million cells, I think. In a in a single... That's impressive. Yep. We should talk about the the the table and the windowing. Yeah. Least, you know, I I have, like... I'm working on a big case. I have a million documents. I type it in. Usually, I'll throw a pair... Throw, like, a team of paralegals locked in the room. There, so, like, 4AM. Like, that that doesn't need to happen. Right? Because now they can extract it, and you can go and re prompt everything. Mhmm. So, like, yeah, I I think now I don't need a paralegal for that, but, like, a good, like, super AI pilled paralegal would do something else.
22:30 You know? Yes. Yeah. Exactly. Whatever it is. They can do more stuff or, like, spend more time on the things that they, you know, didn't have time for before. Or actually sleep, who knows? No. I have a question. So you guys bought a company, Qura. Yes. Is that how you say it? No. You guys used your own product to do the M and A? Yes. We do that all the time. So how did that work? Like, can you talk little bit about Yeah. No. That works great. We're doing that a lot. Our legal team is is... I mean, intake has been built with our legal team. Because right now, we have a Slack workflow where everyone throws their stuff in, and they're like, this is annoying. Yeah. Right? And inefficient, so it's like, great. We should use Legora for that. Yeah. Can you talk a little bit about that? Like, you know, what that process looked like? Yeah. Absolutely. So it's it's... We do a lot of the due diligence ourselves because we can do it with tabular review and other tools. We do still have counsel coming in Mhmm. To, like, assess certain risk or, like, you know, there's and so Qura was simple because it was all in Sweden, but, you know, we've done one in Canada and one in Australia, and so there's, like, payroll in Australia. Okay. Do we want someone to look at this, like, a bit more strategically? So there, we will still use lawyers for the strategic part, where they can actually be sparring partners and help us a lot on, like, how to figure out how to structure this the right way. But a lot of the more menial stuff, we could actually just do ourselves. So, like, a lot of the early things we would find, and we have had cases where we found something, that's been, like, a red flag and then we've we've pulled the brakes. Can you talk a little bit about any of these? No, I think so. Alright. Early on, you said you... Your early customers were Swedish? Mhmm. Like, Swedish law firms? Yep. Have you seen a difference between working with, like, Swedish law versus, like, American law firms? Mhmm. That sort of thing? And you guys are mostly working with, like, big, big law firms. Right?
24:00 Well, we're also... Smaller law firms also. Oh, We we definitely started really, really big law firms, but 50 lawyers also and and get a lot of value. Yeah. So have you seen differences between working with, like, the Swedish ones Yeah. Versus American ones, maybe, like, even Mhmm. I don't know, Asian ones? Yeah. Yeah. I think the European ones have been a bit slower to move. The American ones are very forward, move really quickly. I think it's also very high on their agenda of the American companies, more so than the European ones, I think. It's like a strategic, super important thing. We need to just, like, do a lot here. The American companies generally, both corporates and law firms, also have a much higher appetite to, like, custom build stuff. That's like a thing that they all want to do. Like, hey, we wanna build this, like, huge thing with you guys and, like, structure it in whatever way, whereas Europeans are much more like, we'll just take the off the shelf and What's an example of something they would wanna build themselves? Integrations into their own... Like, some of these customers build their own CRM and their own document management system and their own all sorts of stuff, and they wanna build that, and then they wanna build triggers that, like, you do a review in in in Salesforce and it goes to this one person for whatever reason, and they do something, and then it goes into Legora, and then it goes back in. And there is... So that's like the simple end, and there are some that are extremely ambitious, whereas, like, a whole new product that they want to build with us, and I, you know, we can't really do that. Yeah. Yeah. But but that's definitely a thing where, like, American companies really want to build their own more than European ones do.
25:30 And the European ones are like, hey, like, you know, they give you product suggestions or... Yeah. For sure. I mean, we have great collaboration with all of them. Yeah. So they'll they'll say, hey, this sucks, this is great, we want more of this, and we'll just build a a product that does that. Okay. So they don't see, like, an existential threat here in their business? I mean... Whereas the American ones do, or... I don't know if that's the reason that they want a custom build versus not custom I think they both know that this is, like, extremely important and key to their firm's success and their corporate success. So they... It's both extremely important. I think it's just a different way to achieve it. Okay. Another question I think maybe you you wanna bring this up is, like, product metrics. Like, how do you know that, like, you know, we're actually doing something good here? Yeah. Yeah. How do you think about that? I'm assuming that's also related to evals. So we have the regular product metrics that we care a lot about. Are people spending a lot of time in our product? Do they seem to get a lot of value out of it? Do they renew it? Do they come back a lot? We have really good numbers for an enterprise software of, like, our DAU over MAU and our, like, you know, hours spent per month in the product are extremely high. And then we do some sort of... We do online evals also where it's, like, try to figure out generally, like, do we solve the problem here? Just with, like, super cheap LMS address, try to just, like, have an online metric, and then we can use that also when we do rollouts of new things. We can just like compare feature flag wise, okay, does it did it improve or like did it catastrophically decline? Because one thing actually that might be interesting, that's difficult for our business is we can't look at any traces basically. We can't look at any data because it's all super confidential. And that's, like, extremely frustrating to work with because you you basically don't know what people are doing. So it's like, here's an agent, and it can do so many amazing things, and I have no idea what people are actually doing.
27:00 I would have assumed you guys, like, signed some sort sort of agreement with the law firm, and they're like, oh, I wish. Yeah. I wish. No. But the thing is, like, a lot of the data that they have is not even even theirs. Right? It's their clients. Oh. And it's super confidential, so we can't look at it. And so, basically, we have an evals team that's, like, trying to approximate what we think everyone is doing and, like, build that into eval sets. Wow. So that's just, like, a super annoying kink of the business. Can you do paraphrase? So, for example, ChatGPT has, like, a slash feedback thing Mhmm. That directly fully sets its full trajectory. Mhmm. But, obviously, that would... That is not possible for you guys. Mhmm. But you can use an LLM to paraphrase, like, from this real thing Yep. Make a synthetic thing that is, like, close enough, but change all the names, change all the details, but, like, still Yep. Yeah. We can't... There's... I mean, there's always a a range, right, of, like, if you if you paraphrase all the way down to, like, three values and you pick one of them, then obviously, that's fine. If you if you just remove the names, then that's maybe maybe not fine. Yeah. It's like, we are generally pick the the most safe way. Yeah. Because, like, we we restrict ourselves as most possible because we, you know, there should be no risk at at all. Yeah. But there are things where it's like, okay, we know that the failure modes are probably these 10 or 20, so we can classify it into one of those, and that's Yeah. A helpful signal. We can classify it into the tech... Types of tasks and the practice areas also, and then, like, we have an idea. It's something that we're always trying to to figure out how to do in a smart way.
28:30 Also, like, on the privacy front, I guess, I'm assuming even two years ago, it must have been, like, insanely difficult to get the law firms to be okay with this. Yes. But, like Yep. How do they think about their data being sent over to Anthropic or Yeah. Yeah. Yeah. It must be, like, a huge thing. All of our infrastructure is on Azure. Yeah. Primarily for this reason, because, like, when we started, law firms were like, AWS? No. Like Yeah. No way. Don't trust AWS. Yeah. Yeah. It's different now. No. But it... There was, like, Amazon, Google. We can't trust them within our data, so we... Basically, we had to host everything on Microsoft because they already have three sixty five. Oh, so they're okay with Microsoft now? Exactly. Because they've already done, like, whatever security review, etcetera, for all of their documents and all of their Word and whatever. So, like, Microsoft is an approved vendor in their mind, so we had to host everything on Azure, and that's changed over the past few years. So, like, now people understand, okay, we need to get on cloud, we need to do all this kind of stuff. There was also a lot more people talking about on prem back then. That's also disappeared completely. People understand that, like, you should just do it in the cloud, and it's and it's safe, and it's probably safer than what you're doing. Yeah. So that's changed a lot. But I was And by the it's only been three years, so, like, it's actually like Yeah. It's yeah. Yeah. It's a huge shift. It's it's it's it's great. Mean, I good for you guys, but, yeah. I mean, I don't expect the the, like, you know, the legal firm... Legal industry has somehow ignored this for the last, like, ten years. Yeah. Yeah. I think I think we needed the AI
30:00 kicking the butt for everyone to understand that we need to change our ways a little bit. Okay. Okay. So, also, then, on, like, open source models now are getting really good. Mhmm. So, yeah, how are you thinking about that with the Yeah. We're just thinking about, to to your point, just bring our clients along on, like, the Pareto frontier. Yeah. And, like, the of the map of all the models and what they're good at and their latency and their cost and their governance and and and their regional availability, how do we just optimize across all of that? So, some open weights models are great for certain tasks, then we'll use them there. Some will post train, and and some will just use the frontier models. Yeah. You call you call it model arbitrage. Yeah. You have different favorites. You you did some research on this, where you were firmly Anthropic at the start of the year. Yeah. Grok is improving a lot. Yeah. You wanna give an example of a model that wasn't ranked highly overall, but, like, you wanna shout it out as surprisingly good? Yeah. I think not overall good. Gemini, one of the early Geminis is, yeah, extremely good for certain types of transcription Yeah. And speaker attribution. So we use that for trans... Transcription actually. You have a Yeah. Why are you guys transcribing? Granola? We don't have... We actually probably will build, like, a meeting thing Yeah. But this is just, like, talk to speech, speech to text, text to speech, just like in in any any other app that has this. And we do actually also transcribe files. If you upload hearings or whatever Oh. We'll we'll then, like, transcribe them, put them into text, and so we can do all of our analysis on it. No ElevenLabs. No ElevenLabs for DPA reasons. Same thing as, like, the data processing agreement, I should say. So, like, our data processing agreement has very few vendors on it Yeah.
31:30 Because we just need to keep it first. Exactly. We need to keep it super tight on where any data goes. And so I think, no, that that is one of the benefits of, like, a Gemini and OpenAI where they're like, you you come to us, we have most of the things that you need, and that is actually very valuable. Even if it's like, it doesn't have to be, like, state of the art. As long as it's, like, mostly good enough for all tasks Yep. Then who cares? Yep. Like Yeah. No. Absolutely. I should actually shout out also Microsoft's transcription model. They may want MAI-Transcribe? Yeah. Okay. It's great. Okay. It's better than the old Gemini. Yeah. Yeah. Yeah. They they they did some good results. I don't know... Don't even know, like... I think, like, usually, all of them are, like, top two, top three, top four in terms of, like, the the Arena scores, and that's good enough. Yeah. Exactly. And it's like, you have all the governance and it's super scalable, it's like, then that's probably good enough. Yeah. I was gonna say, like, at this point, our... Pretty much the speech to text model is, like, good enough to where you're like, oh, one word here or there, if it's off, that's No? No? No. For sure. Like, I've done this a lot. And particularly in legal, there's like some... We we had this thing where I think it's... I forget if it's like in Australia where there's like a certain name for like a a person in court, and they would consistently prescribe it as condom. Oh. And so there... Yeah. There's there's certain things that you don't want there. Yeah. But... So usually, you run an LLM layer on top of it Yeah. And then you you extract some keywords, and then you run the transcription model again with the keywords. Mhmm. Like, this is kind of my my workflow so far. Yeah. But, yeah, if you... If if you've just done this on any large scale, you will pick up tons and tons of transcription errors on any domain that is not, like, average day to day conversation.
33:00 Yeah. Absolutely. If there's any sort of jargon in there Yeah. It's gonna be tough. Yeah. Like, I think Whisper and ElevenLabs still transcribe Claude Code as Cloud Code. Yeah. And, like, it's Yeah. I've I've actually had that. Very common. Yeah. So so, like, that's that's how you know it. There's, like, a lot more to improve. Yeah. Absolutely. I do want to cover a little bit on the the agent product side. So we've covered a few of, like, the of the top level stuff. Right? We've already named job tabular review. Mhmm. We named jobs intake. Yep. And you've you've got some other stuff that we haven't even touched on yet. But I just wanna cover the the big two, let's call it, like Yep. Is it... That is is that fair to say that they are the big two? Agent and tabular review? And our And projects. Probably Yeah. Yeah. And our Word add in Okay. Is also something... Because that... A lot of people work work inside of Word, and so our Word add in is is a huge... And and Outlook add in also. Yeah. So I just wanna cover, like, the the big ones that everyone must know. Yep. So tabular review is basically like fan out of, like, a lot across each document. You can ask the same question across all all documents Exactly. On the table Yeah. And then work back and forth. And then can Yep. Million is impressive. What's... Any other, like, engineering considerations or performance things? Like, what... What's hard about this? Before we do that, can you just, like, walk through, like, what is a user experience for a lawyer? And then, like Okay. Actually, wanna use tabular view? Yeah. Yeah. Like, so, like, what are they doing, like, with tabular review? Yeah. And then we can go into that.
34:30 Sure. Yeah. Yeah. So they... It's typically... I mean, a good example is just a due diligence, right, where it's like, we need to review all employment contracts, all supply contracts, all of whatever. And so employment contracts, you put 10,000 employment contracts in there, and then you Like, each cell is one employment contract? Each row is one employment contract. And then for cells, you ask questions, basically. So, like, the first question... The first... No, sorry, per column. So the first column might be, what's the termination period? And then it's gonna, like, fill out for everyone. It's thirty days or whatever it is. And the next question is, like, is there an IP assignment clause or whatever. Next one is, like, how much, you know, how much do we have to pay them if we fire them, or whatever it is, like, much are they getting paid? And then it's gonna fill out the entire thing. You can also just ask the agent to do this. Like, if you just drop 10,000 files into the agent and say, like, do a due diligence on this, it will spin up a tabular review and run all of them and wait for it to be done, and then analyze the results, and it can then highlight, like, red flags. Okay. This... The CEO is getting paid an outrageous amount or there's, like, this weird bonus structure. Okay. Then, like, obviously, we need to look at that, and then the agent from that can draft a memo to send to the client, or it can just do the review and then have, you know, the lawyer, the user, just be like, okay, let's let's look at this, here are the red flags. There's also collaborative slide, you can comment on it for other people to look at, etcetera. And why do you choose that, like, spreadsheet format? Yeah. It's just a lot better to... It's it's super easy to work with... Work in. It's really easy to get an overview super quickly of what's going on and to review it. So all of the stuff that we produce also has citations back to source documents, and so you can basically review it super easily. If you wanna just make sure that everything's right, you can also, like, highlight cells, these are reviewed or whatever, these are the ones that are flagged and you can comment and collaborate. That's much easier than any other form factor that we've tried. Yeah.
36:00 I will say that the spreadsheet form factor has has a lot of pull in some areas, but somehow hasn't really spread. Mhmm. I'd... I find I find it very weird. Like, Clay is another one on the GTM side that, you know, very known for the spreadsheets. Yep. You guys... I don't really know if anyone else does large spreadsheet AI thing. No. No. There there was, like, V7 that, like, tried to launch... There's, like, sheets, there's... Even in finance domain, I don't really have, like, a a top name that, like, does spreadsheets. That's just really well... Yeah. No. I I agree. I think it's, like, a great... If you if you have fan out and you do the exact same thing and you... You're extracting stuff, it's it's such a great form factor to, like, understand what's going on. Do you find, like, you must export to Excel or, like, people are happy to live in your spreadsheet? People are happy to live there. Do you use AG Grid or your own We do use AG Grid. AG Grid. That seems to be like the leading one. I mean, they've done this for, I don't know, fifteen years. Yeah. Yeah. And they know how to... Yeah. So so... Okay. Yeah. Mean, I'm I'm happy on this... That spreadsheet angle. I mean, I think the the only other interesting thing for me engineering wise is the fan outs. Yep. Engineers maybe can... Are well equipped to understand the trade offs of, like, well, okay, before you use fan out on a million documents, maybe test it on, like, two.
37:30 Mhmm. Yep. And... Yeah. Yeah. Does that work before you go nuts? Yep. Do they do that? Yeah. They will. Our tabular view actually will run the first 10 rows. Yeah. Just check. Right? Yeah. Does this make sense? Did it did it show us what we thought it was gonna show us? Yeah. That's a... That's really good. Oh, but also for people, when people build this, it's just really good for them to iterate quickly on the 10 rows until, like, they have a... They're getting the results that they want, and then they can go ham. Yeah. I am happy to move on to the next thing, intake, which is your Slackbot. Yep. Yep. Everyone has a Slackbot. What's different about yours? So ours is just legal specific, and what what happens with ours is it's not actually... It might also... We... We'll probably also build one that's just an interface to the agent, but it's actually specifically to basically kick off an agent that then adds a task to our list. So we have this list thing, which is like linear, for for... You know, that's the closest equivalent, and so it will add it to a specific intake list, and then lawyers and agents can go into Legora and just be like, okay, these are the things that, you know, people are sending to us as our law firm or as our legal team in a big corporate. We can kick off agents for them, we can collaborate on them, and then we can just like mark them as done, and then it gets sent back in Slack. So it's more of a... It's not just like kicking off an agent, it's more of a like, I don't know, storefront thing where it's like, hey, here's the ticket, I want this thing reviewed. Okay. Now it's not... He
原文版权归 Latent Space 及原作者所有，本站转载仅供阅读。如需删除，请联系 support@aisa.one。
Studio