0:00

Welcome to Further Comments. Horace Wu, how are you, man? Damien Riehl, I'm doing ve- well, well, I, I am doing okay. I'm doing okay. I think I'm surviving. I no longer know what time zone I'm in. I no longer know who I'm talking to. So you're gonna have to remind me. Uh, who, who are you? uh, time is a social construct, man. I mean, time is whatever we make of it. Time is a circle… no, time is the only construct that is not a social construct. Time is the only construct which is actually universally given to us. But Einstein says that it's relative, right? There's the theory of relativity where your time is not my time. And so I would say that your time in the Southern Hemisphere in Australia is not my time here, and your getting two hours sleep is not my time either. So, uh, drinking coffee at midnight and all that, yes so for the listeners, Horace is currently drinking coffee, because in Australia it is midnight his time, where in Minnesota where I am, it's 9:00 AM. And, we also have a great guest here today that I'm excited about. We have Ian. Welcome Ian. Thank you. what time zone are you coming from? I'm coming to you from Miami where it is 10:00 AM, so bright and early-ish on a Monday. Excellent. We're, we're thrilled to have you. Ian, for those who don't know, does amazing things with training. So maybe, Ian, maybe you could, say Thank you. a few sentences on what you do. Yeah, sure. So thanks for having me. Real treat to be here. I'm Ian, one of the co-founders of Hotshot, and we're a learning platform for lawyers. And the 30,000-foot overview is I'm a recovering M&A lawyer myself, so I lived the pain of not knowing what I was doing as a baby lawyer. So we focus all on training. Wow. I, I was an M&A lawyer as well, and I also remember those painful days. Mm-hmm. Yeah, we all do. We all do. There's a lot of pain, with both M&A and litigation. But AI is making that pain a lot easier. And it turns out that you can teach such things, how to make your life easier through AI. And I think what we're gonna be talking about a bit today. Well, it's a hot topic right now. Um, everyone is trying to figure out how to train the next generation of lawyers, 'cause they can see, like, that pipeline of talent ending if we don't do something about it. And we kind of talk about this on the podcast, uh, a lot, that AI is so powerful, it's so good at generating stuff that is, uh, third-year, fourth-year level, depending on kind of like, you know, what your prompts are, that a lot of lawyers no longer use junior lawyers,'cause they go, "Hey, it's taking me more time to get a crappier piece of work back. I'm just gonna pass it on to a piece of AI." And, and that's causing catastrophic flow-on effects to what is going to be, like, the next generation's partners. So like, Ian, I, I know you see a lot of this. Like, what, what are you hearing in the market? Yeah, yeah. I mean, look, I'm obviously, I'm trying to take my bias away from this, having a training company, but I have never seen a moment for training like it's having now. I mean, we're having conversations every day with multiple firms and schools about this issue. So sure, AI is doing the work of the, of the junior associate, but the conversation is now, how do we upskill these junior associates much, much, much faster and be extremely thoughtful about getting them those reps, getting them the broad skill base that they need to be able to succeed today, right? And it's not just about training on AI, it's, there's almost a renaissance of back to baseline training. How do we teach people the underlying work itself so they can make sense of what the AI is spitting out? And how do we do that, like, today, not over five years? So those are the sorts of conversations we're having. I speak a lot, and some of the people say, "Well, what should I read, and what should I do to be able to learn how to use AI?" My response to them is, "What do you need to read to learn how to swim?" And there's no book you can read. You need to swim. How do you provide structured training, that is essentially teaching them how to swim, without actually getting in the pool? Yeah, that's right. And that's something we're focused on a lot these days. So there's sort of the two sides to it. There's the training on the AI itself, and it's being thoughtful about giving people the reps on the underlying skills. But to your point, Damien, the structured training that we're seeing the biggest interest in and the biggest movement is not feature function training. It is getting people together in a room and creating simulated exercises that are based on real work, right? And whether that's for a litigator, an M&A lawyer, a member of an L&D team, a member of finance, how do we get people together and create those experiences, hands-on learning? And that's the sort of things we're doing. We're helping firms with the scaffolding that they need to run it themselves, to teach their own way of doing things and their policies, and learning with and from each other. That, to me, is where the light bulb moments always happen. Someone says, "I, I didn't know you can do that," or, "I use it in this way." And it's just magic to see it happen Agreed. And that scaffolding is kind of teaching people this theater of the possible. So many people, so many lawyers are just so head down on their work, they don't even think about what is possible. And so in a sense you, Ian, and everyone that is doing training is really saying,"Okay, let's open your eyes as to the possibilities that you can get your ideation juices flowing to be able to say, 'Gosh, what else can I do? I just did this this morning. I could use AI for that.'" So I think the theater of the possible is a larger work that you're doing and other people are doing. Thanks Damien. And, and look, and hats off to you all, what you're doing with the training initiative. You know, there's sort of a level of foundational knowledge that people still don't have, and I think there's an overestimation of what people know these days. I mean, I involved in a lot of training conversations, and still people will pause and say, "Ian, could… you said LLM, what's that? Like, what, what does AI stand for?" You know, and that's a real issue. So to me, the valuable time together is the applying. It's not the what is this tech and what are my responsibilities to supervise my juniors. That's critical, but we're sort of having this interesting conversation on what's the best place for that, what's the best resource for that, who's best to deliver it, blah, blah, blah, blah, all those sorts of things. Because, you know, the, the, the learning by doing is critical, but if you don't the actual underlying knowledge of it all, there's, there's a real problem and you're still not meeting people where they are. We've talked in the past about, uh, our friend Megan Ma and the amazing things she's doing with Stanford. Do you wanna extrapolate on what she's doing and how that might be helpful? Yeah. So I, I know that she is doing something with a couple of firms, one of them being Dechert. Um, where they're essentially using generative AI as a way of simulating an experienced attorney, um, so that this is what provides the training. And, and it's, it's a really interesting take on what generative AI, generative AI can do, um, because essentially- Mm-hmm it's saying, well- Mm-hmm … you know, we need, we know we need training, we know we don't have time for training, and therefore how do we leverage that to do the work for us? Like, Ian, what, what are you seeing in, in, in, in that space? And I also wanna cover the content you train on, because I think that's gonna be really interesting to a lot of folks as well. Oh, thanks. And, and anyone that knows me knows I don't just shill Hotshot when I speak, so I'll look not to do that. Anyone can contact me if they want to learn the details. Um, I think what Megan is doing is amazing. I think it… I know her a long time. She's actually in some of our videos. Her and her team are doing amazing work, and I think it's a critical part of the puzzle, the giving people hands-on reps, simulated, interactive, because not every firm has the time to do it in person. So it's, it's all part of the puzzle. Um, what we're doing is we're, we're really, we've been leaning into the sorts of content that, that people need to know these days. So certainly we're doing tons when it comes to training people on AI and all the use cases and the workshop material, but we're really looking at the core knowledge, too. It's business skills, professional skills, technology skills. Not just AI. It's interesting how popular a topic on using Word is because this new generation grew up on Google Docs. Um, the economics of law firms, um, you know, best billing practices, blah, blah, blah, all that stuff. So we're looking at all the different, uh, areas that people need and trying to build thoughtful curriculums around that, and then baking that stuff into thoughtful sort of experiential workshops where they're using AI to apply all that knowledge. So we're, we're, we're, we're busy. The work that you're trying to do is, is really God's work. And this brings me to, an image that was brought to me in 2015. For those watching on YouTube, you can see. But for those listening, it's two people dragging a cart that has square wheels, square stone wheels. And somebody provides a round wheel. And the square wheel people say,"No, thanks. We're too busy." Uh, and so, Ugh. is, this is, uh, this is evergreen. Uh, it was true in 2015. It's true today. It was true in time immemorial. So lawyers are just too busy to learn the things that are gonna make their lives so much better. I think we have to get rid of that inertia. That is, we need to be able to wake people up from their stupor. And I can imagine one way to wake people up from their stupor is money. Lawyers are really fearful that their business model is being upended. And fear is a good motivator to have training. Number one, what do you think of the cartoon?

And number two: 8:09

is the motivator of money as fear, is that a motivator for them to use you? A million percent, and that cartoon is our life. In a prior life, I helped start Practical Law in the US, and that cartoon couldn't have been more accurate, you know? Money a huge factor here, and client pushback is a huge factor. I think the economics have changed such that training is essential now. When we started Hotshot, I think it was a bit of a nice to have, and we were sort of focused on professional development teams, which is still critical. But that was about it really. Now that the money has… The whole conversation has changed. So now C-suites are having the conversation about training. It's an economic imperative to get people skilled up quickly, and that's why I think we're having this moment and anyone involved in training is having a moment. 100%. I've been thinking a bit about things and Ian, you're maybe, most suited to be able to answer this question. My thought is like, do we really need to train people on AI? This is a bit of a hot take controversial question, but it's kinda like the age-old question, like should we teach lawyers how to code? We've been talking about that for a decade plus, right? Because coding is a lot of friction. Like it, it takes, you know, Yep. Python or learning JavaScript, that's a lot of friction and it's hard to get up to speed. And really in the end, how useful is it for a lawyer to know JavaScript? Not super helpful, right? So anyway, so that's the old world. Do we need to train people to code? But in the new world, AI training is a bit like working for humans, with humans. You today tell your human associates what to do. You today tell that human associate that the work wasn't very good, do it better. And then you manage that human associate in a way that's helpful. All of those things that are human, interactions are what AI training is. You need to tell the AI what to do, that the work wasn't very good, and manage the AIs in a way that's helpful. So in a sense, aren't you really teaching management training? That is, Yeah? skills that people need to be able to manage humans are the same skills needed to train AI. And in a sense, we're kind of just teaching people that the same skills that you should have had all along you can use with AI. 100%, and that is training. I mean, that's a lot of the focus of these workshops. You know, one example, there was a, a, a big firm customer of ours that did an in-person program. And a very, very experienced person in the room got some output based on a, on a prompt, and the person said, "I knew this was garbage. This isn't for me. It's too long. There are no bullets. This isn't how I write." And the comment back was, "Ah, say that. Click that button. Say that." It came back perfect, jaw-dropping, light bulb moment. But that is training. I do not think lawyer needs to understand how to build something using AI, but I think they need to understand how to work with it. And as you see the, this tremendous push for legal engineers, there'll be people that can do that and help do this, that, and the other thing. But there's a, a long way to go to teach people how to work with AI and what it does and what it does best and what it does not do best. And I, I'm in the trenches every single day on it. I see it every single day. There's a, there's a critical need to teach people about it and how to work with it, but maybe not, the equivalent of whatever coding is for AI. I'm gonna zoom us out, like zo- zoom us way, way, way out and, and ask that more fundamental question that I think is on everyone's mind. Which is how has AI changed training overall? Like, like if you take a snapshot today versus take a snapshot five years ago, what are the fundamental differences? In so many ways, I think that AI training has fundamentally changed everything. I think that what is changing is people are reexamining the core skills and knowledge that people need to have. So it kind of used to be what kind of legal skills do you have? But now it's, it's the professional skills, it's the time management, it's the biz dev, it's the relationship building, it's the critical thinking, it's the judgment, industry knowledge. It's, you know, the mini MBA programs are more, more important than ever. There's the economics of it all. And, you know, not paying lip service to core fundamental legal training. Take my own example as a young baby lawyer, right? First M&A deal partner says, "Ian, you're on a reverse triangular merger. Go and through the diligence and tell me what looks weird." Like, what? But that paid the bills. Me being clueless paid the bills. Now when you're saying, "Okay, use this AI tool," whatever tool the firm is using, um, and spit out a due diligence report in three seconds, amazing. But the associate doesn't know what diligence is, doesn't know what the report means, doesn't know how to tell the client about it. That's a real problem. So to me, training has changed that firms are trying to sort that out. And now we're helping firms run hands-on M&A boot camps, deposition programs, all these things, then layer in the AI. So I think it's just changed in so, so many different ways. Uh, it's in- it's incredible. I, I think a lot about how what's old is new again. People might not know this. I don't know if Horace even know this. My bachelor's degree is in education. I was gonna be an Hmm I was gonna be a teacher. Uh, and so my… I was conducting a Brahms piece, and two of my tenors started punching each other in the face, and I thought, "Oh, I should go to law school." So that's, that's what I did. So anyway, part of education is that you need to teach simply. is, you need to take the complex and make it simple. And so I thought that my educational training was a good background for being a lawyer because lawyers are educators. We teach clients. We teach clients what they should be doing. We teach… Litigators teach the judge. We teach the jury. And all of the people I just mentioned, clients, judge, and jury, are all busy people, so you need to be very succinct in the way that you explain that to them. And this kind of goes back to the old… Even Einstein said, "If you can't explain it simply, you don't understand it well enough." Explaining something simply is, uh, the heart of education. In that way, education is kind of like AI because the large language models, the most frontier models, what they're doing is compressing. They are taking large language models, and they're distilling it, and distillation is compression, and the more distilled you make the large language model, the more intelligent the large language models are. And I just heard something that blew my mind from Alex Wissner-Gross. He's one of the thinkers in AI that I respect a lot. He said that intelligence is compression, and the more we compress the large language models, the more intelligent the large language models become.

Is education compression: 13:57

distillation into simplicity? And, is really that a sign of intelligence? What we do at Hotshot, we think,"how can we make it as practical and short as humanly possible?" Not,"How long could it possibly be?" You don't need a three-hour webinar or panel to explain the concepts of XYZ legal concepts. So, like, that's where our approach to sort of on-demand training came from. So the videos that we make are as short and distilled as possible, so that learning gets done in advance on your own, so that the together time is focused on the doing. So Damien, I, I totally agree, and people just don't have time. People are pressed for time. Um, it's, you know, what's that, you know, it's, it's, it… I'd, I'd write a short letter if I had the time. I think that's Hmm and that's, that's our existence at Hotshot. That's what we do, is to make the complex simple in as short a time as possible. And it's funny, when we have conversations with firms about CLE, I say, "We're not… Don't come to us to fulfill your CLE needs." You know, those rules about how long can something possibly be. Ours is we wanna teach you something quickly and efficiently. Yes, we do have CLE now because we reverse engineered it and worked with the ABA, but that's not our jam. You know, our jam is short, practical, useful. And that relates to short, practical, useful. It used to be that time was a marker of quality. That is the longer the CLE, the more quality you get. As if you attended an eight-hour CLE, man, you know that cold, uh, right? But the reality of human psychology is that after twenty minutes, your mind wanders, and people start reading the newspaper and checking their emails and doing things. You know, this has been time immemorial. So the duration of training is not the thing. It's what people actually pull out of the training. And I often think when I'm presenting, for a one-hour session, if I'm really lucky, they'll take three things out of that session. But really probably, if I'm super lucky, they'll take one thing out of that. From that [session]. And so really, I think as I give the talk, what's the point of me talking for an hour? Because, like, I, I may as well be… If they're only gonna take one thing out of it anyway, I'm just a dancing monkey that's entertaining them, right? And so, so really, the, in, in that sense, like, is our goal to entertain, that is to keep you stimulated in a way that you can maybe take one or two or three or ten things out of the session? Yeah, I think so. I think so. And, and sort of, you know, the other thing that's changing is training, is people being very thoughtful about the best mode to teach the best thing, right? If it is delivery of core information, a good trusted, vetted resource, do it in advance and show up so that you know what everyone will be talking about when we're applying the things together. And it should be really short. Who wants to sit there for an hour and learn some concept that can be taught in 10 or 15 minutes, if you take the time to, to shorten it, right? It's easy to go on really long. So yeah, I agree with you, totally. And what is interesting for the hour, one hour, three hour, four hours, those are the, those are the in-person, hands-on keyboards boot camps that people love. They absolutely love them. They're learning with each other, learning from each other. They're not just sitting there listening to stuff going over their head. Well, that, that's a really good point I wanna dive in on because you said learning from each other and learning with each other. Um, what sort of formats and content … Well, actually, content is not the important part here. What sort of format is the best way to deliver content so that people can learn? Like, has, has that changed? Um, is, is it one-pagers? Is it, is it videos? Like, what, what, what absorb- what do people absorb? Yeah, well, but I could speak from our experience, and, and now we're supporting about 60 of the Am Law 100, so I'm seeing a lot of experience and, and loads of small and boutique firms. I think the best material for those programs are, like I said, the scaffolding. So at least what we're creating are full simulated matters, hypothetical companies, all the sample documents, all the teaching points, all the exercises, every use case you could think of. So that in a box the firm has everything they need to then use their AI in those interesting ways. So that all the build is done and then they're just doing the things together. To me, that's the materials. Not a one-pager. It's, you know, if it's an M&A workshop, it's the full sample purchase agreement. It's the redline from the other side. It's a complicated fake thread of emails where they can draft a term sheet from it. It's all that stuff. Instructions for the presenters on how to run it, how to use the tool to do it. It's that sort of a thing. And the on-demand stuff, at least that we provide, teaches the core concepts, right? So if it's an M&A workshop, and maybe it's a junior that was invited and they don't understand rep and warranty insurance, they could watch a short hotshot course on that. The in-person time is not the time to lecture about the ins and outs of rep and warranty insurance, and that's what we do. Hmm I mean, an example in play is one of our big firms this summer, their summer associate program. We saw 1,700 course views of our courses in the first couple of weeks of their programs. Because they, they want to be prepared for in-person training programs. They want the knowledge. And then the in-person programs are full workshops, um, that we're providing. I love it. And what you just said, I want the listeners to re-reiterate

is that what you just said is: 18:44

use the best tool for the job.

Everyone's heard the meme: 18:47

this meeting could have been an email. So don't do information transfer in a meeting. Do information transfer on demand. Use your in-person time for collaboration because that's the only thing that cannot be done, uh, really. And really, there's 2D collaboration, and there's 3D collaboration. Thousands of years of human evolution, we trust 3D better than 2D. Being in the room and shaking someone's hand and having those side conversations, that's hard evolution to be able to train out of us. And so really use the right tool for the job. Use information transfer asynchronously, but if you're in person, let's collaborate. So that's point number one. Point number two is that, John Sonsteng is a professor. Uh, he was my professor back in 1999. He's eighty-three years old. He has created what I'm calling the magnum opus. He's been authoring many textbooks on how to teach legal practice to lawyers over the last… he started in 1972. So anyway, so he's got this magnum opus of things, and he created a whole bunch of simulations even back in the 1970s. Where Hmm he would create fictional scenarios of litigation, saying, "Your client comes to you in this, this matter. What happens?" And we were just talking this last week, and he said, "That's really hard to be able to create all of these fictional scenarios in a way that's helpful and be able to walk the students through the real-world, things that they'll have to do." He's been doing it for 50 years. So it turns out that AI can do that quite well. So Hmm over the last week, I've been using Fable, to be able to recreate his magnum opus and be able to create simulations in litigation and in transactional work and in a bunch of others, and taking his forty years of textbooks, and be able to use that as context to be able to run through these scenarios. And I'm using Fable's agents. I'm saying, "Fable, do be an orchestrator and have Opus be the subagents to build out all these scenarios." And right now we have, I think fifty different synthetic data of real-world things. And then I'm now gonna be essentially replicating what Megan Ma is doing and being able to say, "Okay, let-- now let's do an oral, uh, examination of the client with this c-context." Um, so anyway, I think that, build versus buy is something that all lawyers are thinking about, and they're thinking about it for substantive legal work. And they're probably also thinking about for training. And so Ian, how do you respond to this kind of build versus buy? Uh, you know, if You've just vibe coded Hotshot, Damien. Is that, is that what you're telling Ian right now? You're telling Ian you're, "i'm, i'm gonna take your business vibe coded!" No, I'm saying that Hotshot is probably better than what I've vibe coded, and I'm inviting Ian to say why. Yes. Yeah, no, I agree. And look, the truth is, it's getting easier and easier to do these things, and the build and the tech isn't the moat anymore. You know, I, uh, when, when ChatGPT first came out, I saw, you know, 87 different law professors build their own little cool little games in a weekend. And I thought, "Okay, maybe that's not, that's not where the future's headed." So Damien, I applaud what you're doing, and we're doing the similar things at scale. It just makes life easier, you know? I agree with you. I… The other, the reality, though, is that a lot of law firms, and I had a conversation on Friday about it, they think they can do it better, they should be the ones doing it. So when I think about build versus buy, it's build and buy. There are things the firm should be doing, and there are specific things only they can teach their people. But it's not creating the simulations, it's not creating the short videos. I mean, that's, that's why we exist. We do it at scale, we do it well. And we also think through, you know, it's a game of Jenga, right? It's easy to create the simulations. There's how does a person get in front of the room and present? What are their key points? What's the takeaway? How do you create the answers to them? How do they go back and reinforce the learning? How do you prove to the firm they learned it? So there's all these things around it, right? That I'm a bit biased, that I think are important to it, too. So yeah. I think it's all those things. A- and I think that there's an aspect too that if somebody internally makes something, that's just the jerk down the hall that is saying this. But if somebody externally comes in, then that's the expert. There's an old, and I, I think it's an even a biblical thing that Jesus said, I'm famous everywhere but my hometown, right? So the idea is that if you're building, you're in the hometown, and people don't respect you nearly as much as Ian and Hotshot, that is the external person coming and saying that. So there's probably value to that. That's point number one. Point number two is that, friend of the pod, Jason Barnwell, when we talked about build versus buy, during the SaaSpocalypse, where I was freaking out saying, "Oh, SaaS is, SaaS is dead," uh, he said, "Damien, have you met people?" Right have-- Like, like have you realized that people that say that this software sucks don't realize that they could go to Settings and then click one button and the software will do exactly what they want to do? And he also said that, "Damien, I garden, and I still go to restaurants, because I could make everything myself, but there's value to someone having a curated experience, really knowing what they're talking about." So I Yeah Hotshot is probably the restaurant, I could garden, but restaurants can provide me better value. What do you think of that? yeah, no, obviously I like that, and I mean, I do agree. And the curated bit is very interesting because starting to hear from some firms that will say, "Well, know, we, we told the associates to go on ChatGPT," or,"Our name legal vendor has some videos that help train." And then almost every time, two or three weeks later, they come back to us and say, "You know what? It just wasn't curated. It wasn't the full thoughtful experience. Clearly educators didn't put this together." So, you know, I, I think it takes a village truly. And the other trend I should point out if we don't get to it, is this collaboration that's becoming so important in training. You know, it really takes, in, in my opinion, the best examples I'm seeing, professional development with innovation, with the practice groups, with knowledge management. They all bring something to the table. Whereas some baked thing that someone built on their own doesn't really have all that feedback, and everyone is necessary to this. You know, it makes me sad sometimes, I have calls with professional development teams and they say, "No, we have nothing to do with the AI training at the firm. That's innovation or that's IT." And it's like AI training and other training is training. The line has blurred and gone away. You need to all be in it together, you know? And it just there's a lot of misconceptions out there still. I was gonna say, I feel like. That's, that's a temporary thing because I think AI sort of like disrupted the entire operation of the law firm. And, and so for a lot of law firms, historically, AI wasn't part of a training program. And, and now this new chief AI officer's come in, and therefore they're in charge of everything to do with AI, including training. And so over time, I think that's gonna like work itself out. It's just for this period where, you know, everyone's just figuring out what to do with this, AI training doesn't fit in traditional training. And I think that's like, um, to your point that you made earlier, Ian, like the content changing over… The content of training, the what's covered in training changing over the last five years. It sounded like we've gone from kind of black letter law training to a lot more soft skills. Um, and, and I feel like that, that's a transformation which I think should have always taken place. But now there's that catalyst of people going, "Well, hang on. If hard letter, like black letter law is so much easier to, to get your hands on, what else can lawyers deliver as, as the value? And therefore, let's train lawyers to be better at those things." Horace, a million percent, and that's some of the areas we're seeing the biggest growth in for us is, in addition to the AI, we just created a whole series on professionalism, communication, receiving assignments, giving feedback, managing your time, the art of business development, understanding the economics of a law firm. All that is becoming in, in, in play. And, you know, it's, they're just different skills, hard, soft. They're professional skills. There's business skills. There's, you know, all these different skills, that is just so important these days. And, and, and sorry, the other, the only thing I'll say is that sort of training is now finding its way into the hard skills training. So like in our M&A workshop, there's an element of write a good red flags email to the client. That's not really, you know, understanding how to find a change of control clause. That's what does a good client email look like. So we're trying to bring all those things together. Sorry, Damien, I didn't mean to cut you off. uh, no, not at all. 100% I agree. And, uh, I would say that, the characterization of soft, as in soft skills, it kind of rubs me a bit the wrong way, Yeah that, uh, when I was a, a patent, uh, IP lawyer, people would talk about hard IP and soft IP. Hard IP is patent law. Soft IP is, oh, copyright, trademark, trade secret. Patent lawyers are like, "Oh, that's soft IP." No, that's really hard. Like, trademark is all about psychology. And I would say that most engineers that become patent [lawyers] are really bad at psychology, to figure out what the psychology of the trademark is. Anyway, so hard Mmhmm soft IP has always rubbed me the wrong way in the same way that soft skills versus hard skills, with law is. Because the soft skills are the hardest to train. That's right. That's right. I uh, yeah, how do you train somebody to be a more communicative person and to be able to be curious about what their clients are doing? Those soft skills are also the most important because that's rainmaking. Yeah. Yeah way that you become a partner is becoming a rainmaker, and that is the soft skills. And the hard skills of d- researching things, that's never gonna make you partner. So maybe you, you could talk about how soft skills are really hard. And look, that hard skills aren't all that hard anymore. This week I'm gearing up to go to the Professional Development Consortium Conference, right? You wouldn't say soft skills among that crowd because they know how important and hard it is to teach that stuff. Um, I agree. But you can even do, like we're creating simulated workshops on business development and how to use AI to do business development. But, you know, I have a really close friend that's a partner at a firm. He's exhausted today 'cause he's been taking people to the World Cup for the past month. That's important. So that has to be part of training now. I just think it's, you know, it ti- it's well past time that that's part of training. You talked a bit about earlier about the separation of roles, saying,"Oh, this is the training department, which is different than the business development, which is different than the knowledge management, which is different than all these." This is reminiscent of not my job, is reminiscent of what a lot of discussions in Silicon Valley today are with the, uh, you know, software development. You have the developers, and then you have the product people, and then you have the designers. These are three different roles. And now every developer think they can be a product and design person, and every designer thinks they can be a product and an engineer. So really, the smartest people in Silicon Valley are saying this is now just converging into one role called a builder. And so there are no separations between devs, product, and design. Everyone is just building. So I wonder if within law firms and within legal organizations, law departments, whether we just kind of have a convergence of things? That AI is gonna make things so easy that what used to be separate roles is now converging into a single one. What do you think of that? Yeah, it could be. I think that would take quite a while to change. I mean, we, we see such pushback still on getting everyone to play with each other nicely in this area and to collaborate. Um, I think it'll take a minute. I think that some of the firms, you know, their innovation teams are now taking in professional development as well as KM, so I think that osmosis will happen. But where they're still segmented, it's, it's gonna be, it's gonna be quite a while. Another thing you mentioned earlier was talking about taking the learnings and then putting it back into the system and kind of iterating upon itself. And this reminds me a lot of a harness I'm using for coding. It's called Compound Engineering, and it takes you through the software development life cycle. Where I, as a user, essentially brainstorm, say, "Here's the software that I'm thinking about doing." And then what it does is it takes those brainstorming, and it asks me questions to be able to further my thinking. And then I jam with it for maybe thirty, forty-five minutes, and then it creates a plan, and then it marches through the plan. But the most important part of Compound Engineering is at the end, where it takes everything that it learned and puts it back into a master document. That then I don't have to learn that again. So this idea of taking what we learned during the session and compounding

it, into itself: 29:37

this is, I would think, what law firms should be doing. Uh, is to be Yeah know, what is the training and then what did we learn and compound it into the next training. And this, this in a sense is kind of re-recursive self-improvement. Within Silicon Valley, they say that the AIs are currently are training the AIs. Uh, that is within OpenAI and Anthropic, ninety-nine plus percent of the code is actually created by AI, and then recursive self-improvement is essentially the AIs making the AIs better. Can we think about law firms the same way? That the work that we're doing, will go back into the training and recursively self-improve the training? I, I hope so. I mean, I, you know, and I think that's part of the advantage of this tech is being able to work with associates and professional staff and build personalized learning plans based on what you've covered, what you know, diagnostics you could do with the tech, all that business, and just build, build, build on each other. So yeah, I think, I think for sure. You know, and, and the, that's the other thing too, is thinking about all the different roles. Different roles need their own different training plans, right? When a partner needs to know about the age of AI is different than a first year, is different than a member of professional development, is different than a paralegal. So how are we thinking about the plans for everyone? There's so much talk about the lawyers and lawyers, but the biggest trend we're seeing lately is this focus on staff, and everyone's using the tools and they need their own plans. You know, and then jump all different topics. You know, the-- We're talking about collaboration within a firm, but what about the schools and the firms working together finally to sort this out, right? Because one feeds the other, and clients, we're seeing some pretty cool workshops where clients are in the room with the firms, with BD, with everyone learning together and creating these plans. I'm, I'm off on a tangent now, but there's just so much to this, you know, to, to cover that we're seeing happening. Uh, it's like everything everywhere all at once when it comes to training. On the topic of everyone everywhere all at once, I think of AI a bit like an accordion. Where you ask a short question that then the AI gives you 100-plus pages of context that you then shrink back into the bullet points. So I think of this kind of accordion thing. Is that accordion kind of the part of your training, to be able to say that we need the 100 pages of context to get the simplicity? I mean, on the back end we're trying to, to filter out all the stuff that maybe they don't need f- for the core learning that they do need. And there are resources if you wanna read a 200-page treatise on the theory of M&A and all that business. We're, we're, we're trying to do that so the lawyer, the law student doesn't have to do that, right? To your earlier point, Damien, what's the shortest, most practical thing possible? I'm gonna challenge something now. And the challenge I'm gonna make is, I recently wrote a blog post, and I think it was entitled something like "Hard Work is the Point." For a lot of learning, going through that process of actually reading that 200-page treatise, that, that hard thinking is actually how someone can absorb that for themselves. So, like condensing that into a 15-minute clip, wonderful for that, like immediate dopamine hit, but what about a longer-term thing that stays with the lawyer? Like, how do we make sure that they still know that skill two years down the track? Yeah. Yeah. Uh, that is fair and I agree with that. I think we don't see our role in providing that much deeper background. We hope they will do it. I think we're dealing with the reality that the associate, it's 8:00, they've been asked to help prep for an expert witness or whatever, do an expert report, they've never done it. Holy moly. Or have to prep for whatever it is. Um, but we are always saying that and I'll be the first to say to a client on a call,"We're not here to replace the training. No one's gonna be an expert by a 15-minute video or a three-hour boot camp." It's something, right? Something has to change and we have to get there and we hope they'll do the longer learning. I think the reality for the billable time for as long as we have it aren't allowing someone to read a 200-page treatise and count it as work and count it as billable work. The, I guess that's a difference, too. I agree. Two points, based on what Ian and Horace just said is, point number one: Ian, you mentioned briefly about the amount of time in your product that is actually inversely proportional. That the less time they spend at your product, the more valuable it is. And this is something that is contrary to many product builders, Mm-hmm. Because "time in product" is often seen as a measure of how useful it is."Look how many hours the people are using my product." And for legal research, where, you know, I was at Thomson Reuters and now Clio, the less amount of time you're in legal research, the more valuable it is — because you've gotten your answer. If you spend a lot of time in legal research, it's a very horrible product — because you've had to spend a lot of time to do the thing. So really what you measure is what you improve, and I think that a lot of people measure " amount of time" as better, but it's actually inversely proportional. That's point number one. Point number two is, what we were just talking about on training, and Horace's really good point that the friction is the thing. Because with friction you actually learn more… unless you don't. And so this is Dan Schwartz, this is something relatively new in the last month or so. He really analyzed whether AI can affect legal thinking or not, and his hypothesis going into this is that, he had two groups. One had less AI and one had more AI. And the hypothesis was that during the synthesis stage, they gave them the cases, the statutes and the regs, the people without AI would be weaker and slower, and the people with AI would be stronger and faster. But then stage two, when they took away the cases and said, "Here's a multiple choice question," and they took away the AI from both of them, the hypothesis was that the people without AI would do better because they had the friction. Really wrestling with it, where the people that just got the answers in a silver platter, they did worse. And then stage two, when they said, "Here are some facts, now apply those facts to the law, cases, statutes and regs," the hypothesis was that the people without AI would do better than the people with AI. And then when they said, "Now revise your memo based on AI," um, they thought everybody would improve, that the AI would do this. This is the hypothesis, but really what actually happened was that, yes, the people without AI did worse than the people with AI when synthesizing the cases, statutes and regs, but then at stage two, the multiple choice test, people without AI did just as well as the people with AI. Mmhmm Um, and that was un- unexpected. They thought that the AI people would do worse, but they actually did just as well. And then at stage three, applying law to facts, the people without AI did worse hmm. than the people with AI. The people with AI applied the client's facts to the law qualitatively better. And then at the last stage, to revise the data, they did kind of a mixed bag where weak writers got lifted and strong writers went down. And so anyway, so this is counterintuitive, and there's a real question is why did this tie at stage two happen with the comprehension? Um, and I would say that maybe that's because AI strengthens legal researching even after removing the AI. Because at an early stage, if you have AI clarity, then that builds a stronger mental model, Which then persists even when you take away the AI. So I, I think that this, this friction, there's a real question at, at what point do you need or want the friction? Because, uh, we've been teaching the Socratic method for-- in law schools, uh, for a hundred years, and that's friction, right? But you can imagine, you know, Gilbert's Notes, Gilbert's Summaries is something that I used in law school to be able to condense the law. I learned far more from Gilbert's Summaries than I did with, uh, Socratic method. And so I would say that maybe giving the answer with AI is maybe building a stronger mental model that is then able to get out and do things. And so I would say that the friction maybe needed is not the friction of mulling with the thing, but is, is essentially the friction of getting the answer and then mulling with what I should do with my clients based on that answer. What do you think, Ian? I think it's absolutely fascinating, and I think that sort of tracks what we're seeing, you know? And we're seeing the market sort of figure it out, 'cause we've got some customers that are insisting on they want to run workshops, no AI involved, read the diligence together, no AI whatsoever, and then in the afternoon, apply it through the tool and then see what it found and all that business. And some are saying, " Nope, we have no time for that. Not interested in that. AI from the jump." With sort of the pre-learning in advance so people know what the topic actually is, right? That's what, that's, it comes back to the videos. But yeah, Damien, that, that, that tracks with what we're seeing. That tracks with what we're seeing for sure. And I think each firm is sort of figuring it out their own way and they're testing all these different methods for what works best. So, uh, I'm gonna, I'm gonna zoom out again, 'cause I, I love, I love zooming in and out. And, and I'm gonna zoom out from, like, we've so far been talking about training of people, right? And, and essentially law firms have a few different types of assets they can draw upon. Um, people's absolutely one of those, and how good the people are is essentially what people want to pay for, or clients want to pay for. Um, but there's this idea that, uh, I hear a lot, which is, um, "Your law firm's private documents contain all your knowledge, and it's so valuable to you. And we're gonna help you unlock it and therefore, you know, introduce all of this, like, uplift to your firm." Like, what, what do we think of that argument? And I, and I say this against somewhat my own interest 'cause the whole point of our company is, you know, to unlock that data. But against my own interests, what do we… Or potentially against my own interests, what do we think about that? Damien, you wanna give that a go first? Uh, well, I think that there's the truth to that and that there's untruth to that. Mmm that is that there is a lot of things that are locked in the data that historically, you know, uh, Ian knows from Practical Law and I know from, Westlaw, and then also Clio and Fastcase, like unlocking the locked up unstructured data, structuring that unstructured data, there's a ton of value there.

And an example, is one: 38:31

searchability. To be able to find the things you need. And of course, if you tag things up, you can search it. But then there's also vector embeddings that make searchability much easier. So I would say that finding things you need, number one, unlocking that is pretty important, especially the precedent that was just like the thing you need to do now that you did three months ago. But then the second thing that it unlocks is once you tag it up, you can run analytics. You can say if you tag up motions to dismiss that are granted for summary judgment, or motions to dismiss that are granted for trade secret matters, you can then say, this judge decides in favor of the plaintiff eighty percent of the time, but that judge finds it in favor of the plaintiff twenty percent of the time. So once you unlock that data in a structured way, you can do analytics that used to just be gut instinct, anecdata. uh, And then you can turn that into real data, real analytics. So I think that there is value and there is truth to, yes, take that unstructured data, documents, and structure it to be able to find things and run analytics on things in a way that you've never been able to in the past. So anyway, so that's the way that it's true. The way that it's not true is that you still need to be able to do the hard work of being able to apply that to your clients. To be able to say, "my client today is not the client from yesterday." And the best motion is the one that works for this judge, for this client at this point in time with these facts. And the best deal, the best contract is the one that works for this deal with these two counterparties, with their goals in this jurisdiction. So I would say that doing the hard work of applying your client's facts to precedent is the thing that really needs to be done. So in that way, the-- your precedent is not going to help you. You still need to do the hard work. Ian, what do you think about such things? Totally agree. And, you know, from my view of the world, to bring it back to training, I would argue that the most important part of the training a firm can do that's unique to them is how they got to that final document. You know, just asking someone to go read that or run a workshop based on that model, great. But how did-- what was the thinking that they applied to make that nuanced change in that clause, and what was the war room like when they negotiated? How did they get there? How did they solve the problem? It's a legal problem, it's a business problem, it's a tax problem. That's the magic of the training that only that firm can do, right? It wasn't the, let's just teach you X Form because that's the form we use at the firm. Why do you use that form, and how'd you get there? AI is not gonna re- You know, some of the work Megan Ma's doing and others that they're sort of taking people's knowledge and all that is, is great. But I really think, know, the training opportunity is becoming less and less, and that's what firms should be focusing on. Their secret sauce and how they got to that document. That's the magic of it all. Horace, do you have anything before we start closing out? Yes, I do. Yes, I do. So I, I was on a call, um, literally about half an hour ago or an hour ago, um, with a, uh, a KM attorney at a large law firm. And, and she was saying that the biggest problem they have is how do we, like, go and find the right precedent? So that we can inform the lawyers in the situation that they're in, you know, what a, a reference point would be or several reference points would be? And, and, like, that search problem is an unsolved problem to date. But then as Damien was saying, how then you apply it really comes down to you need to know more than just that final position. So even if you find that final document, what led to that outcome? And that's not captured anywhere. I mean, yes, it's kind of captured in the history of the versions of the document. But no, no, nobody's noting down like,"This is why we changed this clause." And so, like, for the training, for the knowledge that's trying to be, be, be used by the lawyers, uh, we're not even capturing that. No one's capturing that. So how do we, like, train or do any of it? So that's an open question. I don't know the answer. Um, just to say I think we have a data problem anyway. Yeah I, I love that. And Ian, I'd like your thoughts on that. What, uh, Horace just said, I've been thinking about a lot. In that, we at Clio, we record every single meeting. And because we record every single meeting, we're able to capture what is what formerly called data exhaust. That is add the speech is the context. So everything Horace you were saying that how did we get to this end point of the contract, or how did we get to this litigation decision? Those are all done in meetings. And at Clio, literally, I just read it before this call, we recorded a one-hour session, and we hashed through the pros and the cons of something. And then we're gonna combine that with the five other meetings we had before, throw that into Claude. And it's gonna give us an output that is gonna be way better than just the end document. So I would say that the context is the thing, and the where law firms are really going to fall down is privilege. And the reason for that is because lawyers are reluctant, they refuse to record things because of the liability that might ensue. Both the liability of if my client says something that is, uh, you know, inculpatory, right? Then that could be used as evidence. That recording could be used as evidence against them. Lawyers hate to record things. But because they hate to record things, they're gonna lose all the context that we have at Clio. And all the context that is now able to help them take the way that we got to this contract or the way that we got to this motion. All of that context goes out the window. And so I think that both the value of recording is going to be… the fact that a recording is a treasure trove. A treasure trove for good to be able to capture good data exhaust, and a treasure trove for bad, that is, the litigators can maybe get access to that treasure trove. I think that's going to keep us from capturing data exhaust. What do you think, Ian? Yeah, yeah. I, I, I think I agree, and Horace, that's a very important and hard question, and I think that I come back to what's the point of in-person training with a firm? It's to deliver that information that you said is missing. It's not to deliver a 30-page PowerPoint they've been using for 30 years that explains the basics of XYZ. Otherwise, all that knowledge is totally lost.'Cause no one's gonna, no firm in reality is gonna spend the time to download everyone's internal knowledge, make videos on it, keep it… It's not gonna happen. The, the training programs themselves need to be the delivery of that information, not the core concepts. You know, you got us and you got other resources for that. So that's what I'm trying to, you know, we work on firms to try to, try to get them to see the good stuff, and that's what the associates want. I mean, every single time, that's, that's what they crave. My training programs at Robins Kaplan largely consisted of the senior lawyers, maybe 70-year-old lawyers that now have their FU money and came back to teach the younger generation. They, they essentially told war stories saying, "When I d- took a deposition here, here's the bad things that happened." And I still remember a 70-year-old lawyer said that, "If you have bad facts in your case, put it up right up front." He said, "Put the turd on the mantelpiece." And that is such a vivid memory of training, uh, to be-- that has still stuck with me 25 years ago. Like, That's great on the mantelpiece is something that I still remember. So anyway, so good training is largely war stories that stuck Yeah the senior lawyers' brains that maybe make its way into training. So Ian, how do you incorporate war stories into your training? So what we try to do is take all of the foundational stuff and put it in advance in the pre-learning. That's what the videos are for. That's what, point them to articles, whatever. It's not for the in-person. So what we do is, look, partners are busy, so the reality is what we say to them is, "We can save all your prep time. Show up and talk about yourself and your war stories." They love it. They don't want to hear about the science of learning and flipped classrooms and all that business. We give them talking points. We g- actually, it's like Blue Apron for legal training. Here's your hypos, here's your questions, here's your prompts to get to the war stories. That's it, they, they love it. They love it, they love it, they love it. So we just try to give them the materials to do it. It's the same with the workshops. It's everything in a bow tied up for you. Show up and talk about your own matters, and it's always a huge success. I love it. So Ian, we like to close out our discussions, every time… everyone talks about pessimism, but we like to talk about optimism. We like to talk about what we're optimistic Love it about. We ask our guests, I'll give mine, and then Horace can give his, which will give Ian time to think about his. What are we optimistic about? Horace, since the last time you and I talked, Fable has gotten me very optimistic. Anthropic's new model is, is, uh, Okay? uh… it's gotten me optimistic for two reasons. Number one is because they released Fable, and then all of a sudden the government made it take it away. So there's this, Put it away. Yeah of scarcity, like, "My God, this is so powerful that I can't use it anymore 'cause the government says it's too, too dangerous." And then they give it back to us. And then what Anthropic did was to say, "Okay, you only get half of your usage for, of Opus." And I have my 20X/200 dollar plan."So you only get half of your usage for Fable, and it's gonna go away in a couple of weeks." So anyway, what happened was I've been token maxing on Fable over the last two weeks, and I've been doing things that I've just never been able to do in the past. And I've just said to Fable,"Hey, I have 20 repos that are all in various states of disarray. Go through those 20 repos, do all the things that are undone, and for all the things you're waiting on me for," 'cause I've, uh, it's been waiting on me for a lot of things, "just do all those things. Do all the user acceptance testing. Run all the scenarios. Run the 950 tests for each of the repos to make sure that it works well. Do all the things." And Fable just effing did it. Like it did, went through all the repos. And then after it did it, I said,"Okay, what are features that would be helpful for all 20 repos that I haven't had even thought about?" And it gave me a whole bunch of them. I said, "Yeah, do all those things." And Fable effing did it. Anyway, so, so I would say that I'm optimistic that I just described is not even on the radar of any lawyers. And I, I think that, that this, this kind of agentic just go out in the world and do the things, um, is just an unlock that we didn't even have three weeks ago, four weeks ago, that is now here today. And the last thing as I stop ranting is, uh, my, uh, I've talked about my, uh, litigation with my next door neighbor. Uh, for Ian's benefit, my next door neighbor wants to build a fence six inches from my house. Um, so I used Fable to be able to go through all of my documents and say, respond to discovery requests, and do it in a way that actually cites every single document, that I have. And by the way, go into the world and find all of the aerial photos of our yards to show how my Oh from their yard. And, uh, it pulled in three hundred and fifty different photos from all sorts of sources that could I have pulled in three hundred and fifty sources from the Minnesota Historical Society and from the US Geological Survey and, and, and, or would I have taken the time? And the answer is no. But I just told Fable to do it, and all of a sudden I have Bates numbered photos that I can be able to dump on the other side and say, "Look at, here's all the documents and all the photos and all the evidence that is gonna win my case." The fact that I could have Fable do that is not even on the, the roadmap of any… So anyway, so I'm, I'm, I'm really optimistic about the, the present, not just the future, but the present of what the, the models are capable of and teaching that to our lawyers. Uh, Horace, what are you excited about? Optimistic. First, first, I, I am, I am, like, stunned that Fable can actually pull, like, 300 photos. I did not know we can do that. I knew about the whole coding thing. I knew about lots of the capabilities, but that's cool. Uh, I don't have anything to try that on, but, uh, I'll have a think about it. Um, what I'm, you know, to, to your point, what I'm optimistic about is, uh, the, the presence of competition and how it's gonna benefit everybody for a while longer yet. Uh, so, like, Fable was kept around for… not for free, but at a discounted rate for a little bit longer because OpenAI has gone and released 5.6 Sol. And, and I think, like, that's what's happening in the broader sort of, like, tech landscape, but also it's happening inside of legal right now between Harvey, Legora and everyone else. So for a little while yet, consumers will still win out, and I, I, I like that. I really, really like that. Um, Ian, over to you. Optimism Optimistic about Damien's chances in his suit. I mean, gosh, that's the one I'm most optimistic about. I'm optimistic about where the role of the lawyer going forward, right? I mean, it's a good thing that this drudgery stuff is sort of going away and that we're finding more efficient ways to get people up to speed. I, I think, you know, I certainly didn't go through three years of law school to slog through diligence. I think I'm optimistic about where things are headed and about the market's attitude towards training and developing lawyers and staff in general. It just, this week alone, you know, I'm, I've got calls with heads of major practice groups, with C-suite folks about training, so I'm just optimistic where it's all headed. I think training was always broken, it always needed fixing, and the economics and the tech gave it a kick in the pants. More than a kick. Um, so I'm optimistic about where it's all headed. Ian, what a joy to have you on the pod. I, I think that educators are not paid well enough. That is true for high school and and college educators, and it's certainly true for legal educators. So thanks for doing the good work and important work, that is also underestimated, but should be now pleasantly estimated. Thank you for doing this hard work thank Yes really been a treat to chat with you guys Oh, it's amazing to have you. And hey, by the way, I think I came up with the name of the episode just now. It's gonna be "Choo Choo Choose Your Training". I think I can. I think I can. I think I can. Love it. Thank you, guys. Oh, thank you everyone. And thanks for listening everyone. Thanks everybody Thank you.