Do you still need a purpose-built legal AI solution like Ruli now that Claude Cowork and the Word plugin exist?
I tested Ruli, a legal intelligence platform built for lean in-house teams, against Claude Cowork and the Claude Word plugin on 4 product counsel workflows — advising on a footwear company's pivot to AI (totally not inspired by Allbirds).
This video was made in partnership with the featured vendor. The vendor provided product access and reviewed the video for factual accuracy. Editorial opinions and conclusions remain those of Legal Benchmarks.
00:00If you're working in a lean in-house legal team, wearing multiple hats across contracts, product, and compliance with very little outside support, then this video is for you. Hi, my name is Anna and I'm a lawyer trying to make sense of AI for the legal profession. Today, I'll be breaking down the three core capabilities of Ruly, a legal intelligence platform built for in-house legal teams. I'll share what I like and what I think has room for improvement. And you'll want to stick around because I will be comparing it against a Claude and Claude's new Word plugin. As I was making this video about Ruly, Claude
00:31launched its Microsoft Word plugin. At first, I was not impressed. Check out my reaction to the demo video here. After trying it out, my views have shifted, but let's not get ahead of ourselves. Let's dive into it. Let's take a look at how a tool like Ruly can help with both the specific research questions sitting on your desk today. Let's say I am an in-house counsel at a shoe company called Everybird and I've just been handed the product requirements from business to pivot from a footwear company to an AI-powered shoe intelligence company. This is a shocker,
01:02right? But not so much if you've heard of Allbirds' recent pivot from a footwear company to Newbird. So, here's what we know about the new product launch for Everybird called Project Fitbird. Each shoe will be embedded with chips that allow the users' Fitbird app to track wear patterns, generate fitness insight, and collect biometric and health-related data. And it will be launched in the US. As a product counsel working in a very lean legal team, one of the questions I would need to figure out is what kind of updates I would need to make to the categorization of
01:33personal data that we collect from users, given we are collecting biometric and sensitive data through the AI chips implanted in each shoe. So, let's start with the Ruly assistant on the homepage and throw that question to Ruly. I'm going to enable voice mode for this because I am very busy and don't have the patience to type. Hey Ruly, under US privacy law, what updates do we need to make to our privacy policy in order to launch Project Fitbird? To overexplain what Project Fitbird is because I can actually attach the files context. And
02:03since Ruly is integrated with Google Drive, I can pull this context directly from my Google Drive folder, which I am going to do right now together with my privacy policy. So, once that is fully uploaded, another really cool feature is Ruly has its own legal database where it has indexed US federal and state law and also UK and EU law. I'm going to go ahead and select US law since we are launching in the US. Let's see what we get back. We have a pretty comprehensive privacy memo that's giving me concrete steps I need to take as well as the
02:33state-specific requirements. And what's really cool is by default, without me even prompting, it gave me the privacy policies of other market players that already offer a similar product for me to reference. What's also really helpful is I can hover over the source citation and actually see a snippet of the source or further click into it. And Ruly is not just going to send me off to a external website and make me hunt and verify all of the sources myself. Many of these citations, you can actually see
03:03the relevant federal or state law that Ruly has indexed in its database. In some cases, it even highlights the text and the relevant portion of the law for me, which makes the verification burden much lower. But for some other sections, I noticed this highlight does not appear. And as I go through these sources, I would love to see if the cited law is currently up to date, which I can't really tell because that is also part of the verification as laws do get outdated. Another super useful feature is monitor, which lets you set up a recurring tracker or horizon scan for
03:34monitoring regulatory changes. One flag here, for the time being, you can only run it on a weekly or monthly basis. So, now that I know that I would need to update the privacy policy, next thing I want to know is what language are other market players that's already collecting this kind of data is using in their privacy policy. Now, I'll show you how you can build your custom table to analyze this using Ruly's data grid. It's super easy for me to create a data grid to analyze the privacy policies. And if you don't know where to start, there are already a few templates that you can just select and customize from.
04:05I'll go ahead and create my own. So, next, I'll just give it the instructions here, privacy policy analysis. Very simple. You don't need to overthink this at all because you can always come back to it to customize. So, don't feel stuck, just move forward. I'm going to go ahead and create this table and I'm going to add the privacy policies that I found from other market players. So, once all of the files are uploaded, you can click on this really magical-looking button and see what happens. And a bunch of puppies rushed out. No, I'm just kidding. So, the results are generated and I can also expand and see it in
04:36full-size view. And when I click on each cell, it will also take me to the source, so I can verify the information. What's really cool about this analysis is it's automatically saved in the data grid and can be later used as context. I'll show you what I mean. Let's talk about how we can operationalize the regulatory research and also the market analysis that we just did into our own privacy policy document with Ruly assistant in Word. Of course, you can prompt the assistant or alternatively, what's really cool and how everything
05:06connects together is you can add the data grid privacy policy analysis as additional context for how you draft our privacy policy. So, here's how you can do it. I'm just going to add as the context, select from data grid the privacy policy analysis. I will also add the PRD as background context. So, the output includes results and also a comment that you can apply. However, one thing I have to note here, these suggestions are more of a black box and I cannot directly verify the source themselves for both the comments and the
05:36redline. So, lawyers, we still have a job to do here. We cannot automate this entire process, just click accept all and think that our job is done. Apart from reviewing and updating privacy policies, Ruly's Word assistant can review agreements against your team's playbook and also flag issues outside of the playbook, so you get a full review. And now for a question that I know is on everyone's minds right now, which is how much of what I just showed you Ruly is something that Claude already can do. And if we were to put these tools side by side, how would they perform in legal-specific tasks? I'll show you. So,
06:08I ran the same Fitbird project launch scenario through Claude and tested it through the three core capabilities that we just ran with Ruly. And to be fair, the sources it pulled were from government websites and from law firms, which is pretty decent. But this is where the experience breaks down. When I click on the citations to verify it, unlike Ruly, it sends me to the source and leaves the verification work to me for the most part. So, that means more manual checking, more friction. For regulatory monitoring, Claude does not really ship this as an off-the-shelf feature. But you can rig something together with scheduled tasks in Claude
06:40Collab. I won't go deep into it today, but I'll show you quickly where you can do that. You can create a scheduled task and choose how often you want it to run, as frequently as on an hourly basis. But there are a few limitations. So, first, this only works in Claude Collab and Claude Code. And as you can see, there are no templates and no guided workflows like you can see in Ruly. So, you're basically staring at a blank canvas and you have to know what exactly you want to build in order to build it. Claude can also help you with multi-document data analysis and help you build a
07:10table. I did so here with the privacy policies, but again, the difference for me is traceability. In Ruly's data grid, every cell is traceable back to the exact source, whereas in Claude, I am only shown the page number of the source and it doesn't even have a hyperlink, so I would have to manually check and verify each output in the source material myself, which is very time-consuming. And finally, for the Word add-in, which a lot of people have been really curious about, I ran a quick test. Both Ruly and Claude now offer a Word add-in that helps users draft,
07:40review, and redline directly inside Microsoft Word. On the surface, they could look similar. But when I tested them on proofreading this Everybird privacy policy, there was a meaningful difference. In this Everybird privacy policy, I had already identified 41 proofreading issues. Out of those 41 issues, Ruly was able to catch 27 off-the-shelf, which is about 66%, while Claude's Word add-in caught 17 out of the 41 issues. In practical terms, Ruly
08:10caught 10 more errors than Claude in this document. I won't go into much detail today, but you can improve the review quality of both Ruly and Claude. In Claude, you can build your skills for proofreading that will help you improve the output accuracy rate. So, here's my honest opinion after I ran Ruly and Claude side by side. Claude is a great option for builders who knows what they want and can utilize a suite of tools that Claude gives you to customize your workflows. Ruly, on the other hand, has already pre-built a lot of those
08:41workflows for you. And it's very good at anticipating what a general user wants
Speaker change: and guiding them through the process. So, it's super easy to get started, but it doesn't offer the level of flexibility that some advanced users may be looking for when they want deeper customization. That said, no AI tool is perfect. My view is that lawyers should be experimenting with both general-purpose tools and also legal AI solutions because they can be very much complementary. And that's also
09:11what we're seeing in the market right now. Legal teams are building their AI tech stacks with both general-purpose tools and also legal-specific tools. But enough from me. I also want to hear from someone that is using Ruly on a day-to-day basis. So, I spoke to Lori. I am general counsel and I lead our legal compliance functions at Amatic Insurance Services. I have two compliance managers that report directly to me. We selected Ruly because, one, they did a nice job creating output that we could
09:41leverage and that would be beneficial. We thought it could help us save money, help us mitigate hiring an individual who might become disengaged quickly. One of the other components is they're just really great to work with. Recommendations that we made were implemented almost real time. So, we knew that this was going to be a partnership and we were going to grow together. I utilized Legal Assistant very heavily. Pulled up my metrics, our usage metrics across my team since we implemented in
10:11June of last year. We have saved 7,788 hours because we use this tool. If we were to outsource that to outside counsel at $500 an hour, that's a savings of $894,000. It has enabled us to do the initial assessment and be very intentional about the questions that we ask outside counsel. Really can help you identify what you should be asking and for individuals who aren't comfortable or don't know what to ask, this will help. Taking a step back, Ruly isn't
10:42trying to win on any single feature. Instead, what it does really well is help lawyers surface the context they need to get really decent outputs moving them through research, analysis, to drafting and review. If you're working on a really lean team and you're just tired of transferring the context from the different tools that you're currently using for work, or you don't have the resources to set up your own custom workflows, then Ruly is a great option to explore. So, that's it for today's video. I hope you enjoyed it and if you do, hit the
11:13subscribe button and leave a comment below to share what other tools I should check out. I'm also curious if you already have a legal AI tool, how it performed on the proofreading task. You can find the link to the proofreading test below in the comments and run the test yourself. If you want to find out more about Ruly, do remember to check out the founder interview I have with Brian, the CEO, and until we meet in the next video, I hope you stay healthy and curious. Ciao.
