Note: This post is an adaptation of what started initially as a Twitter thread.
I’ve been going pretty hard on DoNotPay and its founder/CEO Joshua Browder for the past couple of days, and I’ve had a lot of people defending the service, saying that it could be a real boon to those who can’t otherwise afford legal aid.
So, I thought maybe I should give it a fair shake — after all, I’m mostly arguing with what my idea of a “legal AI” is, right? So I signed up for an account at DoNotPay and took the service for a little whirl. There’s no option to test out the real-time AI, or at least not one that I could find. But the site does offer a dazzling array of services under the category “legal tools.” I used the site to undertake three different tasks: Defamation Demand Letter, Divorce Settlement Agreement, and Sue Anyone in Small Claims Court.
Those of you who know me will be unsurprised to learn that I started with “Defamation Demand Letter.” I was a little taken aback by the description “File a demand letter” — you don’t file a demand letter, you just send it — but decided to press on.
The site leads you through a fairly basic set of question and answer prompts, asking you to identify the potential defendant and the defamatory statements, state the basis for their falsity, explain how they damaged you, etc.
After I filled in the prompts, which required my full name, address, and phone number, I pressed “next,” eager to see the “expertly drafted demand letter” DoNotPay generated on my behalf with the “most relevant state legislation regarding defamation” based on my location.
Instead, I got a little progress bar, informing me that my defamation demand letter would be ready in an hour.
That . . . seems a little slow for something that is supposed to be able to respond to a judge and give instructions in real time. But whatever, let’s press on.
I went back to the front page of the site to select “Divorce Settlement Agreement,” observing in passing that DoNotPay describes itself in full color as “the home of the world’s first robot lawyer” and promises the ability to “sue anyone at the press of a button.”
Once again, I stepped through my various prompts, giving information like my address, my putative soon-to-be-ex’s address, our comparative incomes, the number of hypothetical children we had, etc. so that I could get “the fair terms [I] deserve.”
This time, after I was done, I got a little “task progress” bar that said that my divorce settlement would be ready in EIGHT hours. Y’all, eight hours seems like a really, really long time for an AI to need to generate a document. But, whatever, we’re giving it a fair shake.
Time to Sue Anyone in Small Claims Court! I typed “Sue Now” into the search bar and was greeted with the following prompt:
This gave me significant pause. First of all, it says “I’m owed $500” at the top, but I haven’t entered any specific information yet. Second, “generating court filings” and giving people “a script to read in court” is… I mean, that’s the practice of law. It just is. Third, though, just from a UX perspective, the prompt ends with a yes or no question (“Are you ready to proceed?”) but has a text box which will only accept a dollar amount. What am I supposed to enter here?
So I entered a dollar amount and pressed “Enter.”
The next page presented what I thought was an odd question. Why would I have received a demand letter? I’m the one suing, right?
(Later, from context clues, I managed to figure out that what they meant was if I had already generated a demand letter with DoNotPay. I guess you can only use this service to create filing documents if you have already used it for a demand letter. But I digress.)
Another UX note — I originally found this prompt from an article entitled “Sue Anyone for Assault in Small Claims Court,” but “Assault” isn’t one of the options listed here. So I figure, go with what you know.
Ooooookay. I… this is not how I would ever phrase this question. But I’m not a lawyer at all, never mind the World’s First Robot Lawyer. Let’s press on.
Fortunately, these are the only types of contracts where you could ever have a breach and be owed money damages, right? Right?
At this point in the process, I seem to suddenly have been switched from drafting a lawsuit to drafting another demand letter. Not sure when that happened — the service certainly didn’t notify me about it.
But in for a penny, in for a pound. As requested, I selected all the appropriate dates for my hypothetical breach of contract: contract formation on 9/6/2019, due date on 12/6/2019, most recent request for payment on 11/11/2022, final due date of 1/31/2023. As requested, I gave details of what services I had theoretically performed.
. . . and got this extremely puzzling prompt in reply. Photographic evidence? What? At no point did the service ask me to provide any documentary evidence or give any terms of the contract other than the date the payment was due.
And HEY PRESTO! This time, no timer, no progress bar, just an instant PDF or Word demand letter to download!
The PDF document had all my personal information in it and was somehow in a format that could not be redacted. I’ve never seen this behavior before — it persisted through all my tricks like saving it as an archivable PDF and everything.
The author of the PDF was listed as some guy named Platon Konstantinos Mazarakis.
Out of curiosity, I did the zip trick on the Word file so that I could look at the properties without opening it. Interestingly, it has some custom XML in it; I’m not much of an XML expert, so I asked software engineer Debbie Mia to take a look at it. The docprops/core.xml file has Mazarakis as the document author once again but also shows that this Word document, generated by the World’s First Robot Lawyer, was last edited by “Denise.”
In order to get the PDF into a redactable format, I had to export it to TIFF and then reconvert it. There’s a big loss in quality, unfortunately. But let’s take a look at this demand letter, generated for me when I asked to sue someone in small claims court!
There is literally nothing AI about this. This is a straight-up plug-and-chug document wizard, and it is not well done at all.
I will admit, I threw it a couple of curve balls, like placing my defendant in Canada. But it didn’t attempt to do any kind of jurisdictional analysis at all — I could have put him on the moon and it wouldn’t have mattered. There is no date on the first page of the letter.
There was no widow or orphan control. After my “signature” there’s a space for “Additional Remarks,” which it never prompted me for. There’s also a random orphaned date in the second page header.
Did the contract say I would be paid for OFFERING the following services, or for PERFORMING them?
In addition to there being zero jurisdictional analysis, there’s also zero thought to engage with the fact that depending on the choice of law in the contract (which the prompt also didn’t ask me about) this action may well already be time-barred.
Or with the fact that Small Claims Courts have an upper limit on the amount you can claim, and I’m pretty sure $17K exceeds it. And, yes, that fucking orphaned “Sincerely,” is still killing me.
Also also? The prompt never asked whether I was amenable to a payment plan. Just went ahead and hucked that in there on its own. (Also never asked me about the interest rate, I have no idea where it got that.)
Let me be clear: this is a terrible demand letter. Absolutely terrible. Useless or worse than useless — if an actual attorney saw this, she would instantly know that the sender was unsophisticated, unrepresented, and gullible as fuck.
For this service, DoNotPay charged me $36 ($18/month and it charges you for two months at a time). But hey, there’s an unlimited number of documents we can generate, right? Let’s go see how the others are doing!
Turns out… not so great, Bob. The minute the defamation letter hit its one-hour limit, it flipped over to a little clock icon and said it would need more time. Same for the divorce settlement’s eight-hour limit. That was two days ago.
No updates since then. No contact from the company, no way to generate a support ticket, no hint as to what the trouble is. Just a cheerful, brightly colored promise that it’s doing its best.
I have literally no way to know what the fuck is actually going on here, but I can think of two likely options. The first is that the whole tool is just fucking broken, and Joshua Browder is scamming people out of almost $20 a month for a service that simply does not work. The second, though — and I find this much more likely based on the one-hour and eight-hour timelines given — is that this isn’t AI at all; DoNotPay collects the information from the prompt and then hands it to a human to go find the relevant law and customize the doc.
That would explain why the defamation demand letter gave me a one-hour deadline while the breach of contract letter happened instantly. Remember, the defamation demand SPECIFICALLY promises the “most relevant state legislation” “based on your location.”
It doesn’t take an AI an hour to look up the relevant law based on a physical address. Westlaw can do it… well, I was going to say instantly, but anyone who has used Westlaw knows that’s a lie. But Westlaw can do it in thirty seconds. If there are human beings doing the Googling and making the decision as to what the “most relevant state legislation” is for a defamation action (and, let me tell you, there’s a sign of a pretty glaring error just in that phrase on its own) then any argument that this isn’t unauthorized practice of law (UPL) goes straight out the fucking window.
But hey, maybe those aren’t the only two options! Maybe Browder recognized my usage of the site — after all, I did sign up for it with my real name, which I also tweet under — and panic-blocked me because he was salty that I think his idea is dumb. Or maybe there’s some other perfectly innocent reason why this brand-new revolutionary technology that is getting its debut in a live courtroom setting in less than a month has been wedged for almost 48 hours with no updates or signs of life.
Regardless, though, I don’t think any of the lawyers I know have any cause to fear for their jobs any time soon.
P.S. Browder, who had blocked me a few days ago, unblocked and followed me today!
But apparently only for long enough to promise to listen to feedback, and then to block me again. So, you know, good talk etc.
Awww, and the robot blocked me too!
Update: Josh Browder unblocked me and messaged me again, explaining that my account was automatically flagged because my usage “didn’t seem like authentic activity” and informing me that he had refunded my subscription. I asked him if I could see the two other documents I had already generated or unlock my account so that I could generate similar documents; he said he would try to make them available by tomorrow afternoon (January 25th), explaining that “[t]he engineer who understands the blocking code is out until 12 or so.”
Josh also stressed that “the letters aren’t being typed out by hand and in general are all generated instantly 
.” Although I did not claim, nor did I intend to suggest, that the documents on DoNotPay.com were typed by hand, I have included his statement here in the spirit of good faith and full disclosure.
