We already wrote a long story looking at many of the eye-opening claims from Peiter “Mudge” Zatko in his whistleblower report regarding Twitter’s security operations, and the possibility that the company both has shit security practices and violated its FTC consent decree regarding those security practices. As I noted, the report is a mixed bag of things that sure sound pretty serious, and a few that are greatly lacking in context. Some of them might be really bad, but might not be quite so bad if we knew the full context.
This post just focuses on the first claims in Mudge’s report, which (honestly) seem to have been written more to jump on the current news cycle than to address an actual issue at Twitter. It’s entirely unrelated to the other claims in the report, but instead is focused on the question of Twitter and spam/bot reporting. And… it’s weird. It is framed as though it supports Musk’s claims that Twitter is lying about spam. But, the details actually show the opposite.
The media is, unfortunately, falling for the spin. The media is covering it as if the claims about spam and bots help Musk.
But that’s just reporters buying into the framing, and apparently not really understanding the details.
The lawsuit is not about spam:
So, let’s dive into those details. The first and most important thing to remember is that, even as Musk insists otherwise, the Twitter lawsuit is not about spam. It just is not. I’m not going to repeat everything in that earlier story explaining why not, so if you haven’t read that yet, please do. But the core of it is that Musk needed an escape hatch from the deal he didn’t want to consummate and the best his lawyers could come up with was to claim that Twitter was being misleading in its SEC reporting regarding spam. (As an aside, there is very strong evidence that Musk didn’t care at all about the SEC filings until he suddenly needed an escape hatch, and certainly didn’t rely on them).
But — and this is kind of important — many of Musk’s claims were based on either misunderstanding or deliberately misreading Twitter’s SEC filings. As I’ve explained multiple times now, what Twitter reports to the SEC is how much spam is likely included in their “monetizable daily average user” (mDAU) accounting. This is not, and has never been, about “how much spam is on the platform.” The company came up with this other metric — mDAU — that is a segment of the total Twitter population. As Mudge’s report notes, an mDAU is defined as a “valid user account that might click through ads and actually buy a product.”
That’s not every account. There are accounts that are inactive. There are accounts that are automated (but useful — such as those tweeting out the weather or earthquakes or whatnot). There are lots of accounts that may exist on the platform, but may not be counted in mDAU. And that includes some spam/bot accounts. That has always been clear for anyone who reads the details.
Spam in the mDAU is not the same as spam on Twitter:
Next, Twitter’s filings with the SEC are only about how much spam is in their mDAU number. This takes place after Twitter has made use of other processes to try to eliminate spam accounts from the mDAU, and then they do a daily spot check of 100 accounts. That creates a sample size of 9000 over the course of a quarter (the time period between Twitter reports), and is statistically significant for declaring that less than 5% of the mDAU is spam.
Again, this has never meant that less than 5% of all accounts, or all tweets, or all activity is spam or bots. It just means that less than 5% of what is counted in their mDAU number is.
On top of this, Twitter caveats its SEC filings around this admitting that this process is highly subjective and could be inaccurate. It does this at great length:
While these numbers are based on what we believe to be reasonable estimates for the applicable period of measurement, there are inherent challenges in measuring usage and engagement across our large number of total accounts around the world. Furthermore, our metrics may be impacted by our information quality efforts, which are our overall efforts to reduce malicious activity on the service, inclusive of spam, malicious automation, and fake accounts. For example, there are a number of false or spam accounts in existence on our platform. We have performed an internal review of a sample of accounts and estimate that the average of false or spam accounts during the fourth quarter of 2021 represented fewer than 5% of our mDAU during the quarter. The false or spam accounts for a period represents the average of false or spam accounts in the samples during each monthly analysis period during the quarter. In making this determination, we applied significant judgment, so our estimation of false or spam accounts may not accurately represent the actual number of such accounts, and the actual number of false or spam accounts could be higher than we have estimated. We are continually seeking to improve our ability to estimate the total number of spam accounts and eliminate them from the calculation of our mDAU, and have made improvements in our spam detection capabilities that have resulted in the suspension of a large number of spam, malicious automation, and fake accounts. We intend to continue to make such improvements. After we determine an account is spam, malicious automation, or fake, we stop counting it in our mDAU, or other related metrics. We also treat multiple accounts held by a single person or organization as multiple mDAU because we permit people and organizations to have more than one account. Additionally, some accounts used by organizations are used by many people within the organization. As such, the calculations of our mDAU may not accurately reflect the actual number of people or organizations using our platform.
Musk’s entire complaint is that he relied on the SEC mDAU filings, and that THOSE are wrong
As we’ve described, this is a weak sauce argument that is meaningless. Musk claims publicly (and sort of gets at it in some of the legal filings) that he believes “spam on the platform” is more than 5%. But the (already weak and irrelevant) legal argument he is making is that Twitter lied to the SEC in saying that the spam in the mDAU is less than 5%, and that this lie will create a “material adverse event” (MAE) that allows him to scuttle the deal (again, the spam issue is not an excuse to violate the deal — only an MAE).
As we’ve discussed, all of this is nonsense. Musk seems to be — either deliberately as an excuse to get out of the deal or because he doesn’t understand some fairly basic things — conflating “spam on the entire platform” with “spam remaining in the mDAU.”
He has done this multiple times in public, and it has influenced at least the court of public opinion — many of whom actually believe that Twitter only reports that less than 5% of the platform is spam (something it has not reported at all).
Mudge’s whistleblowing report actually confirms Twitter’s position, while pretending otherwise:
And here we get into the specifics of Mudge’s report. He first trashes the entire mDAU concept, noting that it’s a scam in and of itself, in that it basically allows Twitter to fudge the numbers.
Until 2019, Twitter reported total monthly users, but stopped because the number was subject to negative swings for a variety of reasons, including situations such as the removal of large numbers of inappropriate accounts and botnets. Instead, Twitter announced a new, proprietary, opaque metric they called “mDAU” or “Monetizable Daily Active Users,” defined as valid user accounts that might click through ads and actually buy a product. From Twitter’s perspective “mDAU” was an improvement because it could internally define the mDAU formula, and thereby report numbers that would reassure shareholders and advertisers. Executives’ bonuses (which can exceed $10 million) are tied to growing mDAU.
Okay, there’s a bit of editorializing here, and I can see there are reasons to be skeptical of mDAU in general. But there are also reasons to think that (as Mudge admits) not including “large numbers of inappropriate accounts and botnets” is actually… good for shareholders and advertisers in not confusing them that those accounts might actually be monetizable. Yet there are definitely questions about how Twitter might be able to goose the mDAU numbers to its own advantage and maybe “smooth out” bumps in the road or whatnot. Not saying that’s definitely the case, but it’s a risk when you get to define stuff.
That said, Mudge also admits that Twitter is incentivized to not count spam in mDAU:
Executives are incentivized to avoid counting spam bots as mDAU, because mDAU is reported to advertisers, and advertisers use it to calculate the effectiveness of ads. If mDAU includes spam bots that do not click through ads to buy products, then advertisers conclude the ads are less effective, and might shift their ad spending away from Twitter to other platforms with higher perceived effectiveness.
So, as a start, that contradicts the claims of Musk and his fans that Twitter has incentive to look the other way when reporting spam in the mDAU because it benefits the numbers. As Mudge notes, Twitter has incentives not to count spam in the mDAU.
And then he puts forth his argument for why he thinks Musk is correct — even though it’s actually confirming that Twitter is correct:
However there are many millions of active accounts that are not considered “mDAU,” either because they are spam spam bots, or because Twitter does not believe it can monetize them. These millions of non-mDAU accounts are part of the median user’s experience on the platform. And for this vast set of non-mDAU active accounts, Musk is correct: Twitter executives have little or no personal incentive to accurately “detect” or measure the prevalence of spam bots.
So… I’m really confused by this section, and the claims that “Musk is correct.” Because in court they’re not arguing about how much spam is on the overall platform. They’re arguing about how much is in the mDAU. So, rather than supporting Musk, this paragraph simply confirms exactly what Twitter has been saying in SEC filings and in court. What it reports to the SEC is an estimate of how many spam accounts slip through other processes, and are inadvertently counted in the mDAU.
That is all Twitter has ever claimed in a legally binding way.
And here, Mudge is confirming that Twitter is not just exactly correct, but also that it is incentivized to behave exactly this way, and not at all how Musk has described.
The fact that Mudge is saying that there are spam accounts outside the mDAU… is the very point that Twitter has been making and that Musk keeps misrepresenting. mDAU does not include all accounts on the platform. And the only way in which the spam counting could even be remotely relevant to the case (and, again, it’s not) is if Twitter made a material misstatement to the SEC.
And Mudge doesn’t claim that at all. Rather he backs up Twitter’s claims that mDAU is not all user accounts and that Twitter has incentives to keep spam out of mDAU to keep advertisers happy. That supports Twitter’s legal argument and kicks the legs out from under Musk’s.
I simply don’t understand why anyone — including Mudge — thinks all this supports Musk.
I also question the claim that Twitter has no incentive to remove the spam accounts that are outside the mDAU. It seems fairly obvious why they still have incentive to try to tackle that problem as well: because if the platform is overrun with spam and bots then that will drive users away. Those users are in the mDAU and so having too much spam on the platform (even outside the mDAU) drives down the mDAU. That’s just kinda common sense.
Mudge then tries to justify how this supports Musk, but it… just gets worse and again seems to support Twitter. Mudge is a great security researcher, but when it comes to spam stuff, it’s not clear he has a firm grasp on how some of this works.
In fact, Mudge learned deliberate ignorance was the norm amongst the executive leadership team. In early 2021, as a new executive, Mudge asked the Head of Site Integrity (responsible for addressing platform manipulation including spam and botnets), what the underlying spam bot numbers were. Their response was “we don’t really know.” The company could not even provide an accurate upper bound on the total number of spam bots on the platform. The site integrity team gave three reasons for this failure: (1) they did not know how to measure; (2) they were buried under constant firefighting and could not keep up with reacting to bots and other platform abuse; and, most troubling (3) senior management had no appetite to properly measure the prevalence of bot accounts–because as Mudge later learned from a different sensitive source, they were concerned that if accurate measurements ever became public, it would harm the image and valuation of the company.
This also seems really odd. First of all, points one and two are basically life for every site integrity team in every company ever. It’s the nature of the role that it’s mostly all firefighting, and little time for larger perspective. But, more importantly the first point is key. If Twitter knew how to count all the spam on the platform, it would know how to eliminate all the spam on the platform. The company has a bunch of methods that try to limit spam, and as we’ve discussed it kills hundreds of thousands of accounts every day.
Clearly, some make it through, but “not knowing” how much spam is on the overall platform is not the smoking gun people seem to think it is. I have no idea how much spam makes it through into the Techdirt comments. It’s not as easy to count as some people think. If I did know, the answer would be zero, because I’d delete it all.
As for the claim that senior management has no appetite to measure it because it would harm the valuation, that could be, but still seems kind sketchy. First, without details on the “sensitive source,” it’s difficult to judge the credibility of the claim. Second, given that the company has spent years focusing on mDAU for exactly this reason, it’s not at all clear how revealing how much spam was on the actual platform would… impact anything? After all, the company is already focused on reporting the numbers of the users that matter for revenue purposes.
Even the Board of Directors understood the counterproductive incentives in place: In or about the Q3 2021 Board Risk Committee meeting, a Director asked why more progress has not been made around bots and related harmful content on the platform. Our client remembers an executive of the company admitting to Board members that the company had “intentionally and knowingly deprioritized” platform health to focus on growing mDAU. Afterwards, a different Twitter leader who had witness the exchange commented to Mudge, in reference to this admission, “it is very strange what this company does not share with board members, and then some of the statements that they do.”
Again, I’m confused as to what this is supposed to reveal. Wall Street — mainly Elliott Management, had literally forced Twitter to change its plans to increase its mDAU growth numbers. And that included Elliott Management’s seat on the Board. If the Board is forcing the company to grow its users, then of course the company is going to focus on growing the userbase over issues that seem secondary like “platform health.” We can argue if that was the right decision — and whether it makes sense in the long term — but the fact is that the Board and the company’s largest investors were ordering management to focus on user growth, not things like dealing with spam.
I mean, literally, the agreement with Elliott was that Twitter promised “to grow its average number of daily users who see ads by 20%.” That’s mDAU. The Board told Twitter’s top execs “grow mDAU or you’re out,” so Twitter prioritized growing mDAU. That’s… not a scandal. That’s not a bombshell. That’s not revealing anything the Board wouldn’t have already known.
There’s a bit more like this, and then Mudge claims that Agrawal’s somewhat infamous tweets responding to Musk were designed to mislead him. He’s got it almost entirely backwards.
The rest of Agrawal’s May 16 tweets aren’t out-and-out lies but they rely on wordplay to distract and mislead Mr. Musk, and everyone else. Musk appears to be asking a valid and intuitive question, what percent of accounts encountered by the media user are actually bots?
Except it’s Musk here who is using clever wordplay to distract and mislead everyone. As we’ve described over and over again, the 5% number that Musk repeats in these screenshots is about mDAU. The 5% number is what Twitter reports is the amount of spam they believe incorrectly gets counted in mDAU. It’s Musk who keeps pretending the 5% number implies spam across the entire platform, which Twitter has never said it does. As we’ve explained multiple times now, Musk is trying to distract by pretending that the 5% claim is about spam on the entire platform. It never has been. It has always been an estimate of the amount that makes it through and is still counted in the mDAU.
That is clear to anyone who’s actually read Twitter’s filing (both in the Chancery Court and at the SEC).
From there, Mudge claims that Agrawal was misleading in response by accurately detailing how Twitter narrows down the entire platform to focus just on mDAU and to figure out how much spam is there. But, he’s not. Agrawal knows that Musk keeps referring to the 5% number, and that Musk believes that’s the relevant number that has been somehow falsified in the SEC report. Agrawal’s Twitter thread is an explanation of the mDAU process.
Indeed, Mudge more or less admits this, though again he thinks it’s Agrawal being misleading, rather than the other way around.
While pretending he is answering Musk’s question, in fact Agrawal is answering a very different one, namely, Are there fewer than 5% bots in the set of mDAU accounts, as defined in secret by Twitter? Agrawal’s reasoning might appear a bit circular since, by definition, mDAU is more or less Twitter’s best approximation of the set of accounts that aren’t bots. And Agrawal is not exactly trying to help readers understand the bait-and-switch nature of his answer:
I mean, again, those tweets directly counter Mudge’s own claims that Agrawal is not trying to help readers understand. He literally says in the first tweet shown that he’s just talking about how much gets into the mDAU. And this thread was helpful to those who read it. This thread is the key reason why I understand the shell game that Musk is playing here, by pointing to the mDAU reported number as false, but talking about the total amount of spam on the platform.
In this thread, as you can see above, Agrawal is literally clarifying that point for everyone reading — not obfuscating, as Mudge implies.
So I really don’t understand this next claim from Mudge:
Unless you’re a Twitter engineer responsible for calculating mDAU, you probably wouldn’t know what Agrawal is talking about. He is not saying that fewer than 5% of all accounts on the platform are spam. He’s saying, more or less, that Twitter starts with all the accounts on the platform, tries to automatically put all the accounts that could be convinced by advertisers to buy products (but no spam accounts) into mDAU, and then uses humans to estimate the error rate of spam accounts that nevertheless slip through into mDAU. And naturally, Twitter “can’t share” its special sauce for determining mDAU.
And, um, I’m not a Twitter engineer, and not only did I understand that (in large part due to Agrawal’s thread) I think explained it to lots more people in two separate posts, because it seemed very clear to me what he was saying — and it seemed much clearer that Musk was the one misrepresenting things to his adoring fans, pretending that the 5% number is about spam on the total platform, and that Agrawal was explaining “no, the 5% (the number Twitter reports to the SEC and the only one that might sorta, kinda have some legal issues tied to it) is not the total number on the platform, but the bit that inadvertently slips through.” I mean, that’s literally what Agrawal tweeted.
As for the final line of the paragraph, which I read as sarcasm about Twitter’s inability to share its “special sauce”, is a really weird line for a security professional to include in such a filing. As Agrawal made quite clear in his thread, part of the human determination involves looking at private information, including IP addresses and other information that Twitter cannot give out because it would be a huge privacy violation that would certainly violate the consent decree that Mudge claimed was so important elsewhere.
Reading through all of this, anyone who actually understands the details — including what’s at play in the lawsuit — should see that Mudge is actually confirming the only thing that matters for the lawsuit: that the numbers Twitter reported to the SEC for mDAU involves estimating how much spam they mistakenly include in mDAU and not how much spam is on the platform as a whole. If the actual total amount of spam on the platform is higher than that, it doesn’t help Musk, because Musk’s legal argument is predicated on the <5% reported to the SEC.
Mudge is a smart dude, so I’m confused as to how he got this as mixed up as he did.
Oddly, Mudge’s report may help Musk — but in a totally different way
Incredibly, despite all of this, Mudge’s whistleblowing may actually help out Musk in a much bigger way. Musk’s entire legal argument for getting out of the deal is that (1) Twitter refused to provide him with the relevant info to calculate spam and (in his counterclaims) that (2) Twitter is committing fraud by lying to the SEC, and that could create a material adverse event (MAE) that it hid from him, allowing him to get out of the deal.
That’s all nonsense, as discussed above and previously.
However, the new whistleblower report is kicking off an FTC review, apparently EU data protection regulators are looking into it as well, and it’s possible that others are investigating too. Those investigations, and the possibility of a consent decree violation, might… actually… be an MAE that allows him to escape the deal! In addition, Mudge is alleging fraud (though that’s mostly redacted, so no idea how credible it is). And if that’s shown to be the case as well, it might also be an MAE.
Of course, there are a lot of questions before all of that is settled, and I’m not sure it would actually help Musk out. But even as Mudge, Musk, and the media all seem to think the spam stuff helps the case, anyone who actually understands what has been said, what issues are at play in the lawsuit, and how all this works knows that… it doesn’t.
Azure Communication Services support for Teams identities enables built-in calling capabilities for Dynamics 365 Sales
