I just got back from a wonderful vacation to Ireland and Scotland (more on that after I unpack). I didn’t post anything from the road, because I wanted to get back to my home base so that I could research Substack’s latest update.
Since the end of July, users of Substack’s app have the option to scan articles and Notes to detect the presence of AI-generated text before reading them.
You can read more about the details on the official Substack blog here:
I don’t often talk about Substack on Substack because my newsletter is about my life in Germany, not about writing. And I want it to function independently of the platform I use to publish it.
But this is a significant change that may affect how readers experience Alte Frau - New Life, and one I fear will lead to negative unintended consequences in the future - both for me and for Substack as a whole.
To get the obvious question out of the way: I do not use AI to generate text or edit any of my posts, comments, or Notes.1
However, I do occasionally use a translation program (DeepL) that translates German texts into English. And, I sometimes use Grammarly or GrammarTool to check spelling and grammar. When I interview sources, I often use a tool called OtterAI to transcribe the recorded audio.
Technically, apps that translate, transcribe and/or check grammar and spelling are making use of machine learning (‘AI’), too.
But that’s not what they are checking for.
Main concern No. 1 - false positives
When I first heard about the Pangram integration, my main concern was that its AI detection tool would falsely flag my content as AI-generated. I mean, I was using the em dash before before the internet even existed. And I have been blogging and writing for online publications since the early 2000s. My writing style is probably all over the training data.
And the error rates of various AI agents are just not great. Pangram claims that its detection tool has a false-positive rate around 1 in 10,000.
Outside estimates have indicated it is higher.
Individual writers have reported disturbing results.
The writer Laura Brooke Robson tested Pangram by scanning parts of her novel, as well as Claude-edited versions of it, and found the opposite—that Pangram tended to rate AI-assisted writing as human more often than it flagged her unassisted text as AI.
Substack allows writers to opt out of allowing their readers to use the Pangram tool, but only if the writers scan the text with the tool, first, and then choose whether or not to show the results to readers.
This brings me to …
Main concern No. 2 - theft of intellectual property
So I guess Pangram is scanning my work whether I like it or not.
Substack claims that nothing that is scanned is used for their training data.
I am not sure that I believe them. But I am also not sure it matters.
In reality, that horse has already left the barn. I already opted into allowing my content to be used in AI training data in the Substack publication settings. I discovered early on that opting out tanked my SEO. (For those not well-versed in internet content development - this is how high your links rank in search engine results.)
I am also way more worried about human plagiarizers on Substack than agentic ones. Last year, social scientist Katie Jagielnicka, who publishes the newsletter, The Noösphere, discovered another user was stealing her work.
Despite reporting the issue, complete with evidence that Jagielnicka published her essay months before the plagiarized version, Substack took no action against the second writer. The writer later paywalled the stolen content, even while admitting to ‘unintentionally’ lifting whole passages of it verbatim from Jagielnicka’s work.
This is why I don’t trust Substack’s operations team when it comes to the issue of the integrity of the writing.
So, using AI is bad, but straight up stealing someone else’s intellectual property and passing it off as yours is a non-issue?
Whatever.
Be your own judge
Personally, I think I can probably detect AI writing much better than Pangram can.
I do hate to see it. Not because I think I am being deceived, but because it’s usually painfully obvious and a waste of my time.
I know there are writers of some of the newsletters I subscribe to who use AI in their process, though they probably are deluding themselves into thinking people can’t tell.
We can tell.
If we aren’t unsubbing, it is probably that we find the information shared is—on balance—worth the trouble of sifting through the stilted prose. Or, we think the information shared is readable and useful, whether you got help from an LLM or not.
I don’t need an AI agent to warn me about AI slop.
The whole premise of needing Pangram to detect the use of AI in order to maintain the integrity of the information that is published on Substack is just another example of AI hype disguised as AI criticism.
Shoving AI detection down our throats gives the impression that AI-generated text is so close to human-generated writing that it can only be reliably detected by … another AI.
I disagree.
I think Ed Zitron said it best in his essay, Revenge of the Business Idiot:
LLMs impress the writers who do not want to write, the coders who don’t want to code, the researchers who don’t want to research, and the lawyers that don’t want to actually understand case law. Those that desperately tell you how powerful AI is and that you simply must use it are looking for you to validate their own laziness or distaste for effort, and those who are impressed with LLMs’ outputs tend to be people with low standards.
So, scan it if you want.
Yes, I am somewhat worried about readers who will scan it without even reading a word, first. I have no idea whether my writing style will render a false positive.
You can see I love a good em dash, even now. But I hate the negative/positive pluralisms that ChatGPT has rendered cliche. (‘It’s not X! It’s Y’…) And I refuse to describe anything that doesn’t produce sound as “quiet” or “loud.”
But, aside from a test scan I will run on this article before publishing, I am not going to pre-scan my posts to disable the tool.
Pangram - away.
What Others Are Saying
Some posts by other Substack writers also writing about the Pangram issue.
And, last but not least, what Pangram says about itself.
I am aware some of you are going to wade through this whole post and think, ‘Wow, Maybe you should use AI.’ And to that, I say, “Touché.”
I kind of take everything I read with a grain of salt, so to speak. But it can be kind of tricky,especially if it's something I want to hear.