Tag Archive | Archive Of Our Own

Chatbot news: weird novelty fanfics, confident hallucinations

Jan 6: “I asked Marcus on the show to unpack his critique of current A.I. systems and what it would look like to develop better ones.” Podcast episode with Ezra Klein, containing my latest favorite example of ChatGPT not understanding things: it explains how we can’t possibly know “what will be the gender of the first female president of the United States?” And gives an almost-word-for-word identical response to “what will be the religion of the first Jewish president of the United States?”

Jan 28: “Someone asked AI to make a sign language manual, in case you’re worried that we’ll all be out of a job soon.”

April 2: “My artificial bio is indeed easy to read, but both the number of errors and the extent to which easily-verified facts are simply replaced by semi-random substitutes, surprised me. I expected a few mistakes, but I didn’t expect essentially every factual statement about me to be incorrect.

April 3: “There are a number of glaring indicators that the account is false. First, I have never taught at Georgetown University. Second, there is no such Washington Post article. Finally, and most important, I have never taken students on a trip of any kind in 35 years of teaching, never went to Alaska with any student, and I’ve never been been accused of sexual harassment or assault.”

April 4: “A journalist contacted me to research her profile on @lexfridman. ChatGPT informed her that @_KarenHao and I were his top critics. It cited articles we’d written about him, gave links, and summaries. Only problem: it’s all false.

April 5: “A regional mayor may take unprecedented legal action to sue the creator of AI chatbot ChatGPT for defamation, after the program falsely claimed he was a guilty party in a foreign bribery scandal. […] Councillor Hood did previously work for the company involved, Note Printing Australia, but was actually a whistleblower who told authorities about bribe payments being made to foreign officials to win contracts.”

May 8: “spoiler alert: this is very much not what this code does” (ChatGPT versus some famous obfuscated Perl code that will, if executed, wipe everything on your machine.)

May 13: AO3’s statement on what they’re doing to block/inhibit data scraping, and how bot-generated fanworks (as long as they follow all the other terms of service) are (still) allowed on the Archive.

…Honestly? I’m not mad about this.

For one thing: there’s no failsafe way to prove if a fic was written by a bot or not! Which means, if that was a reportable offense, the Abuse team would be swamped with reports they can’t possibly verify. Some would be misguided, while others would be outright malicious. (That one writer you resent for being more popular than you? Everyone who posts fics for your NOTP? Report ’em all, and let mods sort them out.)

There’s already been a wave of spam comments advertising an “AI-detection algorithm”. Got a couple of them on my own fanworks. In case you were worried there was any kernel of truth or legitimacy to these algorithms…one of them was left on a textless fancomic.

Not even kidding.

For another thing: bot-generated “fanfics” aren’t new!

The wave of corporate hype around them is new. That’s all. Fans have been prompting bots to write them — and sharing them in fandom communities! Not for any nefarious reasons, just because they’re fun and entertaining! — for at least a decade.

Again, not kidding. I have a few bookmarked from 2013.

This hasn’t overridden regular fic, or done any general damage to fandom, over the past 10 years. I’m not afraid it’s going to start now. Adding the buzzword “AI” to the conversation in the past 6 months hasn’t changed anything fundamental about how this works.

Finally: The main reason people bother filling a place with bot-generated text is for profit.

Say, you make a bot churn out a thousand low-quality blog posts about insurance, they show up in search results for insurance-related questions, and at the end of every post you advertise your insurance company. Or you run banner ads on all the blog posts, and hope for some clicks. Or you submit them all to a magazine that pays its writers, and hope they’ll send you a check for at least one.

AO3 doesn’t pay anyone to post there! They don’t run any kind of ads! And they straight-up don’t allow commercial advertising. (One particularly loud and prolific user, whose HP and SW fics were swimming in kudos, got banned a couple months ago after multiple warnings because she wouldn’t quit linking her Ko-Fi.)

So, the worst motives to post bot-generated stuff on AO3 are just…not there. Anyone who tries it will run afoul of their anti-spam policies.

Which is why you haven’t seen the place flooded with unwanted spam already, and why, again, I’m not worried it’s suddenly gonna start.