Summary: Researchers warn of a new disinformation tactic called “AIPasta” — a method that uses generative AI to produce many slightly different versions of the same false message so it appears to come from multiple sources. Unlike traditional CopyPasta, which copies the identical text repeatedly, AIPasta generates paraphrases that preserve the original claim while varying wording and style. This variation increases the illusion of widespread agreement and can make false claims seem more credible, particularly among groups already predisposed to accept them.
In controlled experiments, AIPasta made certain false narratives feel more widely accepted than identical repeated text did. The technique was especially effective at boosting belief in election- and pandemic-related conspiracies among Republican participants. The study also found that AI-paraphrased messages were less likely to be flagged by current AI-text detectors, suggesting AIPasta could be harder for platforms to automatically detect and remove.
Key Facts:
- Emerging threat: AIPasta leverages generative AI to scale and vary disinformation campaigns.
- Consensus illusion: Paraphrased variants make false claims appear more widely endorsed than verbatim repetition.
- Detection challenge: AIPasta examples in the study escaped current AI-text detection tools, increasing the risk they could proliferate on social media.
Source: PNAS Nexus
What is AIPasta?
AIPasta is the name given to a hybrid tactic that combines the repetitive dynamics of CopyPasta with the paraphrasing power of large language models. CopyPasta relies on repetition of an identical message to increase familiarity — and sometimes perceived truth — among audiences exposed repeatedly. AIPasta, by contrast, keeps the core false claim intact while producing many linguistically distinct versions, making it look like multiple independent actors are endorsing the same idea.
The research team, led by Saloni Dash, systematically compared traditional CopyPasta with AI-generated paraphrases produced using a large language model. They focused on two real-world misinformation narratives: claims that the 2020 U.S. presidential election was fraudulent and claims that the COVID-19 pandemic was intentionally caused.
Using a preregistered online experiment with 1,200 U.S. participants recruited through Prolific, the authors measured how exposure to CopyPasta, AIPasta, or control material affected participants’ belief in the false claims, their perception of how widely the claims were accepted, and their willingness to share the messages.
Overall, neither CopyPasta nor AIPasta convinced most participants that the examined conspiracies were true. However, important differences emerged when the sample was examined by political affiliation. Among Republican participants — a group more likely to be receptive to the specific election-related misinformation tested — exposure to AIPasta increased belief in the false claim more than exposure to identical repeated text. For participants across political lines, AIPasta raised perceptions that others accepted the claim, producing a stronger consensus illusion than CopyPasta.
The researchers also evaluated the linguistic properties of AIPasta versus CopyPasta and confirmed that AIPasta was lexically more diverse while preserving the original message’s meaning. Crucially, state-of-the-art AI-text detectors used in the study failed to reliably identify the AI-paraphrased content as machine-generated, indicating that AIPasta could evade existing automated moderation tools.
About this AI and disinformation research news
Author: Saloni Dash
Source: PNAS Nexus
Contact: Saloni Dash – PNAS Nexus
Image credit: The image is credited to Neuroscience News
Original Research: Open access. “The persuasive potential of AI-paraphrased information at scale” by Saloni Dash et al. DOI: 10.1093/pnasnexus/pgaf207 PNAS Nexus
Abstract
The persuasive potential of AI-paraphrased information at scale
This study examines how AI-paraphrased messages can amplify the reach and persuasive force of information campaigns. Building on social and cognitive theories about repetition and information processing, the authors model how CopyPasta — a tactic that relies on repeated identical messaging — can be enhanced using large language models to generate variant messages, termed AIPasta.
The researchers extracted CopyPasta examples from two prominent U.S. disinformation campaigns and used a large language model to create paraphrased variants. They validated that AIPasta increased lexical diversity while retaining the original semantics. In a preregistered experiment (N = 1,200), AIPasta, unlike CopyPasta, increased perceived consensus around the false narratives and, depending on political orientation, increased belief in the specific false claims. Sharing intentions remained similar between AIPasta and control conditions, while CopyPasta sometimes reduced sharing intent.
Despite modest differences in persuasion across most outcomes, the study highlights that current AI-text detectors failed to flag AIPasta. As AI-assisted information operations become more common, the authors warn of a likely shift from identical repetition toward paraphrased, scalable tactics that pose significant challenges for detection and mitigation on social media platforms.