AI Can Now Identify People From Anonymous Text, Research Shows
The ability to maintain online privacy, even under a pseudonym, is becoming increasingly difficult. Recent research demonstrates that large language models (LLMs) can now unmask pseudonymous users with surprising accuracy, challenging long-held assumptions about online anonymity. This capability stems from the LLMs’ ability to correlate information across multiple platforms and infer identities from seemingly innocuous data.
Simon Lermen, a co-author of the research and currently affiliated with the Technical University of Berlin, explained to Ars Technica that these AI agents can perform a task previously considered very difficult: “starting from free text (like an anonymized interview transcript) they can work their way to the full identity of a person.” This represents a significant shift from older deanonymization methods, which typically required structured datasets and manual analysis.
How LLMs Break Pseudonymity
Previous attempts to de-anonymize online users relied heavily on linking structured data – for example, matching usernames and email addresses across different services. This required significant effort in data collection and organization. LLMs, however, operate differently. They can “browse the web and interact with it in many of the same ways humans do,” according to Lermen, using simulated reasoning to match potential individuals based on their online activity.
The researchers demonstrated this capability through several experiments. One involved analyzing responses to a questionnaire conducted by Anthropic regarding the use of AI. By examining the answers provided, the LLM was able to positively identify 7 percent of the 125 participants. While this recall rate might seem modest, it highlights the potential of AI to identify individuals based on relatively general information. The researchers noted that as LLMs continue to improve, their ability to identify users will likely increase.
Complete-to-end deanonymization from a single interview transcript (with details altered to protect the subject’s identity). An LLM agent extracted structured identity signals from a conversation, autonomously searched the web to identify a candidate individual, and verified the candidate matched all extracted claims.
End-to-end deanonymization from a single interview transcript (with details altered to protect the subject’s identity). An LLM agent extracted structured identity signals from a conversation, autonomously searched the web to identify a candidate individual, and verified the candidate matched all extracted claims.
Impact on Online Communities and Privacy
The implications of this research extend beyond individual privacy. The researchers also examined comments from Reddit, specifically the r/movies subreddit and several smaller communities. They found that the more movies a user discussed, the easier it became to identify them. With ten or more shared movies, the identification rate rose to 48.1 percent with 90 percent precision and 17 percent with 99 percent precision. This suggests that even seemingly innocuous online behavior can be used to reveal a user’s identity.
This capability poses risks to individuals who rely on pseudonymity for protection, such as those participating in sensitive discussions or seeking information on potentially controversial topics. The researchers warn that the assumption that pseudonymity provides adequate protection is no longer valid. This could lead to increased self-censorship and a chilling effect on free speech online. Simon Lermen, the lead researcher, has a strong publication record in AI safety, suggesting a focus on the ethical implications of these technologies.
Limitations and Future Research
While the results are concerning, it’s important to acknowledge the limitations of the study. The experiments were conducted in specific contexts – a questionnaire and Reddit comments – and the results may not generalize to all online platforms or types of data. The researchers also compared their method to the older “Netflix Prize attack,” a previous attempt at deanonymization, suggesting a benchmark for improvement.
the success of the LLM-based deanonymization relies on the availability of sufficient information about the target individual online. Users with limited online footprints or those who take steps to protect their privacy may be more difficult to identify. The researchers acknowledge that this is an evolving field and that ongoing research is needed to fully understand the capabilities and limitations of LLMs in deanonymization.
What Comes Next: A Shifting Landscape for Online Privacy
The findings underscore the demand for a re-evaluation of online privacy strategies. Traditional methods of maintaining anonymity, such as using pseudonyms and avoiding the sharing of personal information, may no longer be sufficient. Users may need to adopt more sophisticated techniques, such as using privacy-enhancing technologies and being mindful of their digital footprint.
The research also raises questions about the ethical responsibilities of developers of LLMs. As these models become more powerful, it is crucial to consider the potential for misuse and to develop safeguards to protect user privacy. Further research will likely focus on developing methods to detect and mitigate deanonymization attacks, as well as exploring alternative approaches to online privacy that are resilient to the capabilities of LLMs.