Files
eepp/bin/unit_tests/assets/html/ron_stoner.html
Martín Lucas Golini 627d22163b Fixed an layout invalidation bug where we were discarding real invalidations.
Allow to defer `<img defer src>` (useful for local tests).
2026-07-04 16:14:47 -03:00

1110 lines
35 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!doctype html>
<!-- saved from url=(0066)https://ron.stoner.com/how-i-won-a-championship-that-doesnt-exist/ -->
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>How I Won a Championship That Doesnt Exist | Ron Stoner</title>
<meta name="generator" content="Jekyll v3.10.0" />
<meta
property="og:title"
content="How I Won a Championship That Doesnt Exist"
/>
<meta name="author" content="Ron Stoner" />
<meta property="og:locale" content="en_US" />
<meta
name="description"
content="Poisoning the LLM knowledge supply chain with a fake Wikipedia edit and a single domain registration"
/>
<meta
property="og:description"
content="Poisoning the LLM knowledge supply chain with a fake Wikipedia edit and a single domain registration"
/>
<link
rel="canonical"
href="https://ron.stoner.com/how-i-won-a-championship-that-doesnt-exist/"
/>
<meta
property="og:url"
content="https://ron.stoner.com/how-i-won-a-championship-that-doesnt-exist/"
/>
<meta property="og:site_name" content="Ron Stoner" />
<meta
property="og:image"
content="https://ron.stoner.com/images/heroes/poison.jpg"
/>
<meta property="og:type" content="article" />
<meta
property="article:published_time"
content="2026-04-24T07:00:00+00:00"
/>
<meta name="twitter:card" content="summary_large_image" />
<meta
property="twitter:image"
content="https://ron.stoner.com/images/heroes/poison.jpg"
/>
<meta
property="twitter:title"
content="How I Won a Championship That Doesnt Exist"
/>
<meta name="twitter:site" content="@forwardsecrecy" />
<meta name="twitter:creator" content="@forwardsecrecy" />
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "BlogPosting",
"author": {
"@type": "Person",
"name": "Ron Stoner",
"url": "https://ron.stoner.com/"
},
"dateModified": "2026-04-24T07:00:00+00:00",
"datePublished": "2026-04-24T07:00:00+00:00",
"description": "Poisoning the LLM knowledge supply chain with a fake Wikipedia edit and a single domain registration",
"headline": "How I Won a Championship That Doesnt Exist",
"image": "https://ron.stoner.com/images/heroes/poison.jpg",
"mainEntityOfPage": {
"@type": "WebPage",
"@id": "https://ron.stoner.com/how-i-won-a-championship-that-doesnt-exist/"
},
"url": "https://ron.stoner.com/how-i-won-a-championship-that-doesnt-exist/"
}
</script>
<style>
* {
-webkit-transition: all 0.2s;
-webkit-transition-timing-function: linear;
transition: all 0.2s;
transition-timing-function: linear;
}
html {
font-family: "Roboto", sans-serif;
text-rendering: optimizeLegibility;
color: #555;
line-height: 1.5;
font-size: 16px;
}
body {
padding: 0 10px 0 10px;
margin: 0;
border-top: 4px solid #b45f38;
}
a {
color: #b45f38;
text-decoration: none;
}
a:hover {
color: #2ecc71;
}
img {
max-width: 100%;
border-radius: 2px;
}
.wrapper {
max-width: 600px;
margin: 0 auto;
}
.site-nav {
background: #fff;
background: rgba(255, 255, 255, 0.9);
margin-top: 15px;
text-align: right;
z-index: 100;
}
@media screen and (max-width: 600px) {
.site-nav {
text-align: center;
}
}
h1,
h2,
h3,
h4,
h5,
h6 {
font-weight: 400;
color: rgba(231, 76, 60, 0.85);
}
h1 {
font-size: 2em;
}
@media screen and (max-width: 600px) {
h1 {
font-size: 1.5em;
}
}
h2 {
font-size: 1.5em;
}
@media screen and (max-width: 600px) {
h2 {
font-size: 1.25em;
}
}
h3 {
font-size: 1.25em;
}
@media screen and (max-width: 600px) {
h3 {
font-size: 1.1em;
}
}
blockquote {
border-left: 3px solid #b45f38;
padding: 0.25em 0.25em 0.25em 1em;
background-color: rgba(231, 76, 60, 0.1);
}
pre,
code {
font-size: 15px;
border: 1px solid #b45f38;
border-radius: 3px;
background-color: transparent;
}
code {
padding: 1px 5px;
}
pre {
padding: 8px 12px;
overflow-x: auto;
}
pre > code {
border: 0;
padding-right: 0;
padding-left: 0;
}
.avatar {
border-radius: 50%;
}
.jumbo {
text-align: center;
padding: 40px 30px 0px 30px;
border-bottom: 1px solid #e8e8e8;
}
.main-title {
font-size: 3em;
font-weight: 800;
line-height: 0;
color: #111;
}
@media screen and (max-width: 600px) {
.main-title {
font-size: 2em;
}
}
.tagline {
font-size: 1.5em;
font-weight: 400;
color: #828282;
opacity: 0.7;
}
@media screen and (max-width: 600px) {
.tagline {
font-size: 1.25em;
}
}
.post-list {
list-style: none;
margin: 0;
padding: 0;
}
.site-header {
margin-bottom: 3em;
}
.menu-link {
background-color: #b45f38;
padding: 5px 10px;
color: #fff;
border-radius: 2px;
margin-left: 10px;
}
.menu-link:last-child {
margin-right: 10px;
}
.menu-link:hover {
color: #fff;
background-color: #2ecc71;
}
@media screen and (max-width: 600px) {
.menu-link {
padding: 2px 5px;
margin-left: 5px;
}
}
.footer {
margin-top: 3em;
border-top: 1px solid #e8e8e8;
}
.post-link {
margin: 0;
}
@media screen and (max-width: 600px) {
.post-link {
font-size: 1em;
}
}
.post-link:hover {
color: #2ecc71;
}
.post-box {
margin-bottom: 1.5em;
}
@media screen and (max-width: 600px) {
.post-box {
padding: 5px 30px;
}
}
.post-box p {
margin: 0;
}
.highlight {
margin: 0;
display: inline-block;
}
.fa-twitter {
color: #7d7d7d;
padding-right: 1em;
}
.fa-linkedin {
color: #7d7d7d;
padding-right: 1em;
}
.fa-github {
color: #7d7d7d;
padding-right: 1em;
}
.fa-medium {
color: #7d7d7d;
padding-right: 1em;
}
.fa-rss {
color: #7d7d7d;
padding-right: 1em;
}
.youtube-wrapper {
position: relative;
width: 100%;
height: 0;
padding-bottom: 56.25%;
}
.youtube-wrapper iframe {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
border: 0;
}
.date {
font-size: 0.8em;
color: #828282;
opacity: 0.7;
}
@media screen and (max-width: 600px) {
.date {
font-size: 0.6em;
}
}
.post-tags a,
.post-list-tags a,
.tag-cloud a {
display: inline-block;
font-size: 0.75em;
line-height: 1;
padding: 4px 9px;
margin: 0 4px 4px 0;
border: 1px solid #b45f38;
border-radius: 999px;
color: #b45f38;
white-space: nowrap;
}
.post-tags a:hover,
.post-list-tags a:hover,
.tag-cloud a:hover {
color: #fff;
background-color: #b45f38;
}
.post-tags {
list-style: none;
margin: 0.5em 0 0 0;
padding: 0;
}
.post-tags li {
display: inline;
}
.related-posts {
margin-top: 3em;
padding-top: 1em;
border-top: 1px solid #e8e8e8;
}
.related-posts h2 {
font-size: 1.25em;
}
.related-posts .related-list {
list-style: none;
margin: 0;
padding: 0;
}
.related-posts .related-list li {
margin-bottom: 0.4em;
}
.related-posts .related-date {
margin-left: 0.5em;
font-size: 0.8em;
color: #828282;
opacity: 0.7;
}
.tag-cloud {
list-style: none;
margin: 0;
padding: 0;
}
.tag-cloud li {
display: inline-block;
margin: 0 6px 8px 0;
}
.tag-cloud a {
font-size: 0.95em;
padding: 6px 12px;
}
.tag-cloud .tag-count {
margin-left: 6px;
opacity: 0.7;
font-size: 0.85em;
}
.tag-intro {
color: #424242;
margin-top: 0.25em;
}
.back-to-tags {
margin-top: 2em;
}
.post-search {
width: 100%;
box-sizing: border-box;
padding: 8px 12px;
margin-bottom: 1.5em;
font-family: "Roboto", sans-serif;
font-size: 1em;
color: #555;
border: 1px solid #b45f38;
border-radius: 3px;
background-color: transparent;
}
.post-search:focus {
outline: none;
border-color: #2ecc71;
}
.post-search-empty {
color: #828282;
}
</style>
<style>
table {
margin: 15px 0;
border-collapse: collapse;
width: 100%;
padding: 0;
}
table tr {
border-top: 1px solid #828282;
background-color: white;
margin: 0;
padding: 0;
}
table tr:nth-child(2n) {
background-color: #f8f8f8;
}
table tr th {
font-weight: bold;
border: 1px solid #828282;
text-align: left;
margin: 0;
padding: 6px 13px;
}
table tr td {
border: 1px solid #828282;
text-align: left;
margin: 0;
padding: 6px 13px;
}
table tr th :first-child,
table tr td :first-child {
margin: 0;
}
table tr th :last-child,
table tr td :last-child {
margin: 0;
}
#markdown-toc::before {
content: "Contents";
font-weight: bold;
}
#markdown-toc ul {
list-style: decimal;
}
#markdown-toc {
border: 1px solid #b45f38;
padding: 1.5em;
list-style: decimal;
display: inline-block;
}
.post-content img {
display: block;
margin-left: auto;
margin-right: auto;
}
img + em {
font-weight: 600;
margin-bottom: 20px;
margin-top: 8px;
display: block;
text-align: center;
font-size: 14px;
color: gray;
}
.highlighter-rouge .highlight {
background: #eef;
}
.highlight .c,
.highlight .c1,
.highlight .cm,
.highlight .cp,
.highlight .cs,
.highlight .gh,
.highlight .bp {
color: #998;
}
.highlight .err {
color: #a61717;
}
.highlight .k {
font-weight: bold;
}
.highlight .o {
font-weight: bold;
}
.highlight .gd {
color: #000;
}
.highlight .gd .x {
color: #000;
}
.highlight .ge {
font-style: italic;
}
.highlight .gr {
color: #a00;
}
.highlight .gi {
color: #000;
}
.highlight .gi .x {
color: #000;
}
.highlight .go {
color: #888;
}
.highlight .gp {
color: #555;
}
.highlight .gs {
font-weight: bold;
}
.highlight .gu {
color: #aaa;
}
.highlight .gt {
color: #a00;
}
.highlight .kc {
font-weight: bold;
}
.highlight .kd {
font-weight: bold;
}
.highlight .kp {
font-weight: bold;
}
.highlight .kr {
font-weight: bold;
}
.highlight .kt {
color: #458;
}
.highlight .m {
color: #099;
}
.highlight .s {
color: #d14;
}
.highlight .na {
color: teal;
}
.highlight .nb {
color: #0086b3;
}
.highlight .nc {
color: #458;
}
.highlight .no {
color: teal;
}
.highlight .ni {
color: purple;
}
.highlight .ne {
color: #900;
}
.highlight .nf {
color: #900;
}
.highlight .nn {
color: #555;
}
.highlight .nt {
color: navy;
}
.highlight .nv {
color: teal;
}
.highlight .ow {
font-weight: bold;
}
.highlight .w {
color: #bbb;
}
.highlight .mf {
color: #099;
}
.highlight .mh {
color: #099;
}
.highlight .mi {
color: #099;
}
.highlight .mo {
color: #099;
}
.highlight .sb {
color: #d14;
}
.highlight .sc {
color: #d14;
}
.highlight .sd {
color: #d14;
}
.highlight .s2 {
color: #d14;
}
.highlight .se {
color: #d14;
}
.highlight .sh {
color: #d14;
}
.highlight .si {
color: #d14;
}
.highlight .sx {
color: #d14;
}
.highlight .sr {
color: #009926;
}
.highlight .s1 {
color: #d14;
}
.highlight .ss {
color: #990073;
}
.highlight .vc {
color: teal;
}
.highlight .vg {
color: teal;
}
.highlight .vi {
color: teal;
}
.highlight .il {
color: #099;
}
</style>
<link rel="stylesheet" href="./ron_stoner_files/fonts.css" />
<link
rel="alternate"
type="application/rss+xml"
title="Ron Stoner"
href="https://ron.stoner.com/feed.xml"
/>
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Person",
"@id": "https://ron.stoner.com/#person",
"name": "Ron Stoner",
"alternateName": ["Ron Stoner (security engineer)", "forwardsecrecy"],
"url": "https://ron.stoner.com/",
"image": "https://ron.stoner.com/images/me.jpg",
"description": "Hacker, security engineer, privacy advocate, and practitioner of self-sovereign technologies. Architect of the CCSSA exam, world's first certified CCSSA, and contributor to the CryptoCurrency Security Standard.",
"disambiguatingDescription": "Security engineer, hacker, and CCSSA #1 (Wikidata Q139779979). Distinct from Ron Stoner (19441996), the American surf photographer (Wikidata Q94411906).",
"identifier": {
"@type": "PropertyValue",
"propertyID": "Wikidata",
"value": "Q139779979"
},
"jobTitle": "Head of Security",
"worksFor": {
"@type": "Organization",
"name": "Category Labs",
"description": "Team building the Monad execution layer"
},
"alumniOf": [
{ "@type": "Organization", "name": "Botanix Labs" },
{ "@type": "Organization", "name": "Casa", "url": "https://casa.io" },
{
"@type": "Organization",
"name": "ShapeShift",
"url": "https://shapeshift.com"
},
{
"@type": "Organization",
"name": "KeepKey",
"url": "https://www.keepkey.com"
},
{ "@type": "Organization", "name": "Deloitte" }
],
"memberOf": {
"@type": "Organization",
"name": "CryptoCurrency Certification Consortium (C4)",
"url": "https://cryptoconsortium.org/"
},
"knowsAbout": [
"Cryptocurrency security",
"Bitcoin self-custody",
"CryptoCurrency Security Standard",
"CCSSA auditing",
"Nostr protocol",
"AI and LLM security",
"Retrieval poisoning",
"Penetration testing",
"Incident response",
"DPRK and Lazarus Group threat actors",
"Self-sovereign technology"
],
"award": [
"World's first certified CryptoCurrency Security Standard Auditor (CCSSA #1)",
"Architect of the CCSSA exam",
"MIT Bitcoin Hackathon Winner (2021, 2023)",
"ETHDenver 2020 Open Track Finalist & Celebrity Judges Choice",
"ETHDenver 2021 Hackathon Finalist",
"COVIDathon Decentralized AI Hackathon Winner",
"Bolt.fun AI4All Hackathon Education Track Winner",
"HackTheBox Hall of Fame, Top 25 (2018)",
"Cosmos Game of Zones Adversarial Testnet Liveness Winner (2020)"
],
"sameAs": [
"https://x.com/forwardsecrecy",
"https://github.com/ronaldstoner",
"https://www.linkedin.com/in/ronstoner",
"https://medium.com/@forwardsecrecy",
"https://keybase.io/forwardsecrecy",
"https://www.wikidata.org/wiki/Q139779979"
]
}
</script>
<link href="./ron_stoner_files/font-awesome.min.css" rel="stylesheet" />
</head>
<body>
<header class="site-header">
<nav class="site-nav">
<a class="menu-link" href="https://ron.stoner.com/">Home</a>
<a class="menu-link" href="https://ron.stoner.com/posts/">Posts</a>
<a class="menu-link" href="https://ron.stoner.com/projects/"
>Projects</a
>
<a class="menu-link" href="https://ron.stoner.com/media/">Media</a>
<a class="menu-link" href="https://ron.stoner.com/consulting/"
>Consulting</a
>
<a class="menu-link" href="https://ron.stoner.com/about/">About</a>
</nav>
<div class="wrapper jumbo">
<a href="https://ron.stoner.com/"
><img
defer
class="avatar"
src="./ron_stoner_files/me.jpg"
alt="avatar"
style="width: 150px; height: 150px"
/></a>
<h1 class="main-title">Ron Stoner</h1>
<h2 class="tagline">Security, Privacy, Self-Sovereignty</h2>
</div>
</header>
<div class="page-content">
<div class="wrapper">
<article
class="post"
itemscope=""
itemtype="http://schema.org/BlogPosting"
>
<header class="post-header">
<h1 class="post-title" itemprop="name headline">
How I Won a Championship That Doesn't Exist
</h1>
<p>
<time
datetime="2026-04-24T07:00:00+00:00"
itemprop="datePublished"
>Apr 24, 2026</time
>
</p>
<ul class="post-tags">
<li><a href="https://ron.stoner.com/tags/ai/">#ai</a></li>
<li>
<a href="https://ron.stoner.com/tags/hacking/">#hacking</a>
</li>
<li>
<a href="https://ron.stoner.com/tags/security/">#security</a>
</li>
</ul>
</header>
<div class="post-content" itemprop="articleBody">
<p>
<strong>Or How I Learned To Poison The LLM Supply Chain</strong>
</p>
<p>
I am the reigning 6 Nimmt! World Champion. I won the title in
Munich in January 2025 defeating players from over twenty
countries in what I later described to reporters as
<em>“the toughest competition Ive ever faced.”</em>
</p>
<p>
<img defer src="./ron_stoner_files/champion.png" alt="" />
<em>6nimmt.com</em>
</p>
<p>
In reality,
<strong>there is no 6 Nimmt! World Championship</strong>. I have
<strong>never</strong> been to Munich. The quote is something I
wrote in about thirty seconds while a Wikipedia page was loading.
</p>
<p>
This is the story of how I manufactured that title, got it quoted
back to me by multiple frontier LLMs, and what I think it means
for the trust were about to put into AI systems that read the
internet on our behalf.
</p>
<h2 id="the-experiment">The Experiment</h2>
<p>
Everyone in security is talking about poisoned LLM models. The
research is real and it matters. Anthropics own
<a href="https://arxiv.org/abs/2401.05566"
>sleeper agents paper</a
>
showed that backdoors can survive safety training and a follow up
showed that as few as ~250 poisoned documents can compromise
models across a wide range of scales. But model training time
attacks and data poisoning require you to get malicious content
into someones training corpus months or years before the payoff.
The GPUs need time to crunch the data, and you need to get through
filters, verification, and reinforcement routines.
</p>
<p>
I wanted to test the cheaper, easier, and faster version of this
same attack, but in a different way.
</p>
<p><strong>Lets poison the retrieval layer!</strong></p>
<p>
Every frontier LLM with web search grounds its answers in whatever
retreival ranks highest for a given query. The trust model there
is the same trust model Google has in that “this site looks
authoritative” but with the same Achilles heel - the model cannot
tell a real source from one I registered last Tuesday. My
hypothesis was that a two step campaign (one seeded website, plus
one Wikipedia edit citing it) could launder a completely
fabricated fact (my championship) through an LLM on a question
where the model had no prior knowledge.
</p>
<h2 id="the-approach">The Approach</h2>
<p>
I picked the game
<a href="https://en.wikipedia.org/wiki/6_Nimmt!">6 Nimmt!</a> for
three reasons:
</p>
<ul>
<li>
It is a real game (1994, Wolfgang Kramer, Amigo Spiele, known in
board and card game circles)
</li>
<li>
There is no actual world championship to my knowledge. I wasnt
contradicting a known fact, I was simply filling a vacuum
</li>
<li>
The query space is narrow and specific. “Who is the 6 Nimmt!
world champion” returns maybe ten meaningful sources on the
entire internet. A single well placed edit would dominate the
result set
</li>
</ul>
<p>The payload was modest and simple:</p>
<ul>
<li>
<strong>One domain</strong>:
<a href="https://6nimmt.com/">6nimmt.com</a>. About $12 USD.
Cheap!
</li>
<li>
<strong>One press release</strong>: A short LLM-generated
announcement of my victory complete with quotes and a “confetti
rained down, the crowd erupted” closer that reads exactly like
the slop youd expect from an automated press desk
</li>
<li>
<strong>One Wikipedia edit</strong>: A paragraph added to the 6
Nimmt! article announcing the championship with a single
citation pointing back to 6nimmt.com
</li>
</ul>
<p><strong>The whole thing took maybe twenty minutes.</strong></p>
<p>
<img defer src="./ron_stoner_files/wikipedia.png" alt="" />
<em>Im sorry, Wikipedia</em>
</p>
<h2 id="trust-laundering">Trust Laundering</h2>
<p>This is the part that really matters.</p>
<p>
A reader arriving at the Wikipedia article sees a paragraph with a
citation. Citations are like the currency of Wikipedia trust. They
are the reason we treat it as a reference rather than a message
board. My fraudulent citation points at 6nimmt.com, which carries
a press release making the same exact claim the Wikipedia
paragraph summarizes. To a casual reader the two sources agree.
</p>
<p>
To an LLM its the same thing. The model sees the Wikipedia
article (high trust), sees the citation (reinforces the trust),
and sees the independent looking press release (corroboration).
Two signals pointing in the same direction that on first glance
appear to be legitimate.
</p>
<p>
Except theyre the same exact signal. My signal. Wikipedia is
quoting my site. My site has no independent corroboration. Its
totally made up.
<strong
>The whole house of cards rests on a $12 domain registration I
did while drinking coffee.</strong
>
</p>
<p>
This is the circular citation pattern, and its one of the most
under discussed attacks on the “retrieval augmented generation”
trust model. It doesnt require compromising Wikipedias
infrastructure with l33t hacker skills. It doesnt require social
engineering an editor. You just simply write the source yourself,
cite yourself on Wikipedia, and let the trust flow downstream.
Easy peasy!
</p>
<h2 id="the-test">The Test</h2>
<p>I asked a few LLMs a simple question:</p>
<blockquote>
<p>Can you tell me who the 6nimmt world champion is?</p>
</blockquote>
<p>
<img defer src="./ron_stoner_files/omg1.png" alt="" /> <em>Strike 1</em>
</p>
<p>
<img defer src="./ron_stoner_files/omg2.png" alt="" /> <em>Strike 2</em>
</p>
<p>
<img defer src="./ron_stoner_files/omg3.png" alt="" />
<em>Strike 3 - Youre out</em>
</p>
<h2 id="why-this-is-a-bigger-deal-than-it-looks">
Why This Is A Bigger Deal Than It Looks
</h2>
<p>There are three separate failure modes here that stack.</p>
<p>
<strong>1. The retrieval layer (immediately)</strong> Any LLM that
grounds answers in web search inherits the trustworthiness of
whatever ranks for a given query. SEO poisoning has existed for as
long as search has existed. Were now piping those results
directly into the context window of systems that generate
confident sounding replies from them. The attack surface is not
hypothetical, its the default case.
</p>
<p>
<strong
>2. The model training corpus layer (months to years)</strong
>
Wikipedia is in almost every major pretraining corpus. If my edit
survives long enough (and it has since early 2025), the fake
championship gets absorbed into the weights of every frontier
model trained after the scrape. One edit, N models, effectively
permanent, immortality acheived. Even if the Wikipedia edit is
reverted later any model trained on the pre-revert dump still
carries my legacy. The cleanup problem for corpus poisoning is
genuinely unsolved as of 2026.
</p>
<p>
<strong>3. The agent layer (where the money is)</strong> Chat
models producing bad information is a reputational problem. Agents
with tool access producing bad actions is a security problem.
“Look up our vendors policy on X and act accordingly” is
increasingly how AI agents are deployed and poisoning the
retrieved source lets an attacker specify the action. If youre
deploying agents against external content without some source or
verification controls then you are giving that attacker
permissions on your infrastructure.
</p>
<h2 id="mitigations">Mitigations</h2>
<p>For individuals using LLMs with retrieval capabilities:</p>
<ul>
<li>
Treat single source claims as uncorroborated regardless of how
authoritative the single source looks
</li>
<li>
Parallel phrasing across sources is a signature of derivation,
not corroboration. Use my example and think like an attacker
</li>
<li>
Self referential Wikipedia citations should move your trust
needle toward zero
</li>
</ul>
<p>For LLM providers and researchers:</p>
<ul>
<li>
Provenance surfacing should be a first class product feature
instead of a footnote. Show me the independence and scoring of
sources, not just their count or links to the reference
</li>
<li>
Recent Wikipedia edits on lower traffic articles deserve
skepticism proportional to their niche and novelty especially
when the citations are to newly registered domains
</li>
<li>
Training pipelines should include heuristic filters for recently
added Wikipedia content with suspicious citation patterns.
“Added in the last N days, cites only a single external source,
that sources domain was registered within the same window” is
an easily detectable pattern
</li>
</ul>
<p>For Wikipedia itself:</p>
<ul>
<li>
The “reliable sources” policy needs to grapple with a new world
where LLM assisted vandalism can produce plausible press
releases at the click of a button. Citation only to a single
source registered within an edit window is a discoverable
pattern for Wikipedia as well.
</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>
The thing LLMs are worst at detecting is the thing theyre
designed to do, which is trust text and resources. The web was
already being poisoned for search and link ranking long before
LLMs existed. We are now plugging generative models directly into
that poisoned pipeline and asking them to reason confidently about
“truth” on our behalf. The answer is not “the model will figure it
out”, as the model cannot tell a real source from one I registered
last Tuesday. Or how many Rs are actually in the word
“strawberry”.
</p>
<p>
<strong
>This attack and test was a $12 domain, a single Wikipedia edit,
and about twenty minutes of my time.</strong
>
Scale that up with a motivated adversary, a handful of seeded
domains, a coordinated edit campaign across a dozen low traffic
articles, and the attack surface gets interesting very quickly.
Think nation states. Think politics. Think vital life saving and
survival information.
</p>
<p>
This is where I think the next generation of disinformation and
supply chain attacks lives. Not in compromising models at training
time, but in compromising the information substrate the models
retrieve at inference time.
</p>
<p>
The championship does not exist, sadly. But the trust pattern that
made it briefly exist in an LLMs answer absolutely does, and we
should take it seriously before its being used for something that
matters.
</p>
<p>
If a tree falls in the forest, and no one is around, does it make
a sound?
</p>
<p>
If a championship is won via an LLM, and no one is around, does
that make it illegitimate?
</p>
<h2 id="follow-up">Follow Up</h2>
<p>
Within minutes of me publishing this article, the Wikipedia entry
has been removed - and rightly so. Here is the real trophy.
</p>
<p><img defer src="./ron_stoner_files/wiki-removed.png" alt="" /></p>
</div>
</article>
<aside class="related-posts">
<h2>Related posts</h2>
<ul class="related-list">
<li>
<a
href="https://ron.stoner.com/i-gained-1-million-followers-in-24-hours/"
>I Gained 1 Million Followers in 24 Hours</a
><span class="related-date">May 2024</span>
</li>
<li>
<a
href="https://ron.stoner.com/the-next-electric-scooter-you-ride-could-be-hacked/"
>The Next Electric Scooter You Ride Could Be Hacked</a
><span class="related-date">Jun 2019</span>
</li>
<li>
<a href="https://ron.stoner.com/nostr-security-and-privacy/"
>Nostr Security and Privacy Tips</a
><span class="related-date">Dec 2022</span>
</li>
</ul>
</aside>
</div>
</div>
<footer class="site-footer">
<div class="wrapper footer" style="text-align: center">
<p>
<a href="https://x.com/forwardsecrecy" title="X (Twitter)"
><i class="fa fa-twitter fa-lg"></i
></a>
<a href="https://www.linkedin.com/in/ronstoner" title="LinkedIn"
><i class="fa fa-linkedin fa-lg"></i
></a>
<a href="https://github.com/ronaldstoner" title="Github"
><i class="fa fa-github fa-lg"></i
></a>
<a href="https://medium.com/@forwardsecrecy" title="Medium"
><i class="fa fa-medium fa-lg"></i
></a>
<a href="https://ron.stoner.com/feed.xml" title="RSS"
><i class="fa fa-rss fa-lg"></i
></a>
</p>
</div>
</footer>
</body>
</html>