<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Gonka]]></title><description><![CDATA[Gonka is a decentralized AI infrastructure designed to optimize computational power for AI model training and inference, offering an alternative to monopolistic]]></description><link>https://what-is-gonka.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sun, 15 Nov 2026 21:14:02 GMT</lastBuildDate><atom:link href="https://what-is-gonka.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Trustless Decentralized Training: What It Is, Why It's Hard, and How We'll Get There]]></title><description><![CDATA[Authors: Tamaz Gadaev, Egor Shulgin]]></description><link>https://what-is-gonka.hashnode.dev/trustless-decentralized-training-what-it-is-why-it-s-hard-and-how-we-ll-get-there</link><guid isPermaLink="true">https://what-is-gonka.hashnode.dev/trustless-decentralized-training-what-it-is-why-it-s-hard-and-how-we-ll-get-there</guid><dc:creator><![CDATA[Egor Shulgin]]></dc:creator><pubDate>Sat, 07 Mar 2026 04:34:07 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/68d22acd26df97ed6e9a8621/27442020-d858-442e-baf9-f5b214e6497b.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Bitcoin proved that strangers on the internet can build and maintain infrastructure at global scale, without trusting each other. The network's aggregate computing power exceeds that of many traditional data centers. So here's a question worth taking seriously: what if we could do the same thing for AI training?</p>
<p>Picture it. Thousands of volunteers, each contributing their GPUs, collectively training models without any central coordination. But in a network where anyone can join, some will lie, others will cheat, and a few will actively try to break things. Training models in a room full of potential saboteurs is a challenge. Let's break down the problem, the defenses, and where it's all heading.</p>
<h2><strong>1. The Landscape: Three Dimensions of Distributed Training</strong></h2>
<p>"Distributed training" gets thrown around a lot, and it tends to blur several distinct ideas together. Untangling them shows where the hard problems actually live.</p>
<h3><strong>Distributed ≠ Decentralized</strong></h3>
<p><strong>Distributed training</strong> means multiple machines work on one training job. A GPU cluster in a data center counts: split the batches, compute gradients in parallel, aggregate. For an overview of geo-distributed setups, see our <a href="https://what-is-gonka.hashnode.dev/beyond-the-data-center-how-ai-training-went-decentralized">previous article</a>. <strong>Decentralized training</strong> goes further: it removes the central orchestrator entirely. Peers communicate directly and make local decisions based on what their neighbors tell them, with no single point of failure and no central authority enforcing behavior.</p>
<h3><strong>Orchestration, Trust, and Data Control</strong></h3>
<p>Three questions shape any distributed training setup's threat surface:</p>
<ol>
<li><p><strong>Orchestration:</strong> Central coordinator, or peer self-organization?</p>
</li>
<li><p><strong>Trust:</strong> Can you assume honest participants, or must you defend against adversaries?</p>
</li>
<li><p><strong>Data Control:</strong> Does the system control data assignment, or do participants hold private datasets you never see?</p>
</li>
</ol>
<p>These axes carve out distinct scenarios:</p>
<table style="width:709px"><colgroup><col style="width:303px"></col><col style="width:122px"></col><col style="width:138px"></col><col style="width:146px"></col></colgroup><tbody><tr><td><p></p></td><td><p><strong>Trust</strong></p></td><td><p><strong>Data Control</strong></p></td><td><p><strong>Orchestration</strong></p></td></tr><tr><td><p><strong>Data-center training</strong></p></td><td><p>✓</p></td><td><p>✓</p></td><td><p>Centralized</p></td></tr><tr><td><p><strong>Federated learning</strong></p></td><td><p>✗</p></td><td><p>✗</p></td><td><p>Centralized</p></td></tr><tr><td><p><strong>Collaborative learning</strong></p></td><td><p>✗</p></td><td><p>✓</p></td><td><p>Either</p></td></tr></tbody></table>

<p><strong>Federated learning</strong> is the most studied trustless variant. A central server coordinates workers who train on private data they won't expose, as is common in healthcare and finance. <strong>Collaborative learning</strong> is less explored and potentially more transformative. Independent participants train one model on a public dataset (like Common Crawl), without guaranteed trust. It can work with centralized or decentralized orchestration, but the hardest variant drops the central server entirely, removing all single points of failure and letting anyone participate. That's our focus.</p>
<h3><strong>Chaos Before Trust</strong></h3>
<p>Hardware failure in a data center is managed, but it still takes real engineering effort. In collaborative training, failures are the norm. Volunteers join, disconnect, and lose their internet connection mid-step. Any system has to be fault-tolerant before it can be <strong>Byzantine-tolerant</strong>, meaning able to produce correct results even when some participants are actively malicious. Asynchrony and fault tolerance matter in their own right, but here we're zeroing in on the trust problem: how do you learn from contributors you can't verify?</p>
<h2><strong>2. The Threat: Learning from People You Can't Trust</strong></h2>
<p>When independent parties contribute updates to a shared model, how do you know they're playing fair, especially when there's a reward mechanism paying each contributor for work done? You can't assume good faith. A permissionless network has to treat some participants as <strong>Byzantine actors</strong>: they might inject poisoned updates, skip computations while claiming rewards, or collude to steer the model.</p>
<h3><strong>The Attack Triangle</strong></h3>
<p>Threats break down by motivation, and each corner demands a different defense.</p>
<table style="width:739px"><colgroup><col style="width:121px"></col><col style="width:100px"></col><col style="width:337px"></col><col style="width:181px"></col></colgroup><tbody><tr><td><p><strong>Attacker</strong></p></td><td><p><strong>Motivation</strong></p></td><td><p><strong>Behavior</strong></p></td><td><p><strong>Detection Difficulty</strong></p></td></tr><tr><td><p><strong>Lazy</strong></p></td><td><p>Greed</p></td><td><p>Claims work; submits random numbers or replays old updates</p></td><td><p>Medium</p></td></tr><tr><td><p><strong>Saboteur</strong></p></td><td><p>Destruction</p></td><td><p>Sends NaNs, infinities, or extreme noise to blow up the loss</p></td><td><p>Low (loud)</p></td></tr><tr><td><p><strong>Backdoor</strong></p></td><td><p>Control</p></td><td><p>Crafts plausible updates that cause the model to misclassify specific inputs</p></td><td><p>High (blends in)</p></td></tr></tbody></table>

<p>The <strong>Backdoor</strong> is arguably the scariest. A coordinated group could embed a trigger so the model assigns an attacker-chosen label to any input containing a specific pixel pattern (Bagdasaryan et al., 2020). The updates look statistically normal and pass simple variance checks.</p>
<h3><strong>The Meta-Threat: Validator Collusion</strong></h3>
<p>To catch these attacks, you typically need validators: nodes that spot-check others' work. But that raises an old question: <em>who verifies the verifiers?</em></p>
<p>If validation pays less than training, a rational validator might just rubber-stamp everything. Worse, workers and validators can form a cartel: fake work, fake approval, split the reward. A badge doesn't guarantee honesty in a decentralized system.</p>
<p>So how do we get around this? Not by solving trust in one step, but by layering defenses on top of each other.</p>
<h2><strong>3. The Defense Stack</strong></h2>
<p>Defending against these threats takes three main approaches: filtering suspicious updates statistically, verifying that computation actually happened, and making cheating financially irrational. Let's look at each.</p>
<h3><strong>Layer 1: Robust Aggregation (Statistical Filtering)</strong></h3>
<p>We may not know <em>who</em> is lying, but we can analyze <em>what</em> they submit. The core assumption, Honest Majority, says most participants are honest. Their updates cluster around the true value; malicious ones look like outliers.</p>
<p>The Coordinate-wise Median and Trimmed Mean (Yin et al., 2018) examine each parameter's distribution and chop off the tails. The median is provably tolerant of up to 50% corruption; the trimmed mean tolerates a fraction proportional to its trimming parameter (e.g., a 10% trim handles up to ~10% corrupted participants). The downside: both methods can throw away honest updates sitting at the extremes, even when nothing's wrong.</p>
<p>Blanchard et al. (2017) took a different approach with Krum, which treats each update as a vector and computes pairwise Euclidean distances. Honest updates cluster together; attackers land far away. It picks the single update closest to its neighbors. More robust than coordinate-wise methods, but O(N² · d) in compute cost, where N is the number of participants and d is the number of model parameters. For large models, that's expensive.</p>
<p>Karimireddy et al. (2021) found a more efficient path with Centered Clipping. Their method tracks a momentum-based "center" from prior rounds. If a new update exceeds a safe radius, instead of getting tossed, it gets scaled down. O(N) cost, direction preserved, and honest high-variance updates survive.</p>
<h3><strong>Layer 2: Verification (Integrity Checks)</strong></h3>
<p>Statistical filters handle malicious updates, but they can't catch malicious laziness. A participant who generates statistically plausible random numbers might pass aggregation without doing any real work. That's "work fraud," and you need a different tool to stop it.</p>
<p>The naive solution is redundancy: assign the same computation to multiple workers and compare results. Secure? Yes. But it triples your compute cost. In a race to train frontier models, that's a non-starter. A smarter approach: assume work is valid by default, but have a separate pool of verifiers re-compute a random sample, say 1% of submissions. Think ticket inspectors on a train. You don't check every passenger. You just make sure the fine for riding without a ticket hurts enough that most people buy one. A participant who consistently fakes work will get caught eventually.</p>
<p>There's a wrinkle, though. Deep learning isn't perfectly reproducible. GPUs reorder operations for speed, floating-point rounding drifts across runs, and bit-for-bit matching between two honest runs of the same computation is impossible. Proof of Learning (PoL) (Jia et al., 2021) works around this. Instead of demanding exact equality, it records intermediate checkpoints and verifies that the training trajectory is consistent with the claimed computation, making it as hard to fake a proof as to do the actual work.</p>
<p>Why not ZK-proofs? Zero-knowledge SNARKs could, in theory, prove correctness cryptographically. In practice, generating a proof for the billions of floating-point ops in a single training step is currently orders of magnitude slower than the training itself. ZK is making rapid progress and becoming viable for certain inference tasks, but the sheer arithmetic volume of training keeps it out of reach for now.</p>
<h3><strong>Layer 3: Economic Incentives</strong></h3>
<p>Even sophisticated attackers respond to cost. If the expected penalty for cheating exceeds the expected reward, rational actors stay honest.</p>
<p>Staking and slashing is the first mechanism. Participants lock up a deposit to join. Get caught by a spot-check? You lose the stake: not just the reward for that task, but the whole deposit. A 100× penalty-to-reward ratio means you only need to verify about 1% of submissions to make cheating irrational.</p>
<p>What about lazy validators? The protocol can use honeypots: injecting tasks with known answers. Any validator who rubber-stamps a wrong result proves they weren't checking, and gets slashed. Zhao et al. (2024) formalize this intuition with provable incentive-security guarantees.</p>
<p>Finally, there's reputation. Creating a new identity costs nothing. Building a track record does. Nodes that consistently pass verification accumulate a reputation score, which multiplies their weight in aggregation. A veteran with six months of honest work might carry 10× the influence of a newcomer. An attacker can spin up 1,000 fresh identities overnight, but they can't fake the history. By the time they'd built that reputation, they'd have contributed more honest value than the attack could destroy.</p>
<h2><strong>4. Where We Are and What's Next</strong></h2>
<p>None of these defenses works alone:</p>
<ul>
<li><p><strong>Verification</strong> catches fraud but costs compute.</p>
</li>
<li><p><strong>Robust aggregation</strong> filters outliers but misses subtle backdoors.</p>
</li>
<li><p><strong>Economics</strong> deters rational adversaries but won't stop someone acting out of ideology or spite.</p>
</li>
</ul>
<p>The path forward is combining all three. A real system needs enough verification to make fraud risky, enough statistical filtering to catch what slips through, and enough economic pain to keep honest participation the dominant strategy.</p>
<p>The building blocks exist: Byzantine-resilient aggregators, probabilistic verification schemes, game-theoretic incentive models. Researchers have published them. What's left is the engineering: composing them into a coherent protocol and stress-testing it at scale.</p>
<p>Decentralized training isn't solved. But it's no longer a fantasy, either.</p>
<h2><strong>References</strong></h2>
<p>Blanchard, P., et al. (2017). <a href="https://arxiv.org/abs/1703.02757"><em>Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent</em></a>. NIPS.</p>
<p>Yin, D., et al. (2018). <a href="https://arxiv.org/abs/1803.01498"><em>Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates</em></a>. ICML.</p>
<p>Karimireddy, S. P., et al. (2021). <a href="https://arxiv.org/abs/2012.10333"><em>Learning from History for Byzantine Robust Optimization</em></a>. ICML.</p>
<p>Jia, H., et al. (2021). <a href="https://arxiv.org/abs/2103.05633"><em>Proof-of-Learning: Definitions and Practice</em></a>. IEEE Symposium on Security and Privacy.</p>
<p>Bagdasaryan, E., et al. (2020). <a href="https://arxiv.org/abs/1807.00459"><em>How To Backdoor Federated Learning</em></a>. AISTATS.</p>
<p>Zhao, Z., et al. (2024). <a href="https://arxiv.org/abs/2404.09005"><em>Proof-of-Learning with Incentive Security</em></a>. arXiv preprint.</p>
]]></content:encoded></item><item><title><![CDATA[A New Proof-of-Work for the Age of AI]]></title><description><![CDATA[Bitcoin did something remarkable: it showed at scale that strangers could coordinate without trusting each other, without needing a bank or government or…]]></description><link>https://what-is-gonka.hashnode.dev/a-new-proof-of-work-for-the-age-of-ai</link><guid isPermaLink="true">https://what-is-gonka.hashnode.dev/a-new-proof-of-work-for-the-age-of-ai</guid><category><![CDATA[decentralization]]></category><category><![CDATA[AI]]></category><category><![CDATA[proofOfWork]]></category><category><![CDATA[llm]]></category><category><![CDATA[Web3]]></category><dc:creator><![CDATA[Anastasia Matveeva]]></dc:creator><pubDate>Wed, 19 Nov 2025 05:59:13 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1763531901651/b66e9123-1fd4-44e0-b212-82c82be25164.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h1 id="heading-consensus-through-waste">Consensus through Waste</h1>
<p>Bitcoin did something remarkable: it showed at scale that strangers could coordinate without trusting each other, without needing a bank or government or any central authority. For the first time, you could send value to someone on the other side of the world without asking permission from anyone. The network couldn't be shut down. Your money couldn't be censored. And it worked.</p>
<p>Bitcoin suggested proof-of-work as a way to achieve consensus among untrusted participants. The basic idea is straightforward: miners compete to solve a puzzle by finding a number (a nonce) that, when combined with a block's data and run through the SHA-256 hash function, produces a result with a specific property, typically a hash value that starts with a certain number of leading zeros. If you want a hash with, say, 70 leading binary zeros, you need on average 270 attempts. There's no shortcut or clever algorithm that avoids trying different nonces. You just have to compute until you get lucky.</p>
<p>This mechanism was revolutionary. It made the blockchain expensive to attack, as you'd need to redo all this computational work to rewrite history. It aligned incentives: miners earn rewards proportional to their computational work, not their pre-existing wealth, although in practice, access to capital, hardware, and energy matters. For the first time, a truly decentralized system worked at scale.</p>
<p>The tradeoff, though, is that the work itself is not intrinsically valuable. The electricity goes into finding hashes with leading zeros and that's it. Bitcoin essentially trades computational waste for network security. For over a decade, this tradeoff has been good enough to keep Bitcoin a strong asset.</p>
<h1 id="heading-a-new-era-of-decentralization">A New Era of Decentralization</h1>
<p>We're in the middle of a rapid transformation in artificial intelligence. LLMs are becoming infrastructure, essential services that people and organizations depend on. Most LLM inference happens on centralized servers controlled by a handful of companies. This creates familiar problems:</p>
<ul>
<li><p>Single point of control: one company decides which models are available and who gets access.</p>
</li>
<li><p>Censorship risk: governments or corporations can pressure centralized providers to censor or discriminate.</p>
</li>
<li><p>Vendor lock-in: users and developers have no choice but to trust the current gatekeepers.</p>
</li>
</ul>
<p>These are essentially the same problems Bitcoin was designed to address. The question is: can we build a decentralized LLM network that solves these problems without creating the same waste problem Bitcoin has?</p>
<h1 id="heading-the-existing-solutions-and-their-limitations">The Existing Solutions and Their Limitations</h1>
<p>Proof-of-stake (PoS) removes the computational waste by replacing work with capital. Validators lock up tokens as collateral and are chosen to validate blocks proportional to their stake. It uses minimal energy.</p>
<p>The problem is that capital accumulation isn't equally distributed. In networks like Bittensor, wealthy validators attract delegation from smaller token holders. This creates a feedback loop: more capital attracts more delegators, which generates more rewards, which attracts more delegation. Over time, voting power concentrates among those who started with the most wealth. A subnet with excellent GPUs and high-quality inference might have little influence if its validators don't hold much capital.</p>
<p>The result is that voting power decouples from actual computational contribution. You've solved the waste problem but created a wealth concentration problem instead.</p>
<h1 id="heading-an-alternative-approach">An Alternative Approach</h1>
<p>The question then becomes: can you keep proof-of-work's fairness properties but direct the computational effort toward something useful?</p>
<p>There was another line of research that tried to solve the waste problem differently. Beginning around 2017, researchers explored Proof of Useful Work, still based on proof-of-work but redirecting miners' computational effort toward tasks with potential economic or scientific value instead of arbitrary hash puzzles. Some grounded PoW hardness in fine-grained problems, others experimented with federated learning, matrix multiplication tasks, or generating zero-knowledge proofs. The appeal was clear: miners would do real work, preserving PoW's fairness while reducing waste. Until very recently, none of these attempts tackled LLM inference specifically. They focused on discrete computational problems or batch learning, not the live transformer inference that powers today's AI services.</p>
<p>LLM inference is actually quite a natural candidate. It's computationally expensive, economically valuable, and increasingly important. If you could use computational work on inference to secure the network, you'd align the network's security with real computational needs.</p>
<p>Instead of computing hashes, miners would compute transformer inference.</p>
<p>This is the basic idea behind transformer-based proof-of-work. There are some non-trivial design challenges.</p>
<p>Note that this mechanism isn't locked into transformers, it can adapt to whatever model architecture becomes more useful or prevalent in the future.</p>
<h1 id="heading-designing-the-task">Designing the Task</h1>
<h2 id="heading-challenge-1-estimating-computational-resources">Challenge 1: Estimating Computational Resources</h2>
<p>In Bitcoin, mining is the miner's full-time job. But if you want a network that actually serves users, nodes need to spend most of their time on inference requests, not PoW. You have two options.</p>
<p>The first is theoretically feasible but research-intensive: use actual inference computations on existing trained models to estimate the computational resources of participants. You'd run inference, measure the computational cost, and calibrate the weights accordingly. This would be efficient, but it requires significant work to account for differences in input and to account for the potential exploitability of trained models' structure.</p>
<p>The second approach is more pragmatic: make the PoW task time-constrained. Design each puzzle to take a short, fixed, predictable amount of time, say a few minutes. The network then commits to keeping the same computational resources available for the rest of the epoch. This gives you more flexibility in designing a uniform puzzle.</p>
<h2 id="heading-challenge-2-task-alignment-with-llm-computations">Challenge 2: Task Alignment with LLM computations</h2>
<p>If you commit to time-constrained PoW, a new problem emerges: if the PoW task is arbitrary, hardware optimization could drift away from useful work.</p>
<p>Bitcoin shows what happens when you don't align incentives. Over time, the industry built specialized hardware designed only for finding hashes.</p>
<p>With transformer-based PoW, the incentive structure flips. If the PoW task <em>is</em> transformer inference, then optimizing hardware for PoW also optimizes hardware for serving real users. The hardware optimization naturally points toward something useful.</p>
<p>The key is to ensure two things: the task must actually be real transformer inference, and it must be fresh each epoch so participants can't pre-compute solutions outside the allocated time window.</p>
<p>Each PoW round derives a new, randomly initialized transformer. Participants see the challenge and have only a fixed time window to solve it, so there's no opportunity to analyze or pre-compute ahead of time. Every challenge is genuinely novel. This keeps the work aligned with real inference. You can't shortcut it, and you can't specialize hardware for a specific task because the task changes every round. Hardware improvements benefit general transformer inference, not mining-specific optimization.</p>
<h2 id="heading-challenge-3-security">Challenge 3: Security</h2>
<p>Finally, there's the question of hardness: is the PoW problem actually secure?</p>
<p>In Bitcoin, the security argument is straightforward. Finding a hash with leading zeros requires brute force. SHA-256 has no known mathematical shortcuts. The problem is hard in a simple, well-understood way.</p>
<p>The Bitcoin setup is pretty simple: vary a nonce (a number) and check if the resulting hash starts with leading zeros. The nonce is just a scalar you increment.</p>
<p>Let's try to understand what a direct translation of a Bitcoin task would be to a transformer-based setup. The nonce becomes an input sequence, a vector or token sequence that you vary, which can still be generated by positive integers, just like Bitcoin nonces. The "leading zeros" become a condition on the output. The transformer's output vector must satisfy some property. This could be that the output is close to zero, or falls within a certain distance of a target vector, or has a specific magnitude, or satisfies some other well-defined criterion. The exact definition matters because some conditions might have a mathematical structure that allows shortcuts.</p>
<p>The key difference from Bitcoin is that running the transformer to check a candidate is expensive. Bitcoin can hash millions of nonces per second on commodity hardware. Checking a transformer input requires actual forward pass computation. Participants can't brute-force billions of candidates, they're limited by inference speed. This is the computational work we're measuring.</p>
<p>Making this system competitively secure with Bitcoin is a more involved technical story – one for a different post. The basic idea is that randomly initialized transformers, combined with careful problem design, create a search space that requires running full transformer inference. The full security analysis deserves its own treatment.</p>
<p>---</p>
<p>Proof-of-work has worked well for 15 years. But Bitcoin's design also means we spend enormous computational resources finding hashes with no utility, while alternatives like PoS concentrate wealth among existing capital holders.</p>
<p>Transformer-based PoW is another option. It keeps PoW's security and fairness properties but directs computational effort toward something the world actually needs, a consensus mechanism for the AI era: secure like PoW, aligned with real computational needs, and genuinely useful. That's a different foundation to build on.</p>
]]></content:encoded></item><item><title><![CDATA[Build a production‑ready AI app on Gonka: end‑to‑end guide]]></title><description><![CDATA[Gonka is a proof‑of‑work AI network that load‑balances OpenAI‑style inference across verified hardware providers, pays in GNK, and spot‑checks results to…]]></description><link>https://what-is-gonka.hashnode.dev/build-a-productionready-ai-app-on-gonka-endtoend-guide</link><guid isPermaLink="true">https://what-is-gonka.hashnode.dev/build-a-productionready-ai-app-on-gonka-endtoend-guide</guid><category><![CDATA[AI]]></category><category><![CDATA[inference]]></category><category><![CDATA[compute]]></category><category><![CDATA[depin]]></category><dc:creator><![CDATA[Dima Orekhov]]></dc:creator><pubDate>Tue, 21 Oct 2025 17:54:47 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1761065200726/fe13bbb1-0d19-4884-b0ae-b4ca601d2916.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Gonka is a proof‑of‑work AI network that load‑balances OpenAI‑style inference across verified hardware providers, pays in GNK, and spot‑checks results to deter fraud. If you can call an OpenAI‑style API, you can ship on Gonka. For a more detailed overview, see the <a target="_blank" href="https://gonka.ai/">Gonka website</a>.</p>
<p>Here’s what we’re building — imagine we’re at a whiteboard together:</p>
<ul>
<li><p>A tiny Spanish practice app that runs an endless loop of bite‑sized exercises.</p>
</li>
<li><p>You click “Start New Exercise,” the AI generates a single cloze prompt (one blank in a sentence).</p>
</li>
<li><p>You type your answer, click “Check Answer,” and the AI grades it with a short, personal explanation.</p>
</li>
<li><p>You move on to the next one. Rinse, repeat. It’s fast, focused, and a little addictive.</p>
</li>
</ul>
<p>Live demo: <a target="_blank" href="https://carrera.gonka.ai/">carrera.gonka.ai</a></p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1761064524343/4ff87af3-83ac-47be-9603-025ada1e0289.gif" alt class="image--center mx-auto" /></p>
<p>Under the hood, there are two pieces: a React frontend and a 50‑line Node proxy that signs requests to Gonka. That’s the only new concept vs. a typical OpenAI integration — we sign on the server so your key stays safe, and everything else looks like a standard OpenAI‑style chat completions call.</p>
<p>High‑level flow at a glance:  </p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1761064834485/17910f79-9528-4b5e-81c2-a76f9d7a1c76.png" alt class="image--center mx-auto" /></p>
<p>Along the way, we’ll cover a quick developer account setup (so you can sign requests), a tiny prompting strategy, and reproducible build/deploy steps. If you already know OpenAI’s API, you’ll feel right at home.</p>
<p>The two components of the app are:</p>
<ul>
<li><p>A Vite React web app that can target any OpenAI‑compatible endpoint (mock/local/prod)</p>
</li>
<li><p>A Node/Express proxy that signs requests using your Gonka account and forwards them to the network</p>
</li>
</ul>
<p>Let's now run the app locally with your own Gonka account!</p>
<h1 id="heading-quick-start">Quick Start</h1>
<p><strong>Prereqs:</strong> Node 20+ installed</p>
<ol>
<li>Clone this repository</li>
</ol>
<pre><code class="lang-bash">git <span class="hljs-built_in">clone</span> git@github.com:product-science/carrera.git
<span class="hljs-built_in">cd</span> carrera
</code></pre>
<ol start="2">
<li>Create a Gonka account and set env</li>
</ol>
<pre><code class="lang-bash"><span class="hljs-comment"># Create with inferenced CLI</span>
<span class="hljs-comment"># See quickstart docs for how to download the CLI:</span>
<span class="hljs-comment"># https://gonka.ai/developer/quickstart/#2-create-an-account</span>

<span class="hljs-comment"># ACCOUNT_NAME can be any locally-unique name. </span>
<span class="hljs-comment"># Just a human-readable name for you account key-pair</span>
ACCOUNT_NAME=<span class="hljs-string">"carrera-quickstart"</span>

NODE_URL=http://node2.gonka.ai:8000
inferenced create-client <span class="hljs-string">"<span class="hljs-variable">$ACCOUNT_NAME</span>"</span> \
  --node-address <span class="hljs-string">"<span class="hljs-variable">$NODE_URL</span>"</span>

<span class="hljs-comment"># Export your private key for server-side use only</span>
<span class="hljs-built_in">export</span> GONKA_PRIVATE_KEY=$(inferenced keys <span class="hljs-built_in">export</span> <span class="hljs-string">"<span class="hljs-variable">$ACCOUNT_NAME</span>"</span> --unarmored-hex --unsafe)
</code></pre>
<ol start="3">
<li>Run the proxy</li>
</ol>
<pre><code class="lang-bash"><span class="hljs-built_in">cd</span> gonka-proxy
npm i
npm run build
NODE_URL=http://node2.gonka.ai:8000 ALLOWED_ORIGINS=http://localhost:5173 PORT=8080 npm start
</code></pre>
<ol start="4">
<li>Health check the proxy</li>
</ol>
<pre><code class="lang-bash">curl http://localhost:8080/healthz
<span class="hljs-comment"># Expect: {"ok":true,"ready":true}</span>
</code></pre>
<ol start="5">
<li>Run the frontend (preconfigure Base URL)</li>
</ol>
<pre><code class="lang-bash"><span class="hljs-built_in">cd</span> web
npm i
VITE_DEFAULT_API_BASE_URL=http://localhost:8080/v1 npm run dev
</code></pre>
<ol start="6">
<li>In the app, open Settings → the Base URL is prefilled. Choose a model (e.g., <code>Qwen/Qwen3-235B-A22B-Instruct-2507-FP8</code>, you can view the list of currently available models at <a target="_blank" href="https://node1.gonka.ai:8443/api/v1/models">https://node1.gonka.ai:8443/api/v1/models</a>) and click “Test connection”.</li>
</ol>
<h1 id="heading-walkthrough-the-gonka-proxy"><strong>Walkthrough: the Gonka proxy</strong></h1>
<p>This is the Gonka integration point. The proxy signs requests with your key, then forwards OpenAI‑style calls to the network. You can spin up a proxy like this for your own app and start processing inferences on Gonka. You’ll likely want to add authentication so only authorized users can request inference.</p>
<p>Environment variables used by the proxy:</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// gonka-proxy/src/env.ts</span>
<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> env = {
  PORT: num(<span class="hljs-string">"PORT"</span>, <span class="hljs-number">8080</span>),
  GONKA_PRIVATE_KEY: str(<span class="hljs-string">"GONKA_PRIVATE_KEY"</span>),
  NODE_URL: str(<span class="hljs-string">"NODE_URL"</span>),
  ALLOWED_ORIGINS: (process.env.ALLOWED_ORIGINS ?? <span class="hljs-string">"*"</span>)
    .split(<span class="hljs-string">","</span>)
    .map(<span class="hljs-function">(<span class="hljs-params">s</span>) =&gt;</span> s.trim())
    .filter(<span class="hljs-built_in">Boolean</span>),
};
</code></pre>
<p>Then you create the Gonka client using the Gonka OpenAI SDK for TypeScript (gonka-openai). Go and Python versions are also available; more language support is coming. Keep an eye on the <a target="_blank" href="https://github.com/gonka-ai/gonka-openai">repo</a>.</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// gonka-proxy/src/gonka.ts</span>
<span class="hljs-keyword">import</span> { GonkaOpenAI, resolveEndpoints } <span class="hljs-keyword">from</span> <span class="hljs-string">"gonka-openai"</span>;
<span class="hljs-keyword">import</span> { env } <span class="hljs-keyword">from</span> <span class="hljs-string">"./env"</span>;

<span class="hljs-keyword">export</span> <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">createGonkaClient</span>(<span class="hljs-params"></span>) </span>{
  <span class="hljs-keyword">const</span> endpoints = <span class="hljs-keyword">await</span> resolveEndpoints({ sourceUrl: env.NODE_URL });
  <span class="hljs-keyword">return</span> <span class="hljs-keyword">new</span> GonkaOpenAI({ gonkaPrivateKey: env.GONKA_PRIVATE_KEY, endpoints });
}
</code></pre>
<p>Expose the OpenAI‑compatible chat completions endpoint (<code>/v1/chat/completions</code>):</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// gonka-proxy/src/server.ts</span>
app.get(<span class="hljs-string">"/healthz"</span>, <span class="hljs-function">(<span class="hljs-params">_req, res</span>) =&gt;</span> res.json({ ok: <span class="hljs-literal">true</span>, ready }));

app.post(<span class="hljs-string">"/v1/chat/completions"</span>, <span class="hljs-keyword">async</span> (req, res) =&gt; {
  <span class="hljs-keyword">if</span> (!ready || !client) <span class="hljs-keyword">return</span> res.status(<span class="hljs-number">503</span>).json({ error: { message: <span class="hljs-string">"Proxy not ready"</span> } });
  <span class="hljs-keyword">const</span> body = req.body <span class="hljs-keyword">as</span> ChatCompletionRequest;
  <span class="hljs-keyword">if</span> (!body || !body.model || !<span class="hljs-built_in">Array</span>.isArray(body.messages)) {
    <span class="hljs-keyword">return</span> res.status(<span class="hljs-number">400</span>).json({ error: { message: <span class="hljs-string">"'model' and 'messages' are required"</span> } });
  }

  <span class="hljs-keyword">try</span> {
    <span class="hljs-keyword">const</span> streamRequested = <span class="hljs-built_in">Boolean</span>(body.stream);
    <span class="hljs-keyword">const</span> { stream: _ignored, ...rest } = body;

    <span class="hljs-keyword">if</span> (!streamRequested) {
      <span class="hljs-keyword">const</span> response = <span class="hljs-keyword">await</span> client.chat.completions.create({ ...rest, stream: <span class="hljs-literal">false</span> });
      <span class="hljs-keyword">return</span> res.status(<span class="hljs-number">200</span>).json(response);
    }
  <span class="hljs-comment">// ...</span>
</code></pre>
<p>Streaming passthrough (SSE) is also included in server.ts, so you can toggle stream: true from clients that support it.</p>
<p>There's also a Dockerfile for reproducible builds and easy deployment of the app.</p>
<pre><code class="lang-dockerfile"><span class="hljs-comment"># gonka-proxy/Dockerfile</span>
<span class="hljs-comment"># ---- build ----</span>
<span class="hljs-keyword">FROM</span> node:<span class="hljs-number">20</span>-alpine AS build
<span class="hljs-keyword">WORKDIR</span><span class="bash"> /app</span>
<span class="hljs-keyword">COPY</span><span class="bash"> package*.json tsconfig.json ./</span>
<span class="hljs-keyword">COPY</span><span class="bash"> src ./src</span>
<span class="hljs-keyword">RUN</span><span class="bash"> npm ci &amp;&amp; npm run build</span>

<span class="hljs-comment"># ---- runtime ----</span>
<span class="hljs-keyword">FROM</span> node:<span class="hljs-number">20</span>-alpine
<span class="hljs-keyword">WORKDIR</span><span class="bash"> /app</span>
<span class="hljs-keyword">ENV</span> NODE_ENV=production
<span class="hljs-keyword">COPY</span><span class="bash"> --from=build /app/package*.json ./</span>
<span class="hljs-keyword">RUN</span><span class="bash"> npm ci --omit=dev</span>
<span class="hljs-keyword">COPY</span><span class="bash"> --from=build /app/dist ./dist</span>
<span class="hljs-keyword">EXPOSE</span> <span class="hljs-number">8080</span>
<span class="hljs-keyword">CMD</span><span class="bash"> [<span class="hljs-string">"node"</span>, <span class="hljs-string">"dist/server.js"</span>]</span>
</code></pre>
<h1 id="heading-walkthrough-frontend-provideragnostic"><strong>Walkthrough: frontend (provider‑agnostic)</strong></h1>
<p>The React app stays OpenAI‑compatible and doesn’t need to know about Gonka specifics. It just calls an OpenAI‑style endpoint and renders the loop you saw above.</p>
<p>All backend interactions happen in <code>web/src/llmClient.ts</code>. It makes a single call to the OpenAI‑style chat completions endpoint:</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// web/src/llmClient.ts</span>
  <span class="hljs-keyword">const</span> url = <span class="hljs-string">`<span class="hljs-subst">${s.baseUrl.replace(<span class="hljs-regexp">/\/+$/</span>, <span class="hljs-string">""</span>)}</span>/chat/completions`</span>;
  <span class="hljs-keyword">const</span> headers: Record&lt;<span class="hljs-built_in">string</span>, <span class="hljs-built_in">string</span>&gt; = { <span class="hljs-string">"Content-Type"</span>: <span class="hljs-string">"application/json"</span> };
  <span class="hljs-keyword">if</span> (s.apiKey) headers.Authorization = <span class="hljs-string">`Bearer <span class="hljs-subst">${s.apiKey}</span>`</span>; <span class="hljs-comment">// optional</span>

  <span class="hljs-keyword">const</span> res = <span class="hljs-keyword">await</span> fetch(url, {
    method: <span class="hljs-string">"POST"</span>,
    headers,
    body: <span class="hljs-built_in">JSON</span>.stringify({ model: s.model, messages, temperature }),
    signal,
  });

  <span class="hljs-keyword">if</span> (!res.ok) {
    <span class="hljs-keyword">const</span> body = <span class="hljs-keyword">await</span> res.text().catch(<span class="hljs-function">() =&gt;</span> <span class="hljs-string">""</span>);
    <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> <span class="hljs-built_in">Error</span>(<span class="hljs-string">`LLM error <span class="hljs-subst">${res.status}</span>: <span class="hljs-subst">${body}</span>`</span>);
  }
  <span class="hljs-keyword">const</span> data = <span class="hljs-keyword">await</span> res.json();
  <span class="hljs-keyword">const</span> text = data?.choices?.[<span class="hljs-number">0</span>]?.message?.content ?? <span class="hljs-string">""</span>;
  <span class="hljs-keyword">return</span> { text, raw: data };
</code></pre>
<p>To point the app at a particular API provider, open the Settings modal in the header and set the Base URL. You can also configure the model there; the default is <code>Qwen/Qwen3-235B-A22B-Instruct-2507-FP8</code>. For local testing without a network, use <code>mock:</code> as the Base URL; for real calls, set it to your proxy (we prefilled it in Quick Start).</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// web/src/settings.ts</span>
<span class="hljs-keyword">export</span> <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">getDefaultSettings</span>(<span class="hljs-params"></span>): <span class="hljs-title">Settings</span> </span>{
  <span class="hljs-keyword">const</span> prodBase = (<span class="hljs-keyword">import</span>.meta <span class="hljs-keyword">as</span> <span class="hljs-built_in">any</span>).env?.VITE_DEFAULT_API_BASE_URL || <span class="hljs-string">""</span>;
  <span class="hljs-keyword">const</span> baseUrlDefault = prodBase || <span class="hljs-string">"mock:"</span>;
  <span class="hljs-keyword">return</span> { baseUrl: baseUrlDefault, apiKey: <span class="hljs-string">""</span>, model: <span class="hljs-string">"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8"</span> };
}
</code></pre>
<h1 id="heading-how-the-app-prompts-and-grades"><strong>How the app prompts and grades</strong></h1>
<p>We use two prompts:</p>
<ul>
<li><p>Generation prompt: “You are a Spanish language teacher… output STRICT JSON only… produce a cloze sentence with exactly one blank (____), an answer, instructions, and difficulty.”</p>
</li>
<li><p>Grading prompt: “You are a Spanish teacher grading… output STRICT JSON with status pass/fail and an explanation.”</p>
</li>
</ul>
<p>Example slices from the generation flow:</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// web/src/App.tsx</span>
      <span class="hljs-keyword">const</span> sys: ChatMsg = {
        role: <span class="hljs-string">"system"</span>,
        content: <span class="hljs-string">`You are a Spanish language teacher creating engaging exercises.
Output STRICT JSON only (no prose, no code fences). The object must have exactly these keys:
{
  "type": "cloze",
  "text": "&lt;Spanish sentence with EXACTLY ONE blank marked as ____&gt;",
  "answer": "&lt;the single correct answer for the blank&gt;",
  "instructions": "&lt;clear instruction&gt;",
  "difficulty": "beginner|intermediate|advanced"
}

Content policy:
- Use natural, everyday contexts.
- One blank only (use the exact token ____ in the sentence).
- Keep the sentence 8–20 words.
- Spanish only in text; English allowed in instructions.
- Vary focus: ser/estar, preterite vs imperfect, subjunctive, por/para, agreement, common prepositions, core vocabulary.`</span>
      };
</code></pre>
<p>We add a couple of few‑shot examples so the model “sees” what to output, then parse the JSON and render the exercise. For grading, we do the same with a strict JSON schema for pass/fail, plus a brief explanation.</p>
<p>Parsing is resilient — it strips code fences and extracts the first JSON block if needed:</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// web/src/App.tsx</span>
<span class="hljs-comment">// Best-effort JSON extractor to tolerate models that wrap output in prose or code fences</span>
<span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">extractJsonObject</span>&lt;<span class="hljs-title">TExpected</span>&gt;(<span class="hljs-params">raw: <span class="hljs-built_in">string</span></span>): <span class="hljs-title">TExpected</span> </span>{
  <span class="hljs-keyword">const</span> trimmed = <span class="hljs-built_in">String</span>(raw ?? <span class="hljs-string">""</span>).trim();
  <span class="hljs-keyword">const</span> fence = trimmed.match(<span class="hljs-regexp">/```(?:json)?\s*([\s\S]*?)\s*```/i</span>);
  <span class="hljs-keyword">const</span> candidate = fence ? fence[<span class="hljs-number">1</span>] : trimmed;
  <span class="hljs-keyword">const</span> tryParse = <span class="hljs-function">(<span class="hljs-params">s: <span class="hljs-built_in">string</span></span>) =&gt;</span> {
    <span class="hljs-keyword">try</span> {
      <span class="hljs-keyword">return</span> <span class="hljs-built_in">JSON</span>.parse(s) <span class="hljs-keyword">as</span> TExpected;
    } <span class="hljs-keyword">catch</span> {
      <span class="hljs-keyword">return</span> <span class="hljs-literal">undefined</span>;
    }
  };
  <span class="hljs-keyword">const</span> direct = tryParse(candidate);
  <span class="hljs-keyword">if</span> (direct) <span class="hljs-keyword">return</span> direct;
  <span class="hljs-keyword">const</span> start = candidate.indexOf(<span class="hljs-string">"{"</span>);
  <span class="hljs-keyword">const</span> end = candidate.lastIndexOf(<span class="hljs-string">"}"</span>);
  <span class="hljs-keyword">if</span> (start !== <span class="hljs-number">-1</span> &amp;&amp; end !== <span class="hljs-number">-1</span> &amp;&amp; end &gt; start) {
    <span class="hljs-keyword">const</span> block = candidate.slice(start, end + <span class="hljs-number">1</span>);
    <span class="hljs-keyword">const</span> parsed = tryParse(block);
    <span class="hljs-keyword">if</span> (parsed) <span class="hljs-keyword">return</span> parsed;
  }
  <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> <span class="hljs-built_in">Error</span>(<span class="hljs-string">"Failed to parse JSON from model response."</span>);
}
</code></pre>
<h1 id="heading-conclusion-and-next-steps"><strong>Conclusion and next steps</strong></h1>
<p>You’ve seen the full loop: a tiny proxy that signs and forwards OpenAI‑style requests to Gonka, and a simple, provider‑agnostic frontend that generates and grades exercises.</p>
<p>You now have a clean, reproducible path to run AI apps on a decentralized network — with almost the same developer experience as calling a centralized OpenAI‑style API.</p>
<p>Try these next:</p>
<ul>
<li><p>Add your own exercise types and grading rules</p>
</li>
<li><p>Turn on streaming UI (the proxy already supports SSE)</p>
</li>
<li><p>Add authentication and rate limiting to the Gonka proxy</p>
</li>
<li><p>Deploy the proxy to your infra and set <code>VITE_DEFAULT_API_BASE_URL</code> at build time for the web app</p>
</li>
</ul>
]]></content:encoded></item><item><title><![CDATA[Beyond the Data Center: How AI Training Went Decentralized]]></title><description><![CDATA[The relentless scaling of AI has pushed training beyond the physical limits of any one location, forcing researchers to confront a fundamental challenge…]]></description><link>https://what-is-gonka.hashnode.dev/beyond-the-data-center-how-ai-training-went-decentralized</link><guid isPermaLink="true">https://what-is-gonka.hashnode.dev/beyond-the-data-center-how-ai-training-went-decentralized</guid><dc:creator><![CDATA[Egor Shulgin]]></dc:creator><pubDate>Tue, 14 Oct 2025 15:37:42 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1760458066246/65f3f2cb-ee46-48e9-8332-33e8b0048772.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The relentless scaling of AI has pushed training beyond the physical limits of any one location, forcing researchers to confront a fundamental challenge: how do you coordinate thousands of processors when they're separated by continents, not corridors? The answer lies in more efficient algorithms – methods that work by communicating less. This shift, powered by breakthroughs in federated optimization and crystallized in frameworks like DiLoCo, is enabling organizations to train billion-parameter models over standard internet connections, opening new possibilities for collaborative AI development at scale.</p>
<h3 id="heading-1-the-starting-point-distributed-training-in-data-centers"><strong>1. The Starting Point: Distributed Training in Data Centers</strong></h3>
<p>Modern AI training is inherently distributed. It has been consistently observed that scaling up data, parameters, and computation leads to substantial improvements in model performance, making it impossible to train foundation models (billions of parameters) on a single machine. The industry's default solution is the "centralized-distributed" model: purpose-built data centers housing thousands of GPUs in a single location, connected by ultra-fast networks like NVIDIA's NVLink or InfiniBand. These specialized interconnects operate at speeds orders of magnitude higher than standard networking, enabling the GPUs to function as one cohesive system.</p>
<p>The most common training strategy in this setting is <strong>data parallelism</strong>, where the dataset is split across GPUs. (Other approaches exist, like pipeline or tensor parallelism, which split the model itself across GPUs and are required for training the largest models, though they are more complex to implement.) Here's how a single training step using <strong>mini-batch Stochastic Gradient Descent (SGD)</strong> works (the same principle applies to Adam):</p>
<ol>
<li><p><strong>Replicate and Distribute:</strong> Load a copy of the model onto each GPU. Split the training data into small mini-batches.</p>
</li>
<li><p><strong>Parallel Computation:</strong> Each GPU independently processes a different mini-batch and calculates a gradient – the direction to adjust the model's parameters.</p>
</li>
<li><p><strong>Synchronize and Aggregate:</strong> All GPUs pause, share their gradients, and average them to produce a single, consolidated update.</p>
</li>
<li><p><strong>Update:</strong> Apply this averaged update to every GPU's model copy, ensuring they all remain identical.</p>
</li>
<li><p><strong>Repeat:</strong> Move to the next mini-batch and start again.</p>
</li>
</ol>
<p>In essence, this is a cycle of parallel computation followed by mandatory synchronization. The constant communication, happening after every single training step, is only feasible because of the expensive, high-speed connections inside the data center. This reliance on frequent synchronization is the defining characteristic of centralized-distributed training. It works brilliantly until you leave the building.</p>
<h3 id="heading-2-hitting-the-wall-the-great-communication-bottleneck"><strong>2. Hitting the Wall: The Great Communication Bottleneck</strong></h3>
<p>To train the largest models, organizations must now build infrastructure at a breathtaking scale, often requiring multiple data centers in different cities or continents. This geographical separation creates a formidable obstacle. The algorithmic approach of step-by-step synchronization, which thrives inside a data center, breaks down when stretched across the globe.</p>
<p>The problem is network speed. Within a data center, InfiniBand transfers data at 400 Gb/s or more. The wide-area networks (WANs) connecting distant centers operate at speeds closer to 1 Gbps. That's a performance gap of several orders of magnitude, driven by the fundamental limits of distance and cost. Mini-batch SGD, with its assumption of near-instant communication, is incompatible with this reality.</p>
<p>This disparity creates a crippling bottleneck. When model parameters must synchronize after every step, powerful GPUs spend most of their time idle, waiting for data to crawl across slow networks. The result: the AI community cannot harness the massive computational resources distributed globally, from enterprise servers to consumer hardware, because existing algorithms require high-speed, centralized networks. This represents a vast, untapped reservoir of power.</p>
<h3 id="heading-3-the-algorithmic-shift-federated-optimization"><strong>3. The Algorithmic Shift: Federated Optimization</strong></h3>
<p>If frequent communication is the problem, the solution is to communicate less. This simple insight is the foundation of an algorithmic shift that adapts techniques from <strong>Federated Learning</strong> – a field originally focused on training models on decentralized data residing on endpoint devices, such as mobile phones, while preserving privacy. The core algorithm, <a target="_blank" href="https://arxiv.org/abs/1602.05629"><strong>Federated Averaging (FedAvg)</strong></a>, showed that by allowing each device to perform multiple training steps locally before sending an update, the number of required communication rounds could be reduced by orders of magnitude.</p>
<p>Researchers realized this principle, doing more independent work between synchronizations, was the perfect solution for the performance bottleneck in geo-distributed setups. This led to the <a target="_blank" href="https://arxiv.org/abs/2003.00295">Federated Optimization (FedOpt)</a> framework, which decouples local computation from global communication using a dual-optimizer approach.</p>
<p>The framework operates with two distinct optimizers:</p>
<ul>
<li><p>An <strong>Inner Optimizer</strong> (such as standard SGD) runs on each machine, performing many independent training steps on its local data slice. Each model replica makes significant progress on its own.</p>
</li>
<li><p>An <strong>Outer Optimizer</strong> handles infrequent global synchronization. After many local steps, each worker calculates the total change in its model parameters. These changes are aggregated, and the outer optimizer uses this averaged update to adjust the global model for the next cycle.</p>
</li>
</ul>
<p>This dual-optimizer architecture fundamentally changes the training dynamic. Instead of frantic, step-by-step communication between all nodes, the process becomes a series of extended, independent computation periods followed by a single aggregated update. This algorithmic shift, born from privacy research, provided the crucial breakthrough needed to make training over slow networks practical. The question was: could it work for massive language models?</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1760456440228/9630a74e-2cd9-45e6-a085-392fad94024d.png" alt="Source: Charles, Z., et al. (2025). &quot;Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo.&quot; arXiv:2503.09799" class="image--center mx-auto" /></p>
<p><em>Source: Charles, Z., et al. (2025). "Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo." arXiv:2503.09799</em></p>
<h3 id="heading-4-the-breakthrough-diloco-proves-it-works-at-scale"><strong>4. The Breakthrough: DiLoCo Proves It Works at Scale</strong></h3>
<p>The answer came in the form of <a target="_blank" href="https://arxiv.org/abs/2311.08105"><strong>DiLoCo (Distributed Low-Communication)</strong></a>, an algorithm that demonstrated the practical viability of federated optimization for large language models. DiLoCo provided a specific, carefully tuned recipe for training modern transformers over slow networks:</p>
<ul>
<li><p><strong>Inner Optimizer:</strong> <strong>AdamW</strong>, the state-of-the-art optimizer for large language models, runs for many local training steps on each worker.</p>
</li>
<li><p><strong>Outer Optimizer:</strong> <strong>Nesterov momentum</strong>, a robust, well-understood algorithm, handles the infrequent global update.</p>
</li>
</ul>
<p>The original experiments showed that DiLoCo could match the performance of fully synchronous data-center training while reducing communication between nodes by up to 500x. This was practical proof that training giant models over the internet is achievable.</p>
<p>The breakthrough gained rapid traction. <a target="_blank" href="https://arxiv.org/abs/2407.07852"><strong>OpenDiLoCo</strong></a>, an open-source implementation, reproduced the original results and integrated the algorithm into a truly peer-to-peer framework using the Hivemind library, making the technique more accessible. This momentum culminated in successful large-scale pre-training runs by organizations like <a target="_blank" href="https://arxiv.org/abs/2412.01152"><strong>PrimeIntellect</strong></a>, <a target="_blank" href="https://nousresearch.com/nous-psyche/"><strong>Nous Research</strong></a>, and <a target="_blank" href="https://arxiv.org/abs/2411.02908"><strong>FlowerLabs</strong></a>, which demonstrated successful pre-training of billion-parameter models using low-communication algorithms over the internet. These pioneering efforts moved DiLoCo-style training from a promising research paper to a validated method for building foundation models outside centralized providers.</p>
<h3 id="heading-5-the-frontier-advanced-techniques-and-future-research"><strong>5. The Frontier: Advanced Techniques and Future Research</strong></h3>
<p>The success of DiLoCo has ignited a surge of research focused on pushing efficiency and scale even further. A crucial step in maturing the approach has been the development of <a target="_blank" href="https://arxiv.org/abs/2503.09799"><strong>DiLoCo Scaling Laws</strong></a>, which establish that DiLoCo's performance scales predictably and robustly with model size. These scaling laws predict that as models grow larger, a well-tuned DiLoCo can outperform traditional data-parallel training on a fixed compute budget while using orders of magnitude less bandwidth.</p>
<p>To handle models at the 100B+ parameter scale, researchers have extended DiLoCo's design with techniques like <a target="_blank" href="https://arxiv.org/abs/2506.21263"><strong>DiLoCoX</strong></a>, which combines the dual-optimizer approach with pipeline parallelism. DiLoCoX enabled pre-training of a 107B parameter model over a standard 1 Gbps network. Further refinements include <a target="_blank" href="https://arxiv.org/abs/2501.18512"><strong>Streaming DiLoCo</strong></a>, which overlaps communication and computation to hide network latency, and <a target="_blank" href="https://arxiv.org/abs/2401.09135"><strong>asynchronous approaches</strong></a> that prevent a single slow node from bottlenecking the entire system.</p>
<p>Innovation is also happening at the algorithmic core. Research into newer inner optimizers like <strong>Muon</strong> has led to <a target="_blank" href="https://arxiv.org/abs/2505.23725"><strong>MuLoCo</strong></a>, a variant that allows model updates to be compressed down to 2 bits with negligible performance loss, resulting in an 8x reduction in data transfer. Perhaps the most ambitious research direction is <strong>model parallelism over the internet</strong>, where the model itself is split across different machines. Early research in this area, such as <a target="_blank" href="https://arxiv.org/abs/2301.11913"><strong>SWARM parallelism</strong></a>, developed fault-tolerant methods for distributing model layers across heterogeneous and unreliable devices connected by slow networks. Building on these concepts, groups like <a target="_blank" href="https://arxiv.org/abs/2506.01260"><strong>Pluralis Research</strong></a> have demonstrated the possibility of training billion-parameter models where different layers are hosted on GPUs in entirely different geographic locations, opening the door to training on distributed consumer hardware connected only by standard internet.</p>
<h3 id="heading-6-the-trust-challenge-governance-in-open-networks"><strong>6. The Trust Challenge: Governance in Open Networks</strong></h3>
<p>As training moves from controlled data centers to open, permissionless networks, a fundamental problem emerges: trust. In a truly decentralized system with no central authority, how can participants verify that the updates they receive from others are legitimate? What prevents a malicious participant from poisoning the model or a lazy one from claiming rewards for work they never did? This governance problem is the final hurdle.</p>
<p>One line of defense is <strong>Byzantine robustness</strong> – a concept from distributed computing that designs systems to function even when some participants are faulty or actively malicious. In a centralized system, a server can apply robust aggregation rules to discard malicious updates. This is harder in a peer-to-peer setting where there is no central aggregator. Instead, each honest node must evaluate updates from its neighbors and decide which to trust and which to discard.</p>
<p>Another approach involves cryptographic techniques that replace trust with verification. An early idea, <a target="_blank" href="https://arxiv.org/abs/2103.05633"><strong>Proof-of-Learning</strong></a>, proposed that participants log training checkpoints to prove they expended the necessary computation. Other techniques like <strong>Zero-Knowledge Proofs (ZKPs)</strong> allow a worker to prove they executed the required training steps correctly without revealing underlying data, though their current computational overhead remains a challenge for verifying training of today's massive foundation models.</p>
<h4 id="heading-conclusion-the-dawn-of-a-new-ai-paradigm"><strong>Conclusion: The Dawn of a New AI Paradigm</strong></h4>
<p>The journey from the high-walled data center to the open internet marks a profound shift in how artificial intelligence is created. We began with the physical limits of centralized training, where progress was dictated by access to expensive, co-located hardware. This led to the communication bottleneck, a wall that made training giant models over distributed networks impractical. Yet this wall was not broken by faster cables, but by more efficient algorithms.</p>
<p>The algorithmic shift, rooted in federated optimization and crystallized by DiLoCo, proved that communicating less frequently was the key. This breakthrough is now being rapidly advanced by techniques that establish scaling laws, overlap communication, explore novel optimizers, and even parallelize the model itself across the internet. The successful pre-training of billion-parameter models by a diverse ecosystem of researchers and companies is a testament to this new paradigm's power.</p>
<p>With the trust challenge being addressed through robust defenses and cryptographic verification, the path is clearing. Decentralized training is evolving from an engineering solution into a foundational pillar for a more open, collaborative, and accessible AI future. It promises a world where the ability to build powerful models is not confined to a handful of tech giants, but is distributed globally, unlocking the collective computational power and ingenuity of everyone.</p>
<hr />
<h3 id="heading-references"><strong>References</strong></h3>
<p>McMahan, H. B., et al. (2017). <a target="_blank" href="https://arxiv.org/abs/1602.05629"><em>Communication-Efficient Learning of Deep Networks from Decentralized Data</em></a>. International Conference on Artificial Intelligence and Statistics (AISTATS).</p>
<p>Reddi, S., et al. (2021). <a target="_blank" href="https://arxiv.org/abs/2003.00295"><em>Adaptive Federated Optimization</em></a>. International Conference on Learning Representations (ICLR).</p>
<p>Jia, H., et al. (2021). <a target="_blank" href="https://arxiv.org/abs/2103.05633"><em>Proof-of-Learning: Definitions and Practice</em></a>. IEEE Symposium on Security and Privacy.</p>
<p>Ryabinin, Max, et al. (2023). <a target="_blank" href="https://arxiv.org/abs/2301.11913">SWARM parallelism: Training large models can be surprisingly communication-efficient</a>. International Conference on Machine Learning (ICML).</p>
<p>Douillard, A., et al. (2023). <a target="_blank" href="https://arxiv.org/abs/2311.08105"><em>DiLoCo: Distributed Low-Communication Training of Language Models</em></a>.</p>
<p>Jaghouar, S., Ong, J. M., &amp; Hagemann, J. (2024). <a target="_blank" href="https://arxiv.org/abs/2407.07852"><em>OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training</em></a>.</p>
<p>Jaghouar, S., et al. (2024). <a target="_blank" href="https://arxiv.org/abs/2412.01152"><em>INTELLECT-1 Technical Report</em></a>.</p>
<p>Liu, B., et al. (2024). <a target="_blank" href="https://arxiv.org/abs/2401.09135"><em>Asynchronous Local-SGD Training for Language Modeling</em></a>.</p>
<p>Charles, Z., et al. (2025). <a target="_blank" href="https://arxiv.org/abs/2503.09799"><em>Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo</em></a>.</p>
<p>Douillard, A., et al. (2025). <a target="_blank" href="https://arxiv.org/abs/2501.18512"><em>Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch</em></a>.</p>
<p>Psyche Team. (2025). <a target="_blank" href="https://nousresearch.com/nous-psyche/"><em>Democratizing AI: The Psyche Network Architecture</em></a>. Nous Research Blog.</p>
<p>Qi, J., et al. (2025). <a target="_blank" href="https://arxiv.org/abs/2506.21263"><em>DiLoCoX: A Low-Communication Large-Scale Training Framework for Decentralized Cluster</em></a>.</p>
<p>Sani, L., et al. (2025). <a target="_blank" href="https://arxiv.org/abs/2411.02908"><em>Photon: Federated LLM Pre-Training</em></a>. Proceedings of the Conference on Machine Learning and Systems (MLSys).</p>
<p>Thérien, B., et al. (2025). <a target="_blank" href="https://arxiv.org/abs/2505.23725"><em>MuLoCo: Muon is a practical inner optimizer for DiLoCo</em></a>.</p>
<p>Long, A., et al. (2025). <a target="_blank" href="https://arxiv.org/abs/2506.01260"><em>Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism</em></a>.</p>
]]></content:encoded></item><item><title><![CDATA[Decentralized AI Inference Validation: Building Trust in Trustless Environments]]></title><description><![CDATA[Validating a claimed AI model is critical for decentralized networks. Address GPU non-determinism, model substitution, quantization, and pre-fill attacks.]]></description><link>https://what-is-gonka.hashnode.dev/decentralized-ai-inference-validation-building-trust-in-trustless-environments</link><guid isPermaLink="true">https://what-is-gonka.hashnode.dev/decentralized-ai-inference-validation-building-trust-in-trustless-environments</guid><category><![CDATA[llm]]></category><category><![CDATA[decentralization]]></category><category><![CDATA[Blockchain]]></category><category><![CDATA[Web3]]></category><category><![CDATA[AI]]></category><category><![CDATA[Machine Learning]]></category><category><![CDATA[Security]]></category><dc:creator><![CDATA[Anastasia Matveeva]]></dc:creator><pubDate>Tue, 30 Sep 2025 19:39:54 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1759262169908/327ee91a-0f9a-4014-b98d-49dd528e3f24.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In the <a target="_blank" href="https://what-is-gonka.hashnode.dev/decentralized-ai-inference-balancing-security-and-performance">previous post</a>, we discussed the fundamental tension between security and performance in decentralized systems for LLM inference. In this one, we promised to dive deeper into the details of  inference validation: <strong>how do you actually verify that someone ran the exact model they claimed to run?</strong></p>
<p>One of the key demand drivers for decentralized inference is that no single participant can unilaterally alter the model and everyone is expected to run the exact model with parameters prescribed by the network. Centralized providers can change models or parameters at will, and it is not transparent whether the model was optimized (e.g., quantized) or the output was modified for any other reason. Decentralized networks, on the other hand,  enforce consistency through consensus: any changes to the supported models need to be voted on by the majority of the network. However, each participant acts independently and without proper validation, the chain can’t guarantee model integrity.</p>
<h2 id="heading-why-validation-is-tricky">Why Validation is Tricky</h2>
<p>To understand how validation works, it's helpful to recall what happens inside a transformer during inference. When input tokens are processed, the final layer of the transformer produces logits—raw numerical scores for each token in the vocabulary. These logits are then converted into probabilities using softmax function, creating a probability distribution over all possible next tokens. At each generation step, one token is sampled from this distribution to continue the sequence.</p>
<p>Before diving into potential attack vectors and the particular implementation of inference validation, it's worth understanding why validation can be challenging in the first place. The biggest ambiguity comes from <strong>GPU non-determinism</strong>: identical models with identical inputs can produce slightly different results due to hardware variations and even on the same device due to floating-point precision differences.</p>
<p>GPU non-determinism makes it useless to simply compare output token sequences. Instead, it makes sense to look into the transformer's internal computations. Comparing the output layer, i.e., how close are the probability distributions over the model's vocabulary, is a natural choice. To ensure we're comparing distributions for the same sequence, our validation procedure enforces that the validator reproduces the exact same token sequence that the executor generated, then compares these probability distributions at each generation step. This forms a <strong>validation artifact</strong> or <strong>proof</strong> that can verify model authenticity.</p>
<p>The probabilistic nature of this process adds a delicate balance to our problem. We want to punish participants who consistently cheat, but we need to separate them from honest participants who, by Murphy's law, got unlucky and hit a high number of false positives (honest inferences marked as potentially fraudulent) on a given day. Set the conditions too strict, and honest participants get flagged as fraudsters. Too loose, and real cheaters slip through.</p>
<h2 id="heading-the-economics-of-cheating">The Economics of Cheating</h2>
<h3 id="heading-potential-gain">Potential gain</h3>
<p>Let's start with an obvious attack example: <strong>model substitution</strong>. Imagine the network agrees to deploy Qwen3-32B FP8, a large language model that requires significant compute resources. A rational economic actor might think: "What if I just run Qwen2.5-3B instead and pocket the difference?"</p>
<p>Running a 3B parameter model instead of a 32B model can reduce compute costs by an order of magnitude. If you can fool the validation system, you're essentially getting paid for premium compute while delivering budget results.</p>
<p>More subtle attackers can use <strong>quantization</strong>, claiming to run FP8 precision while actually using INT4 quantization. The performance difference might be less dramatic, but the cost savings are still substantial, and the outputs might be similar enough to slip past naive validation.</p>
<p>On the more sophisticated end, there are <strong>pre-fill attacks</strong> that allow producing proofs for the cheap model's output as if it was generated via the full model expected by the network. Here's how it can work:</p>
<p>For example, the chain agrees to deploy Qwen3-235B with a specified set of parameters.</p>
<p>1. Executor generates sequence using Qwen2.5-3B: <code>[Hello, world, how, are, you]</code></p>
<p>2. Executor computes artifacts using Qwen3-235B for those exact tokens in a single inference pass: <code>[{Hello: 0.9, Hi: 0.05, Hey: 0.05}, ...]</code></p>
<p>3. Executor submits Qwen3-235B probabilities as proof, claiming that inference is from Qwen3-235B</p>
<p>In this case, the probabilities <em>are</em> from the correct model, making them look legitimate, but the actual sequence generation was done cheaper. Since the full model could theoretically generate the same output as the smaller one, the results could appear completely legitimate from a validation perspective.</p>
<h3 id="heading-theres-also-potential-loss">There’s also potential loss</h3>
<p>While the benefits of cheating the system could be quite significant, the losses are as well.</p>
<p>The real problem that a cheating participant is trying to solve is not to pass a single fraudulent inference as an honest one, but to consistently get under the radar so that the discount they get on compute outweighs potential punishment from the network.</p>
<p>In the Gonka network, every participant also serves as a validator and validates a portion of all the inferences run by the network, proportional to their weight. Coins for executed work are split between executors and validators. Rewards for completed inferences are distributed at the end of the epoch, and each participant has a reputation that determines how often their inferences should be validated.</p>
<p>The <strong>reputation metric</strong> is designed so that when each new participant enters the network, their reputation is 0 and all of their inferences are validated (1 time on average). As long as the inferences are validated successfully, the reputation slowly grows, and the validation rate drops to as little as 1% for consistently honest participants. Currently, it takes 30 epochs (~30 days) to build your reputation from 0 to 100. If a participant with perfect reputation changes strategy, is caught on fraudulent inferences, and that is confirmed by consensus, they get validated more and more often. What's more important is that if they're caught cheating, their reputation drops back to 0, and it'll take them another 30 epochs to grow it back.</p>
<p>The fixed ~<strong>24h epoch length</strong> matters because if a participant is caught cheating a statistically significant number of times at any point in the epoch, they'll lose reward for the whole epoch including any of the honest inferences. So instead of saving, say 50% of their compute power, they might actually lose it all, and if what they're saving is closer to 15%, the risk would be losing the reward for 85% of compute that they actually spent.</p>
<p>This creates an economic disincentive as the risk of losing both the reward for the current epoch and reputation (which takes multiple epochs to rebuild) makes cheating much less attractive. <em>The problem we're solving with the validation mechanism isn't to catch every single questionable inference, but to draw a line that ensures we're consistently catching fraudsters with high enough probability while not damaging the reputation of honest participants.</em></p>
<h2 id="heading-validation-implementation">Validation Implementation</h2>
<p>So how do we catch these attacks? The problem can be divided into two parts: verifying that the distribution in the proof is close to the distribution produced by the claimed model and confirming that the output text was actually produced from the submitted artifact.</p>
<h3 id="heading-probability-distribution-comparison">Probability distribution comparison</h3>
<p>When an executor generates an inference output (e.g., <code>[Hello, world, how, are, you]</code>), they record a validation artifact which contains the final output together with the top-\(k\) probabilities for each position in the output sequence (e.g., <code>[{Hello: 0.9, Hi: 0.05, Hey: 0.05}, ...]</code> for the first position, etc.). The validator then forces their model to follow the exact same token sequence and computes the normalized distance between probabilities for each position \(i\):</p>
<p>$$\text{dist}i = \sum{j=1}^k \frac{ |p(t^{inf}_j) - p(t^{val}_j)| }{p(t^{inf}_j) + p(t^{val}_j) + \varepsilon}$$</p><p>where \(p(t^{inf}_j)\) is the probability for the \(j\)-th top token at this position in the inference artifact and \(p(t^{val}_j)\) is the probability for the same token in the validator's distribution.</p>
<p>The final distance metric is averaged sum of per-token distances:</p>
<p>$$\text{dist} = \frac{1}{N} \sum_{i=1}^N \text{dist}_i.$$</p><p>For catching completely different models or even different quantizations of the same model, comparing top-\(k\) probabilities turns out to be sufficient. As shown on the plots below, when we tested Qwen3-32B FP8 against INT4 quantization, we achieved 99% fraud detection with zero false positives. The probability distributions for other models are also different enough that the cheating becomes easily catchable within a single epoch while honest participants remain safe.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1759258731861/c1f739b6-a202-4d22-bb19-5e0cd8cd7ed8.png" alt="Number of successful vs failed validations: Qwen3-32B FP8 (honest) vs INT4 quantization (fraud)" class="image--center mx-auto" /></p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1759258886890/be87fd84-4b64-470e-83b7-c9aec83e8d95.png" alt class="image--center mx-auto" /></p>
<p><em>Number of successful vs failed validations: Qwen3-32B FP8 (honest) vs INT4 quantization (fraud)</em></p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1759258943582/b177b6c7-4aec-4336-84f2-e8a33c6257c3.png" alt class="image--center mx-auto" /></p>
<p><em>Distance between probability distributions as a function of sequence length:<br />Qwen3-32B FP8 (honest) vs INT4 quantization (fraud)</em></p>
<p>The downside of this approach alone is that it doesn't protect against pre-fill attacks. If the probabilities actually match the full model distribution but the sequence was generated via a smaller model, there are several additional approaches to catch this.</p>
<h3 id="heading-perplexity">Perplexity</h3>
<p>The most intuitive way of catching pre-fill attacks is checking perplexity: besides verifying that the artifact was produced  via the claimed model, we can also check that the output text is <em>likely to be</em> to be generated via the artifact (i.e., picked from the submitted distribution):</p>
<p>$$\text{PPL} = \exp \left( -\frac{1}{N} \sum_{i=1}^N \log p(t_i \mid t_{\lt i})\right).$$</p><p>If the perplexity is too high, it suggests the sequence was likely generated by a different model. This works well against attacks using significantly different models. When we tested sequences generated by Qwen2.5-3B but claimed to be from Qwen2.5-7B, the perplexity differences were stark enough to set clear thresholds. The graph below shows a transformation over the perplexity function, fitting it into [0, 1] with the majority of 7B model outputs falling below 0.2 and pre-fill attack outputs (3B model outputs with 7B model probabilities) all falling above 0.4.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1759259049072/38ada5d1-5acc-4f0d-a8dc-93ef94943403.png" alt class="image--center mx-auto" /></p>
<p><em>Transformed perplexity function: Qwen2.5-7B (honest) vs Qwen2.5-3B pre-fill attack (fraud)</em></p>
<p>However, perplexity has limitations compared to similar models and different quantizations of the same model. For example, comparing Qwen2.5-7B Base w8a16 vs w8a8 doesn’t show a clear difference.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1759259069832/e9649f7b-418c-487f-b4ab-3d48c3992059.png" alt class="image--center mx-auto" /></p>
<p><em>Transformed perplexity function: Qwen2.5-7B w8a16 (honest) vs w8a8 quantization (fraud)</em></p>
<p>The good part is that attacks using similar models are often economically pointless. If they're running a model that's 70-90% as expensive as the claimed model, plus doing a scoring pass on the real model, they might actually spend <em>more</em> compute than just honestly  running the real model.</p>
<p>It's worth noting that a single unlikely output generated by an honest participant wouldn't drop their reputation significantly. If the unlikely output is not consistent for the participant, meaning it's just an occasional statistical outlier, they will still receive the full reward at the end of the epoch.</p>
<h3 id="heading-deterministic-approach-binding-output-sequence-to-rng-seed">Deterministic approach: binding output sequence to RNG seed</h3>
<p>If you're looking for a more deterministic solution, another option is to modify the artifact and bind the generated sequence to the RNG seed.</p>
<p>When a participant generates the inference output, they use a deterministic seed derived from the request: <code>run_seed = SHA256(user_seed || inference_id_from_chain)</code>. This seed initializes the random number generator used for token sampling. The validation artifact then includes this seed along with the top-\(k\) probabilities.</p>
<p>During validation, the validator uses the same seed to verify that each token in the sequence was actually sampled from the provided probability distribution. This protects against pre-fill attacks because if a different model generated the sequence, the validator won't be able to reproduce the same output using the claimed model and the given seed, the sampling would follow a different distribution.</p>
<p>This provides a deterministic yes/no answer and is much cheaper than a full forward pass, allowing us to quickly rule out pre-fill attacks. If the sequence does match what the claimed model would generate with that seed, we can then use the distance check to confirm the model was actually the one claimed.</p>
<h2 id="heading-looking-forward">Looking Forward</h2>
<p>We're sharing these thoughts as we continue to improve Gonka network in the belief that the future of AI is decentralized. The need to bind output to particular model parameters will only grow.</p>
<p>The approach the network has been selecting so far proves to be practical and works well enough for the current setup of the system, while its particular components could also be reused to validate inference authenticity in different environments.</p>
<p>As always, we're open to questions, discussion, and improvement suggestions. If you're interested in what we're doing, join <a target="_blank" href="https://gonka.ai/">Gonka</a>, come to our <a target="_blank" href="https://discord.com/invite/RADwCT2U6R">Discord</a> and feel free to drop any questions in the comments here.</p>
]]></content:encoded></item><item><title><![CDATA[Decentralized AI Inference: Balancing Security and Performance]]></title><description><![CDATA[Explore the core challenge of decentralized AI inference. Learn how a trustless, reputation-based network can balance security and performance to compete…]]></description><link>https://what-is-gonka.hashnode.dev/decentralized-ai-inference-balancing-security-and-performance</link><guid isPermaLink="true">https://what-is-gonka.hashnode.dev/decentralized-ai-inference-balancing-security-and-performance</guid><category><![CDATA[AI]]></category><category><![CDATA[Blockchain]]></category><category><![CDATA[Web3]]></category><category><![CDATA[llm]]></category><category><![CDATA[decentralization]]></category><category><![CDATA[Security]]></category><category><![CDATA[Machine Learning]]></category><dc:creator><![CDATA[Autoimago]]></dc:creator><pubDate>Mon, 22 Sep 2025 08:53:02 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1758554072063/28afa00a-0cf4-42b5-af95-a2f3f5c1e373.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Author: <a target="_blank" href="mailto:anastasia.matveeva@productscience.ai">Anastasia Matveeva</a> - Co-creator of Gonka protocol.</p>
<p>When we started buildin<a target="_blank" href="mailto:anastasia.matveeva@productscience.ai">g</a> Gonka, we had this vision: what if anyone could run AI inference and get paid for it? What if we could tap into all the unused compute power out there instead of relying on expensive centralized providers?</p>
<p>The current AI landscape is dominated by a few major cloud providers: AWS, Azure, and Google Cloud, which control most of the world's AI infrastructure. This centralization comes with some serious problems that many of us have experienced firsthand. A few companies controlling AI infrastructure means they can set whatever prices they want, censor applications they don't like, and create single points of failure. When OpenAI's API goes down, thousands of applications break. When AWS has an outage, half the internet stops working.</p>
<p>Even the "efficient" frontier isn't cheap: Anthropic said Claude 3.5 Sonnet cost "a few tens of millions" to train, and while Claude Sonnet 4 is now generally available, Anthropic hasn’t shared its training costs. Their CEO, Dario Amodei, previously projected frontier training runs approaching ~$1 billion (with the next wave headed to multi-billion). Running inference on these models isn't cheap either. A single LLM inference can cost hundreds to thousands of dollars per day for a moderately active application.</p>
<p>In the meantime, there are tons of unused (or used in a meaningless way) compute power sitting around the world. Think about all those Bitcoin miners burning electricity to solve hash puzzles that provide no value, or data centers running at partial capacity. What if that computational power could be used toward something actually valuable, like AI inference?</p>
<p>A decentralized approach could pool compute across participants, lowering capital barriers and reducing single-vendor bottlenecks. Instead of relying on a few big companies, we can create a network where anyone with a GPU could participate and get paid for running AI inference.</p>
<p>We knew that building a working decentralized solution would be complicated. There are a million moving parts, from measuring consensus mechanisms to training protocols to resource allocation. Today I want to focus on just one piece: <strong>running inference of a particular LLM</strong>. How hard can that be?</p>
<h2 id="heading-what-we-mean-by-decentralized">What We Mean by Decentralized</h2>
<p>When we talk about <strong>decentralized</strong> AI inference, we mean something specific. Unlike simply running AI models across multiple servers, we're building a system where anyone can join, contribute compute power, and get rewarded for honest work.</p>
<p>The key requirement is that the system has to be <strong>trustless</strong>. This means you shouldn't have to trust any single person or company to run the system correctly. If you're going to let random people on the internet run your AI models, you need cryptographic guarantees that they're doing what they claim to be doing (at least with a high enough probability).</p>
<p>This trustlessness requirement has some interesting implications. For one thing, it means the system needs to be <strong>verifiable</strong>: you need to be able to prove that the same model with the same parameters was used to generate a given output. This is especially important for smart contracts, which need to be able to verify that they're getting legitimate AI responses.</p>
<p>But there is a challenge: the more verification you add, the slower everything gets, because the network's compute is spent on validation. If you trust everyone completely, you don’t verify inferences and get pretty much the same performance as centralized players. But if you don't trust anybody ever and always verify everything, you slow down your system to the point where it can't compete against centralized solutions.</p>
<p>This is the core tension we've been wrestling with: finding the right balance in the <strong>security-performance</strong> tradeoff.</p>
<h2 id="heading-blockchain-and-inference-validation">Blockchain and Inference Validation</h2>
<p><a target="_blank" href="mailto:anastasia.matveeva@productscience.ai">S</a>o how do you actually verify that someone ran the right model with the right parameters? The blockchain came as an obvious choice—not without its own challenges, but it still is  the most reliable way we know of to create an immutable record of what happened.</p>
<p>The basic idea is pretty straightforward. When someone runs an inference, they need to provide proof that they used the correct model. This proof gets recorded on the blockchain, creating a permanent, tamper-proof record that anyone can verify.</p>
<p>The problem is that blockchain is slow. Really slow. If we tried to record every single step of inference on-chain, that amount of data would paralyze the network too quickly. This is the constraint that drove a lot of decisions while designing Gonka network.</p>
<p>When designing the network and thinking about distributed computations, multiple strategies come to mind. Can you shard the model across multiple nodes, or should the full model sit in one node? The main constraints come from network bandwidth and blockchain speed. To make our solution viable, we opted for a full model fitting into one node, although it may change in the future. This does add minimal requirements to join the network as each node needs enough compute power and memory to run the entire model. That said, one model can still be sharded between multiple GPUs belonging to the same node, which gives us some flexibility within the single-node constraint. We use vLLM, which allows customization of tensor and pipeline parallelism parameters to achieve optimal performance.</p>
<h2 id="heading-how-this-actually-works">How this Actually Works</h2>
<p>So, we agreed that each node hosts a full model and runs a full inference, we avoid the need to coordinate across multiple machines during the actual computation. Blockchain is just used for record keeping. We only record transactions and artifacts for inference validation. The actual computation happens off-chain.</p>
<p>We want the system to be decentralized, with no single point directing inference requests to the network nodes. In practice, each participant deploys two nodes (at least): the network node and one or more inference (ML) nodes across multiple servers. The network node handles communication (consisting of a chain node that connects to the blockchain and an API node that manages user requests), while your ML nodes perform LLM inferences.</p>
<p>When an inference request comes to the network, it arrives at an API node of one of the participants (acts as a Transfer Agent), which randomly selects an Executor (a different participant whose ML node will be executing the inference). To save time and parallelize blockchain records with actual LLM computations, the Transfer Agent (TA) first sends the input request to the Executor and records the input on-chain while the Executor’s ML node is already running inference. Once computation is complete, the Executor sends the output to the TA’s API node while its own chain node records a certain validation artifact on-chain. The TA’s Network node passes the output back to the client and records it on-chain as well. Of course, these records still create overall network bandwidth constraints.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1758574976752/c527ae79-837e-45a4-a258-9aa7442a55f8.png" alt class="image--center mx-auto" /></p>
<p>As you can see here, blockchain records are not slowing down either the moment when inference computation starts, nor the moment when final results are available to the client. The validation that inference was done honestly happens afterwards, in parallel with other inferences. If an Executor is caught cheating, they'll lose rewards for the whole epoch, and the client will be notified and get their money back.</p>
<p>The final questions are: what's in the artifact, and how often would we validate inferences?</p>
<h2 id="heading-security-vs-performance-tradeoff">Security vs. Performance Tradeoff</h2>
<p>The fundamental challen<a target="_blank" href="mailto:anastasia.matveeva@productscience.ai">g</a>e is that security and performance are at odds with each other.</p>
<p>If you want maximum security, you verify everything. But that's slow and expensive. If you want maximum performance, you trust everyone. But that's risky and opens you up to all kinds of attacks.</p>
<p>After some trial, error<a target="_blank" href="mailto:anastasia.matveeva@productscience.ai">,</a> and parameter picking, we settled on an approach that tries to balance both concerns. We had to carefully tune how much verification to do, when to do it, and how to make the verification process as efficient as possible. Too much verification, and the system becomes unusable. Too little, and it becomes insecure.</p>
<p>Keeping the system lightweight was essential. We had to keep the artifact fairly lightweight by storing top-k next token probabilities. We use them to measure the likelihood that a certain output was indeed generated by the claimed model with the claimed parameters, and to catch any alterations, such as using of smaller models or quantizations, with high enough confidence. We'll focus more on this in a separate article about the particular implementation of the inference validation procedure.</p>
<p>In the meantime, how to decide which inferences to validate and which not? We opted for a reputation-based approach. When a new participant joins the network, they have 0 reputation, and 100% of their inferences get validated by at least one participant. If concerns are raised, eventually, consensus mechanism would be used to either clear your inference or to drop your reputation and potentially kick you out of the network. As your reputation grows, fewer of your inferences would be validated, going to as little as a randomly selected 1% of all your inferences. This dynamic approach allows us to keep the overall percentage of validations low while efficiently catching those participants who try to cheat the network.</p>
<p>Participants get paid at the end of each epoch proportionally to their weight in the network. Tasks are distributed proportionally to weight as well, so payments are expected to be proportional both to the weight and the amount of work done. This means we don't need to catch and punish fraudsters immediately, it is sufficient to catch them within the current epoch before payouts occur.</p>
<p>Economic incentives drive this tradeoff as much as technical parameters do. By making it expensive to cheat but profitable to be honest, we can create a system where the rational choice is to participate honestly.</p>
<h2 id="heading-room-for-optimization">Room for Optimization</h2>
<p>After months of buildin<a target="_blank" href="mailto:anastasia.matveeva@productscience.ai">g</a> and testing, we've built a system that combines the benefits of blockchain for record-keeping and security, and performance close to centralized for each particular inference. The fundamental tension between security and performance is real, and there's no perfect solution, just different tradeoffs.</p>
<p>We believe that as the network scales, it has a real chance to compete with centralized players while maintaining fully decentralized community control. As it grows, there's plenty of room for further optimization. If you're interested in following along with this process, check out our GitHub and documentation, join our Discord community to participate in discussions, and join the network itself.</p>
]]></content:encoded></item></channel></rss>