Skip to main content

Decentralized AI Inference Validation: Building Trust in Trustless Environments

Updated 10 min readMarkdown
Decentralized AI Inference Validation: Building Trust in Trustless Environments

In the previous post, we discussed the fundamental tension between security and performance in decentralized systems for LLM inference. In this one, we promised to dive deeper into the details of inference validation: how do you actually verify that someone ran the exact model they claimed to run?

One of the key demand drivers for decentralized inference is that no single participant can unilaterally alter the model and everyone is expected to run the exact model with parameters prescribed by the network. Centralized providers can change models or parameters at will, and it is not transparent whether the model was optimized (e.g., quantized) or the output was modified for any other reason. Decentralized networks, on the other hand, enforce consistency through consensus: any changes to the supported models need to be voted on by the majority of the network. However, each participant acts independently and without proper validation, the chain can’t guarantee model integrity.

Why Validation is Tricky

To understand how validation works, it's helpful to recall what happens inside a transformer during inference. When input tokens are processed, the final layer of the transformer produces logits—raw numerical scores for each token in the vocabulary. These logits are then converted into probabilities using softmax function, creating a probability distribution over all possible next tokens. At each generation step, one token is sampled from this distribution to continue the sequence.

Before diving into potential attack vectors and the particular implementation of inference validation, it's worth understanding why validation can be challenging in the first place. The biggest ambiguity comes from GPU non-determinism: identical models with identical inputs can produce slightly different results due to hardware variations and even on the same device due to floating-point precision differences.

GPU non-determinism makes it useless to simply compare output token sequences. Instead, it makes sense to look into the transformer's internal computations. Comparing the output layer, i.e., how close are the probability distributions over the model's vocabulary, is a natural choice. To ensure we're comparing distributions for the same sequence, our validation procedure enforces that the validator reproduces the exact same token sequence that the executor generated, then compares these probability distributions at each generation step. This forms a validation artifact or proof that can verify model authenticity.

The probabilistic nature of this process adds a delicate balance to our problem. We want to punish participants who consistently cheat, but we need to separate them from honest participants who, by Murphy's law, got unlucky and hit a high number of false positives (honest inferences marked as potentially fraudulent) on a given day. Set the conditions too strict, and honest participants get flagged as fraudsters. Too loose, and real cheaters slip through.

The Economics of Cheating

Potential gain

Let's start with an obvious attack example: model substitution. Imagine the network agrees to deploy Qwen3-32B FP8, a large language model that requires significant compute resources. A rational economic actor might think: "What if I just run Qwen2.5-3B instead and pocket the difference?"

Running a 3B parameter model instead of a 32B model can reduce compute costs by an order of magnitude. If you can fool the validation system, you're essentially getting paid for premium compute while delivering budget results.

More subtle attackers can use quantization, claiming to run FP8 precision while actually using INT4 quantization. The performance difference might be less dramatic, but the cost savings are still substantial, and the outputs might be similar enough to slip past naive validation.

On the more sophisticated end, there are pre-fill attacks that allow producing proofs for the cheap model's output as if it was generated via the full model expected by the network. Here's how it can work:

For example, the chain agrees to deploy Qwen3-235B with a specified set of parameters.

1. Executor generates sequence using Qwen2.5-3B: [Hello, world, how, are, you]

2. Executor computes artifacts using Qwen3-235B for those exact tokens in a single inference pass: [{Hello: 0.9, Hi: 0.05, Hey: 0.05}, ...]

3. Executor submits Qwen3-235B probabilities as proof, claiming that inference is from Qwen3-235B

In this case, the probabilities are from the correct model, making them look legitimate, but the actual sequence generation was done cheaper. Since the full model could theoretically generate the same output as the smaller one, the results could appear completely legitimate from a validation perspective.

There’s also potential loss

While the benefits of cheating the system could be quite significant, the losses are as well.

The real problem that a cheating participant is trying to solve is not to pass a single fraudulent inference as an honest one, but to consistently get under the radar so that the discount they get on compute outweighs potential punishment from the network.

In the Gonka network, every participant also serves as a validator and validates a portion of all the inferences run by the network, proportional to their weight. Coins for executed work are split between executors and validators. Rewards for completed inferences are distributed at the end of the epoch, and each participant has a reputation that determines how often their inferences should be validated.

The reputation metric is designed so that when each new participant enters the network, their reputation is 0 and all of their inferences are validated (1 time on average). As long as the inferences are validated successfully, the reputation slowly grows, and the validation rate drops to as little as 1% for consistently honest participants. Currently, it takes 30 epochs (~30 days) to build your reputation from 0 to 100. If a participant with perfect reputation changes strategy, is caught on fraudulent inferences, and that is confirmed by consensus, they get validated more and more often. What's more important is that if they're caught cheating, their reputation drops back to 0, and it'll take them another 30 epochs to grow it back.

The fixed ~24h epoch length matters because if a participant is caught cheating a statistically significant number of times at any point in the epoch, they'll lose reward for the whole epoch including any of the honest inferences. So instead of saving, say 50% of their compute power, they might actually lose it all, and if what they're saving is closer to 15%, the risk would be losing the reward for 85% of compute that they actually spent.

This creates an economic disincentive as the risk of losing both the reward for the current epoch and reputation (which takes multiple epochs to rebuild) makes cheating much less attractive. The problem we're solving with the validation mechanism isn't to catch every single questionable inference, but to draw a line that ensures we're consistently catching fraudsters with high enough probability while not damaging the reputation of honest participants.

Validation Implementation

So how do we catch these attacks? The problem can be divided into two parts: verifying that the distribution in the proof is close to the distribution produced by the claimed model and confirming that the output text was actually produced from the submitted artifact.

Probability distribution comparison

When an executor generates an inference output (e.g., [Hello, world, how, are, you]), they record a validation artifact which contains the final output together with the top-$k$ probabilities for each position in the output sequence (e.g., [{Hello: 0.9, Hi: 0.05, Hey: 0.05}, ...] for the first position, etc.). The validator then forces their model to follow the exact same token sequence and computes the normalized distance between probabilities for each position $i$:

$$\text{dist}i = \sum{j=1}^k \frac{ |p(t^{inf}_j) - p(t^{val}_j)| }{p(t^{inf}_j) + p(t^{val}_j) + \varepsilon}$$

where \(p(t^{inf}_j)\) is the probability for the $j$-th top token at this position in the inference artifact and \(p(t^{val}_j)\) is the probability for the same token in the validator's distribution.

The final distance metric is averaged sum of per-token distances:

$$\text{dist} = \frac{1}{N} \sum_{i=1}^N \text{dist}_i.$$

For catching completely different models or even different quantizations of the same model, comparing top-$k$ probabilities turns out to be sufficient. As shown on the plots below, when we tested Qwen3-32B FP8 against INT4 quantization, we achieved 99% fraud detection with zero false positives. The probability distributions for other models are also different enough that the cheating becomes easily catchable within a single epoch while honest participants remain safe.

Number of successful vs failed validations: Qwen3-32B FP8 (honest) vs INT4 quantization (fraud)

Number of successful vs failed validations: Qwen3-32B FP8 (honest) vs INT4 quantization (fraud)

Distance between probability distributions as a function of sequence length:
Qwen3-32B FP8 (honest) vs INT4 quantization (fraud)

The downside of this approach alone is that it doesn't protect against pre-fill attacks. If the probabilities actually match the full model distribution but the sequence was generated via a smaller model, there are several additional approaches to catch this.

Perplexity

The most intuitive way of catching pre-fill attacks is checking perplexity: besides verifying that the artifact was produced via the claimed model, we can also check that the output text is likely to be to be generated via the artifact (i.e., picked from the submitted distribution):

$$\text{PPL} = \exp \left( -\frac{1}{N} \sum_{i=1}^N \log p(t_i \mid t_{\lt i})\right).$$

If the perplexity is too high, it suggests the sequence was likely generated by a different model. This works well against attacks using significantly different models. When we tested sequences generated by Qwen2.5-3B but claimed to be from Qwen2.5-7B, the perplexity differences were stark enough to set clear thresholds. The graph below shows a transformation over the perplexity function, fitting it into [0, 1] with the majority of 7B model outputs falling below 0.2 and pre-fill attack outputs (3B model outputs with 7B model probabilities) all falling above 0.4.

Transformed perplexity function: Qwen2.5-7B (honest) vs Qwen2.5-3B pre-fill attack (fraud)

However, perplexity has limitations compared to similar models and different quantizations of the same model. For example, comparing Qwen2.5-7B Base w8a16 vs w8a8 doesn’t show a clear difference.

Transformed perplexity function: Qwen2.5-7B w8a16 (honest) vs w8a8 quantization (fraud)

The good part is that attacks using similar models are often economically pointless. If they're running a model that's 70-90% as expensive as the claimed model, plus doing a scoring pass on the real model, they might actually spend more compute than just honestly running the real model.

It's worth noting that a single unlikely output generated by an honest participant wouldn't drop their reputation significantly. If the unlikely output is not consistent for the participant, meaning it's just an occasional statistical outlier, they will still receive the full reward at the end of the epoch.

Deterministic approach: binding output sequence to RNG seed

If you're looking for a more deterministic solution, another option is to modify the artifact and bind the generated sequence to the RNG seed.

When a participant generates the inference output, they use a deterministic seed derived from the request: run_seed = SHA256(user_seed || inference_id_from_chain). This seed initializes the random number generator used for token sampling. The validation artifact then includes this seed along with the top-$k$ probabilities.

During validation, the validator uses the same seed to verify that each token in the sequence was actually sampled from the provided probability distribution. This protects against pre-fill attacks because if a different model generated the sequence, the validator won't be able to reproduce the same output using the claimed model and the given seed, the sampling would follow a different distribution.

This provides a deterministic yes/no answer and is much cheaper than a full forward pass, allowing us to quickly rule out pre-fill attacks. If the sequence does match what the claimed model would generate with that seed, we can then use the distance check to confirm the model was actually the one claimed.

Looking Forward

We're sharing these thoughts as we continue to improve Gonka network in the belief that the future of AI is decentralized. The need to bind output to particular model parameters will only grow.

The approach the network has been selecting so far proves to be practical and works well enough for the current setup of the system, while its particular components could also be reused to validate inference authenticity in different environments.

As always, we're open to questions, discussion, and improvement suggestions. If you're interested in what we're doing, join Gonka, come to our Discord and feel free to drop any questions in the comments here.

No comments yet. Be the first to comment.