Backtesting & Validation

Neuron validates strategies by running the code that trades live against history it records itself. There are two backtesters, one per kind of strategy, and a validation gate on top that decides which proposed rule changes are good enough to reach the Neuron team.

Token-filter backtests#

What is recorded

Every 5 minutes, the indexer writes a snapshot of every parameter for every token with at least 3 trades in the last hour (only the long description text is left out). Snapshots are kept for 5 days. This is point-in-time data: a backtest sees what an agent would have seen at that moment, not today's values.

How a replay runs

1. 1For each snapshot, find the tokens passing the filters, ranked like the live query, keeping the top candidates (8 or twice "hold while in top", whichever is larger, plus one).
2. 2Drop candidates whose round trip at the position's size would cost more than the max round-trip cost.
3. 3Mark the held token at its snapshot price. If it has no snapshot (fewer than 3 trades that hour), its price is treated as unchanged.
4. 4Apply take profit, stop loss and rotation exactly as the live agent does, and sell at the modeled fill price.
5. 5With no position, buy the best remaining pick at the modeled fill price.
6. 6Book the step's return into the vault, capped at 1,000 bps per step.

The cost model

Both live trading and backtests price fills with the same function (src/lib/agents/costs.ts):

• Fee and tax: the venue fee (the Pons curve fee, or the V4 pool fee; 100 bps when unknown) plus the token's creator tax, paid on the way in and on the way out.
• Price impact: the trade pushes the price by its size divided by the curve's or pool's quote-side depth. When depth is unknown, the model assumes about 5% impact.
• Gas: live Robinhood Chain gas for a 180,000-gas swap in live trading; a fixed $0.002 per trade in backtests.

What you get back

The backtest reports the vault return, maximum drawdown, the share of time in a position, every trade with its reason (take profit, stop loss, rotate or still open), an equity curve of about 200 points, and the snapshot coverage. It also measures signal quality: every 30 minutes, the average next-hour return after a full round trip's costs for tokens passing your filters versus all tokens. If passing tokens do no better than everything else, the filter is not adding information.

Requests can cover 0.25 to 5 days (3 by default) for a simulated vault of $100 to $100,000 ($1,000 by default), with at most two backtests running at once.

Engine backtests#

The engine replay feeds recorded history to the live engine code one hour at a time. Nothing after the replay clock is visible: prices come from the last hourly candle that had closed, and pool APYs from the latest daily DefiLlama point, ignored if more than 3 days old.

The history download covers the 14 coins the spot engines trade plus the 30 most traded perps today, and every lending and LP pool the engines could pick at their loosest settings. It is refreshed when it is a day old and stored so restarts do not wait for it.

Validating fixes from loss reviews#

House agents review their own losses. A loss is a token trade closed in the red, a perp position closed at a loss, or, for strategies that do not close trades (yield, LP, carry, DCA, basket), a UTC day worse than -0.1%. Each agent is reviewed at most once every 6 hours, with losses in between read together. Claude reads the losses, the rules and, for a token trade, the token's parameters at the moment it was bought, explains what happened, and proposes up to three fixes. Each fix goes through this gate:

1. 1Valid. It must parse within every bound, keep the same engine, and actually change something.
2. 2Backtested against the current rules on the same data. Token-filter rules are scored over four 1-day windows of snapshots; engines over four 30-day windows (120 days). Both use a $750 simulated vault.
3. 3Profitable. Total return in the backtest must be positive.
4. 4Still trading. For token-filter rules, it must close at least 3 trades, or 30% of what the current rules close, whichever is more.
5. 5Clearly better, not just riskier. It must beat the current rules in at least 3 of the 4 windows, by at least 1 percentage point in total (0.5 for engines), with drawdown no more than the current drawdown plus the larger of 1 point or 20% of it, and at least as much return per unit of drawdown.

If no fix passes, the backtest results go back to Claude once for a second round. The best fix that passes goes to the Neuron team with the analysis and both backtests; a person approves or rejects it. Unanswered proposals expire after 24 hours, because the backtest that justified them goes stale. An approved change takes effect on the worker's next refresh, about a minute later, and appears in the agent's "How it improves" panel on its profile.

Shadow validation#

New decision layers can be tested in shadow before they ever act. In shadow mode (off by default), a candidate policy such as the wallet-evidence entry gate runs with exactly the same inputs as the live agent each tick, and its would-be decision is logged next to the real one. It never trades. An offline scorer then measures, for every entry it would have vetoed, whether the token went on to lose (a good veto) or gain (a missed trade).

Limits of these backtests#

• 5-minute resolution. A token can spike and crash between snapshots; take profit and stop loss are only checked at the next one.
• Step cap without carry. A backtest step is capped at 1,000 bps and the excess is dropped, while the live agent carries it into later reports. Very large moves are understated in both directions.
• Thin-hour pricing. A held token without a snapshot keeps its last price.
• Survivorship in the perp universe. Funding carry is tested on today's most traded markets; markets that faded are missing.
• Constant gas. Engine replays hold each chain's gas price at its download-time value, and bridging costs a flat 5 bps.
• Live-only tools. The AI pre-trade check and news sentiment cannot be backtested. Data tools (smart money, DexScreener, X, GoPlus) only have history from when they were added.
• Model simplifications. LP positions are modeled as full-range, perps without liquidation, carry without basis risk.

Related#

• Risk Management
• Prebuilt Strategies
• Blockchain Indexer
• Public PnL & Trade History