I tested 11 AI models to build a $25,000 options trading strategy. The results shocked me.

The Experiment

I swear to God, I have deja vu.

If you've been following my work, you've heard this before.

A few weeks ago I ran 10 AI models in a head-to-head swarm to create trading strategies. The cheap models won every time. Opus 4.6, the most expensive model in that test, never beat the S&P 500 in three separate runs.

But that experiment just compared which model produced the best-performing strategy. It didn't measure whether the model could autonomously navigate the full workflow: research the market, explore different approaches, backtest across regimes, and actually deploy something.

This time, the grading is goal-oriented. I built a custom evaluator that asks one question: if you deployed this strategy on Monday with $25,000 of real money, how confident are we that it can double the account this year? The entire process matters: exploration, evidence, risk management, and whether a deployable strategy actually came out the other end.

I used 11 models. Grok 4.20, Sonnet 4.6, and GPT-5.4. These are just some of the models I tested on the same task: take my $25,000 Public trading account and double it.

These aren't toy models. Grok 4.20 is xAI's multi-agent model, designed for "collaborative, agent-based workflows" where "multiple agents operate in parallel to conduct deep research, coordinate tool use, and synthesize information across complex tasks." GPT-5.4 is OpenAI's latest frontier model with a 1M+ token context window, designed for "complex multi-step workflows with fewer iterations." Sonnet 4.6 is Anthropic's most capable Sonnet-class model, with "frontier performance across coding, agents, and professional work."

The scores tell the whole story. A model that was designed for multi-agent workflows scored 4. The most expensive models in the test collectively produced nothing I could deploy. Meanwhile, Gemini Flash scored 66, spawned 3 research agents on its own, and deployed the only strategy in the entire experiment with a positive return across every regime tested.

I thought maybe options trading would be different. More complex. More reasoning required. Surely this is where the expensive models earn their price tag.

They were even worse.

The Setup

$25,000 of real money. One prompt. Eleven models.

Here's the context. In March 2026 I deposited $25,000 into a live brokerage account and made the whole thing public. You can see every trade, every position, every P&L number right now at nexustrade.io/shared-portfolio. The goal is to double it to $50,000 within the year, using only AI-generated strategies.

The $25,000 Public Portfolio Challenge. Every strategy, every trade, tracked publicly.

I gave every model the exact same prompt:

"Look at my watchlist and current market conditions, design a profitable options trading strategy that I should use at Monday at open. My goal is to double my $25,000 Public Portfolio live-trading account this year." The prompt, word for word

Same watchlist. Same live market data. Same account. I swapped out the underlying model and ran the agent 11 times. Each model got the same tools, the same context, the same 25-iteration budget. The only variable was the brain.

And the results were damning.

Why This Matters

This is not a chatbot test

I need to explain why this experiment is different from the usual "I asked ChatGPT to pick stocks" content you see everywhere.

Most AI trading tools are dressed-up chatbots. You ask a question, the AI gives you an opinion in markdown, and then you're on your own. There's no data. No backtesting. No accountability. The AI talks about trading. It doesn't actually trade.

NexusTrade's agent is different. It runs a loop (think, act, observe, repeat) against real tools:

The model doesn't just think. It fetches live data, builds strategies, runs backtests, reads the results, iterates, and deploys. A single run can span 25 iterations and take 45 minutes. Every tool call is logged. Every reasoning step is visible. The full conversation is public on NexusTrade. You can audit every single decision the agent made.

Left: what a chatbot does. Right: what an autonomous agent does. Every model in this test had access to the same tools on the right side. Most of them still acted like the left side.

The critical feature is subagent spawning. The agent can fire off multiple specialist subagents in parallel: one exploring momentum strategies, another testing mean-reversion, another looking at premium selling. The orchestrator reads all their results and picks the best candidate. This is how you explore a wide search space efficiently.

And it's exactly what most models refused to do.

A Real Run

What it looks like when a model actually does its job

Here's the actual sequence from the winning model. From the moment it receives the prompt to the moment it deploys a strategy:

Now compare that to GPT-5.4.

The insight was there. The execution wasn't. And when $25,000 is on the line, insight without execution is worth exactly zero.

The Scoring

How I graded each run

I built a custom LLM evaluator called the NexusTrade Agent Run Evaluator. It answers one question:

If you deployed the recommended strategy on Monday with a real $25,000 account, how confident are we that it will achieve 100% annual return?

The scoring is goal-oriented by design. Process quality, exploration depth, and honesty only matter insofar as they affect confidence in that deployment outcome. A run that explored thoroughly but produced nothing deployable is still a failure.

The rubric weights four dimensions:

Every run produces a structured evaluation. Here's what the evaluator returns:

Actual evaluator output for the Gemini Flash run. The scores color-match the rubric bars above.

This is why even Gemini Flash, the winner, only scored 66. Its deployed strategy averaged +15.66% annually. That's positive across all regimes, which is impressive. But 15.66% hard-capped its fitness at 3/10. To double a $25k account, I need something closer to 100%. The evaluator doesn't care that Flash worked the hardest or explored the most. It cares whether the result can hit the goal.

A score of 90–100 means "deploy Monday." A score of 66 means "promising signal, iterate before going live." A score of 0–10 means the model failed at the most basic level of the task.

The Results

The full scorecard. It's ugly.

Look at this table. Really look at it.

Look at this table. Google's models took first and second. OpenAI's flagship GPT-5.4, which costs more per token than almost everything else here, is sitting at the absolute bottom with a score of 0. Its cheaper sibling GPT-5.4-mini scored 1. Five out of eleven models completely failed the task. They didn't produce a single portfolio. They didn't run a single backtest. They just… talked about trading.

And the single most important differentiator wasn't intelligence. It wasn't reasoning ability. It wasn't benchmark scores. It was whether the model had the instinct to delegate work to subagents instead of trying to do everything itself.

Every Model, Ranked

From worst to first

Ordered by score ascending. Buckle up.

The middle tier: scores 47–50

Four models landed in the 47–50 range. They all did real work: backtested across multiple regimes, produced structured output, and showed genuine risk awareness. None found a strategy worth deploying with confidence. Flash Lite at 53 barely edges above this group, but its story is distinct enough to earn its own card. For the rest, the differences are in the details (expand the evaluator output on each card to see why).

The Winner

And the best model was...

Flash (66) tops the field. Flash Lite (53) edges out Claude and Gemini Pro (both 50) for second.

The Lesson

I will never overpay for AI models again

I've now run this experiment across multiple domains. Agent swarms for equity strategies. Options trading bakeoffs. SQL query generation benchmarks. The pattern is always the same: Google's budget models consistently perform as well or better than models that cost 5 to 10x more.

At some point, this stops being a coincidence and starts being a rule. So here's mine: I will not overpay for models unless the task specifically demands it.

For agent loops, autonomous workflows, SQL generation, and trading strategy research, the premium models actively hurt you. They're trained to be helpful assistants. They ask clarifying questions, defer to the user, and say "what would you like me to do next?" That's great for pair-programming. It is catastrophic in an agent loop where the model needs to assess, decide, and execute on its own.

Flash didn't ask permission. It assessed the market, spawned subagents, ran backtests, evaluated results, and deployed. GPT-5.4 did brilliant analysis and then handed me the work.

But here's what makes this more than a cost optimization story. Look at Gemma 4 31B. It scored 50, tying with Claude Sonnet 4.6. Gemma costs 0.25 tokens per call. Sonnet costs 10. That's a 40x price difference for the same score. And Gemma is MIT-licensed and open-weight.

Cursor already proved the playbook. They took Kimi K2.5, fine-tuned it within a custom harness, and built Composer 2. It competes with Claude and GPT on coding tasks at a fraction of the cost. Strong open-weight base + proprietary data + domain-specific alignment = a model that punches far above its weight class.

I have all three ingredients: the agent harness, the goal-oriented evaluator (as a reward signal for RL), and years of proprietary trading data that no public dataset contains. The base models are already competitive. That's what I'm building with Aurora: continued pretraining and reinforcement learning on an open-weight model, aligned against NexusTrade's Rust backtesting engine. Every agent run in this bakeoff is training data for that model.

The expensive models didn't just lose this experiment. They made the case for why they should be replaced entirely.

Run It Yourself

You can replicate this entire experiment

The agent that ran all eleven of these tests is live at nexustrade.io. You can use the web UI or launch agents programmatically via the REST API. Here's the web UI approach:

"Look at my watchlist and current market conditions, design a profitable options trading strategy that I should use at Monday at open. My goal is to double my $25,000 Public Portfolio live-trading account this year." The exact prompt used for all 11 runs

Or, if you prefer to work programmatically, you can launch agents via the REST API and poll for results. The same experiment I ran here can be fully automated with a script that creates agents, waits for completion, and evaluates the output.

The $25,000 challenge portfolio is public at nexustrade.io/shared-portfolio. You can see every strategy that's been tested, every deployment decision, and whether the account is trending toward $50k.

The Public.com brokerage account. $25,000 of real capital. This is where strategies go after they pass backtesting.

The platform is free if you sign up here. Live trading requires connecting your own brokerage. Options trading involves real risk. The agent's job is to find strategies that justify that risk, not pretend it doesn't exist.

Part Two

What happens when you optimize the winner

The bakeoff picked a winner, but a single score doesn't finish the job. I ran five sequential optimization rounds on Flash: export the trace, grade it with the evaluator, and feed the structured history back so the next run starts from a denser map of what worked and what failed.

The loop didn't behave like textbook optimization. Round 1 scored highest; later rounds drifted. The full methodology, the $676 spend breakdown, and what I'm changing next are in the companion piece:

The optimization experiment →

All strategies referenced are simulated backtest results. Past performance is not indicative of future results. Backtests do not account for slippage, commissions, or liquidity constraints. This article is not financial advice. Options trading involves substantial risk of loss and is not suitable for all investors.