The Sharpe ratio problem nobody mentions

My coursework teaches the Sharpe ratio as the standard way to judge a strategy: mean excess return divided by volatility. It's a fine number for one backtest. It says nothing about how many backtests you ran to find it.

I test a lot of ideas, more than 20 pre-registered hypotheses since May 2026: sizing rules, cost models, macro overlays. If I just report the best Sharpe out of that pile, I've quietly run a multiple-hypothesis test and hidden it. The more variants you try, the more likely one of them looks good by chance alone. A plain Sharpe ratio can't tell "genuinely good" from "got lucky across 20 tries."

So the number that actually gates whether a strategy is allowed near real capital isn't the raw Sharpe. It's the deflated Sharpe ratio (Bailey & López de Prado): it discounts the observed Sharpe for how many variants I tried, how correlated those trials were, and how far the return distribution sits from the clean, symmetric shape Sharpe assumes. I pair it with bootstrap resampling to get a confidence interval around that number, instead of trusting one point estimate off one sample path.

For that confidence interval, I resample whole decision days, not individual trades, and read a 95% band. Trades placed on the same day share the same macro shocks, so treating them as independent draws would understate the true uncertainty and manufacture significance that isn't really there. When a practitioner mentioned in conversation that their desk uses Newey-West (HAC) standard errors instead, I ran the comparison myself. HAC actually came out more favorable on my data. I kept the block-bootstrap approach anyway: switching methods only when one happens to flatter the result defeats the point of having one at all.

It's a less flattering number than the one my lecture slides would compute on the same data. That's the point: it's supposed to survive me trying to fool myself.

Killing my own good idea

I came across an idea in outside research: a simple weighted scorecard, 0 to 100, for deciding whether a new strategy is worth greenlighting. It looked clean and appealingly objective, so before adopting it I ran it through the same adversarial review I use on my actual trading ideas.

It failed the first round. The scorecard quietly dropped any correction for multiple testing: score enough candidates and something clears a high bar by chance alone, the exact mistake the deflated Sharpe ratio exists to catch. Worse, its admission criteria matched the profile of two ideas I'd already tested and found to be false positives; the scorecard would have waved both of them through.

One piece of it survived a second look and made it into my process. The rest didn't. I only found out because I was willing to attack my own preferred idea as hard as I'd attack anyone else's.

The easiest strategy to approve is always the one you didn't try to break.

My AI research assistant

People assume that when I say "AI-assisted trading system," the AI is the one placing trades. It isn't. In my setup, a language model never sizes a position, never decides whether to send an order, and never picks a strategy. Those decisions are mine: made and signed off before a trade ever happens, not approved in the moment.

What I actually use it for is the grunt work I'd otherwise do myself every morning: reading through central-bank statements and data releases, summarizing what changed versus expectations, pulling up historical analogues, and pushing back when my own reasoning looks thin. Some mornings it disagrees with me, which is usually more useful than if it just agreed.

It does flag risk as part of that review: concentration, competing macro drivers, event risk, things worth watching. What it doesn't have is authority: nothing trades that I haven't pre-registered and signed off as a strategy, every order passes hard risk caps I set in advance, and the model has no path to change any of that. It can tell me something looks risky. It can't act on it.

That's the whole design: I get through far more research than I could alone, and I still make every call that matters.

Why point-in-time macro data matters

Most macro data gets revised after it's first released (GDP, inflation, the numbers everyone quotes). If you backtest against today's final, revised figures, your model is quietly using information that didn't exist yet on the day it was supposedly deciding. That's look-ahead bias, and it makes results look better than they'd have actually been.

So I store point-in-time data instead: what was actually known on each date, not what we know now with hindsight. I keep FRED's vintage releases and rebuild the information set as of each decision date. It's more work, and honestly my backtests got worse the day I switched to it. But that's the number I trust. The other one was measuring how good I am at using information from the future.