LLMs as research reviewers, not traders
June 2026 · AI in research
There's a lot of noise about AI "making trades." I think that's the wrong job for it. In my own systems, the language model never sizes a position, never executes, and never promotes a strategy. It reviews.
What it's genuinely good at is the analyst's grunt work: reading a wall of central-bank language and data releases, summarizing what changed versus expectations, surfacing historical analogues, and flagging where my own reasoning is thin. It's a tireless research assistant that argues with me — not a decision-maker I outsource to.
Keeping the model on the research side of the line is deliberate. Execution authority lives in a separate, gated path with hard risk controls; the LLM can inform that path but has no power over it. That separation is the point: I get the speed and breadth of AI over messy, unstructured macro information, while judgment, sizing, and risk stay where they belong.
Used this way, AI widens what one person can monitor — without widening what can go wrong.
Fail-closed design in financial systems
June 2026 · systems design
In most software, when something breaks the default is to keep going and log the error. In a system that can move money, that instinct is backwards — the safe default is to stop.
I build my research and execution tooling to fail closed: if market data is stale, the account identity doesn't match, the model's context is incomplete, or reconciliation doesn't tie out, the system blocks new risk rather than guessing. Doing nothing is a valid — often the correct — action.
This inverts a lot of normal engineering habits. It means writing code that distrusts its own inputs, treats every upstream feed as guilty until proven current, and treats "I'm not sure" as a hard stop, not a warning. The hard part isn't the happy path; it's enumerating the failure modes and deciding, in advance, what "safe" means for each.
The payoff is a bounded worst case. A system that fails closed can miss an opportunity — but it's far harder for it to be catastrophically wrong about risk.
A research result is only as good as its negative controls
June 2026 · methodology
A backtest that looks good tells you almost nothing on its own. The question that matters is: how hard did you try to make it fail?
Most of my research time goes into negative controls, not positive results. Before I believe a pattern, I want it to survive walk-forward validation — choose parameters on the past, test on the untouched next slice — plus realistic transaction-cost stress and no-lookahead checks that confirm every decision used only what was knowable at the time. I also run controls designed to produce nothing: random-direction and opposite-signal versions. If those "work," my test is broken — I haven't found anything.
The discipline I value most is killing ideas cleanly. It's tempting to tweak until a result appears, but that's just overfitting with extra steps. A result I trust is one that kept working after I stopped helping it.
Most ideas don't survive that process. That's the system working, not failing.
Why point-in-time macro data matters
June 2026 · methodology
Most macro series are quietly revised after their first release. Backtest a strategy on today's final GDP or inflation prints and you let the model "see" numbers that didn't exist on the decision date — a subtle look-ahead bias that flatters the results.
The fix is point-in-time (vintage) data: store what was actually known as of each date, and reconstruct that as-of view for every decision. In my own research stack I keep FRED macro vintages and rebuild the information set per decision date. It's less convenient, and the backtests look worse — which is exactly the point. A strategy that only works on revised data isn't a strategy; it's hindsight.