COUNTERPHASE / RESEARCH VALIDATION
CONSTRUCTED EXAMPLE  ·  NO REAL CLIENT DATA
A WORKING RESEARCH CHALLENGE BUILT FOR QUANTITATIVE TEAMS

Your backtest looks convincing.
What did it overlook?

Counterphase audits how research models were tested—what they knew at the time, which outcomes went missing, and whether their decisions relied on data they were never meant to use.

Try this small, constructed experiment: a model trains on four outcomes. Remove one of the sixteen it didn't choose. Should that change the model's decision?

Discuss a private pilot
01 / CHOOSE WHICH MODEL TO CHALLENGE
20 ORIGINAL T0 SLOTS
2 BLOCKS · 2 SELECTED PER BLOCK
02 / TAKE AWAY DATA THE MODEL DIDN'T CHOOSE SELECTED    UNSELECTED    HIDDEN
Each outlined square is an outcome the model did not choose. Click one to hide it.
ORIGINAL OPPORTUNITIES20 / 20unchanged in every challenge
OUTCOMES ACTUALLY CHOSEN4 / 20two per policy block
UNUSED OUTCOMES HIDDEN0 / 16counterfactual intervention
03 / OBSERVE THE DECISION
Hide an unchosen outcome.
Nothing about the four selected outcomes changes. Will the research decision stay the same?
BASELINEMODEL CAN LEARNAFTER INTERVENTIONMODEL CAN LEARNDECISION CONTRADICTIONNONE
NOT JUST A VISUAL TRICKINSPECT THE ACTUAL TEST OUTPUT

If a model changes its decision when we remove data it claims not to use, its research process has a problem worth investigating.

The screen above teaches the idea. The reports below come from a separate, locally tested Python audit—not a live fund or a return backtest.
↯SYNTHETIC FAILING CASESee what broke.17 witnessed counterfactual changes per policy arm in the local demonstration.OPEN THE HTML REPORT ↗ ◎SYNTHETIC PASSING CASESee what held.34 completed challenges per arm; no counterexample within those tests.OPEN THE HTML REPORT ↗
WHAT THIS INSTRUMENT CAN — AND CANNOT — ESTABLISH

The examples are synthetic. Passing a set of tests does not prove that a model never leaked information. These reports cannot independently certify customer timestamps, trading performance, or compliance. The current local runner is for trusted, customer-approved Python code; it does not safely isolate arbitrary programs.

PRIVATE PILOT / PREPARATION

Bring one research result.
Let us challenge it.

NO UPLOAD
NO TRADING
NO LIVE CONNECTION

If your team tests quantitative models or automated research strategies, we can help examine one result you are considering trusting. Run the tools in your own environment, keep proprietary data under your control, and independently reproduce any reported failure.

01Start with the original recordsWhat the model predicted, when it predicted it, the simple benchmark, and which outcomes were missing.
02Run the local auditCheck missing outcomes, whether future information slipped in, and whether a simple baseline still holds up.
03Review what changedReproduce the findings and decide whether the evidence supports the research conclusion.
PILOT INQUIRY

Pilot inquiries will open once a verified contact inbox is configured. You can copy an introduction in the meantime.

CONTACT THE RESEARCH TEAM
Research-validation prototype · Constructed demonstration · No investment recommendations, customer performance claims, or production deployment