Methodology

Caliper is a research platform for trading hypotheses. This page describes the standard we hold that research to. It applies to everything the platform produces, whether the question came from you or from our research assistant.

The unit of work

A research question becomes useful when it is testable: a condition, an outcome, and a measurable quantity. "Momentum works in prediction markets" is a belief. "If a market crosses 80 cents within 48 hours of resolution, how often does it resolve yes, and what would a taker have paid to find out" is a hypothesis. Everything downstream operates on hypotheses.

Evidence about the past

We compute what a strategy would have returned over a past period, under stated assumptions. That is evidence about the past. We do not produce forecasts, price targets, or recommendations to buy, sell, size, or time anything, and we do not tailor research to personal financial circumstances. The line is temporal: measuring the record is our work; advice about the future is outside it.

Ideas must be grounded

Every candidate idea cites the research, measurement, or anomaly that prompted it, or states the first principle it rests on and the measurement that would ground it. An empirical pattern without an economic mechanism is treated as a data-mining artifact until proven otherwise. We look for ideas where crowded readers skip: secondary results, robustness tables, limitations sections, and footnotes. Newer work is weighted over older work, because published edges decay from the moment of publication.

Adverse selection is the default assumption

A trade has a counterparty. A credible idea names that counterparty, explains why they keep paying, and explains why the person taking the trade is not the one being selected against. Sitting across from better information is the normal state of markets, and an idea has to argue its way out of it. "Nobody else has noticed" is not an argument. Every edge is treated as decaying rent with a hazard, and publication marks the top of the decay curve.

Pre-registration

The hypothesis, the variants to test, and the kill criteria are written down before anything is measured, each kill test with an explicit numeric threshold. Once written, they are frozen. Retuning a threshold after seeing its data is a new experiment and gets labeled as one. This is the discipline that separates measurement from curve fitting.

Design against the classic traps

Backtests fail in familiar ways, so the familiar failures are designed out up front:

• No look-ahead. Every filter must be feasible at formation time, data is served point-in-time, and adjusted prices are computed as of each date instead of restated over history.
• Costs as a grid. Fees and spread are modeled at multiple fill assumptions on both legs. Optimistic fill models are upper bounds and are labeled as such.
• Maker results and taker results are different claims and are kept separate.
• The benchmark carries information: ask what the counterparty's price already contains.
• Nothing is silently dropped. Every exclusion carries a written reason, and there is no winsorizing.

Sample honesty

• Events are counted rather than observations, and correlated draws are clustered.
• Significance is checked against permutation nulls over the same filled set, and risk-adjusted metrics are deflated by the true number of trials.
• Concentration is always reported: how much of the result is one trade?
• Fewer than roughly twenty winning trades is too few to trust.
• Underpowered is different from disproven. When the sample cannot decide, the honest output is "unproven" together with the sample size that would decide it.

Reproducible computation

Every backtest runs as code in an isolated sandbox with no network access, against an immutable snapshot of historical data that is fully staged before the computation starts. The same code and the same data produce the same result. Every completed backtest must report the same core metrics: a risk-adjusted return measure, a t-statistic, maximum drawdown, annualized return, trade counts and win statistics, compounded versus constant-size returns, and the full list of assumptions the conclusion depends on. A result missing any of these is flagged rather than presented as complete.

Assumptions are stated

No analysis knows your capital scale, market access, fee tier, or execution speed. Every conclusion therefore states the operational assumptions it depends on, and where it is sensitive to one, how it moves as that assumption changes. An edge that requires institutional access is a different claim from one available at retail.

Replication before extrapolation

A known result is reproduced on an overlapping window before out-of-sample numbers count as evidence, and per-period replication is weighted above pooled significance. Measurement precedes capital: paper first, then a budget-capped pilot only after pre-registered gates pass. A zero-capital tripwire is a legitimate research outcome.

Most of this page is enforced by the system rather than left to good intentions: the result contract, the sandbox isolation, and the pre-staged immutable data are code. Where judgment is still required, this is the standard we apply.