Last post, we established why several metrics used to study a trading strategy might not be generally appropriate at all, or not appropriate for prediction market strategies. Before we can properly get to appropriate metrics, we must first understand how growth in a prediction market strategy works.

One contract

To establish some consistency, we define the following three variables:

$p$ : the market price of a contract (or the market implied probability).
$p^*$ : the trader's assessed probability, based on their model.
$\theta$ : the true probability.

When a trader pays $p$ to acquire a contract, there are two metrics for assessing a particular trade. The expected profit and loss is $\mathbb E[PnL]= p^* - p$ and the expected return on deployed capital is $\mathbb E[R] = \frac{p^*-p}{p}.$ In more scientific terms, profit and loss captures the absolute error of the market from our model, and the return captures the relative error.

What's important to recognize in this formula is that we do not know $\theta,$ only $p^*$ so we must use $p^*$ in any actual modeling.

Compounding

Suppose the contract repeats. To distinguish one day from the next, we can add a subscript to denote the day using $t$ and our last day as $T$ . We also introduce the variable $W_t$ to denote the present wealth, and $f$ to be the fraction of wealth wagered each day.

Each day, returns are found by:

$R_t = \begin{cases} 1 + f_t\,(1-p_t)/p_t & \text{with probability } p_t^* \quad (\text{event occurs}) \\ 1 - f_t & \text{with probability } 1 - p_t^* \quad (\text{event does not occur}) \end{cases}$

The quantity $b = (1-p)/p$ is the implied odds of the contract (often used by sportsbooks). After T independent days, wealth is modelled by:

$W_T = W_0 \prod_{t=1}^T R_t$

Note that compounding is multiplicative, so we must measure the geometric mean rather than the more common arithmetic mean.

The geometric mean measures the average of a logarithm, so we define the log-growth rate (or growth rate) of a strategy with bankroll fraction f as:

$g_t(f_t) = \mathrm{E}[\ln R_t] = p_t^* \ln\!\left(1 + \frac{f_t(1-p_t)}{p_t}\right) + (1-p_t^*)\ln(1-f_t)$

And this is what we want to measure. Compounding works for prediction markets just as much as classical markets.

Why not use an arithmetic mean? The arithmetic expectation of return $\mathbb E[R - 1] = f(p^* b - 1)$ is maximized by f = 1 (bet everything). But growth could be an infinite loss because a single loss wipes out the entire bankroll. The difference between the arithmetic and geometric objectives can be called variance drag: for small f,

$g(f) \approx f\,\mathrm{E}[r] - \tfrac{1}{2}f^2\,\mathrm{Var}(r)$

where r = R − 1 is the single-period return. The variance penalty term caps bet size.

The Kelly Criterion

Taking $g_t'(f_t) = 0$ and solving for $f_t$ produces the Kelly fraction $f^*_t = \frac{p_t^* - p_t}{1 - p_t}$ The numerator is the edge, the denominator the maximum possible loss per dollar. We find that the optimal growth rate is then

$g_t^* = p^*_t \ln\frac{p^*_t}{p_t} + (1-p^*_t)\ln\frac{1-p^*_t}{1-p_t}$

This formula goes by another name: the Kullback–Leibler (KL) divergence. For two probability distributions on a binary outcome, with probabilities q and p respectively, the KL divergence from p to q is:

$D_{\mathrm{KL}}(q||p) = q\ln\frac{q}{p} + (1-q)\ln\frac{1-q}{1-p}$

$D(q||p)$ measures the "information distance" between $p$ and $q$ with respect to the distribution $q$ . It is always a non-negative quantity, and is zero iff $q=p$ , and is asymmetric.

Using this, we can also measure our optimal modelled growth rate as $g_t^* = D_{\mathrm{KL}}(p_t^*||p_t)$

Nature Returns

However, the markets actually operate on the probability distribution $\theta_t$ . So the actual returns will be

$g_t = \theta_t \ln\frac{p^*_t}{p_t} + (1-\theta_t)\ln\frac{1-p^*_t}{1-p_t},$

which we can rewrite (by adding and subtracting the entropy of $\theta$ ) as

$g_t = D_{\mathrm{KL}}(\theta_t || p_t) - D_{\mathrm{KL}}(\theta_t || p_t^*).$

This decomposition is our key result: The realized growth rate is the opportunity $D_{\mathrm{KL}}(\theta||p)$ , or how incorrect is the market, minus the cost $D_{\mathrm{KL}}(\theta||p^*),$ or how wrong the trader is.

We can apply this to get a long-run growth rate

$\mathbb E[g_t] = \mathbb E[D_{\mathrm{KL}}(\theta_t || p_t)] - \mathbb E[D_{\mathrm{KL}}(\theta_t || p_t^*)].$

The first term describes the average mispricing of the market. This is where a Brier score is useful (see part 2), but it is entirely outside the control of a trader. Only the second term can be controlled.

Error Estimation and Fractional Kelly

If we have a perfectly unbiased estimator, i.e. $p_t^* = \theta_t + \epsilon_t$ where $\mathbb E[\epsilon_t] = 0$ and $\epsilon_t$ is small, we can estimate using a Taylor expansion that $D_{\mathrm{KL}}(\theta||p^*) \approx \frac{\epsilon^2}{2\theta(1-\theta)}.$

This term is proportional to the Fisher information for a Bernoulli random variable of parameter $\theta,$ which measures the estimation variance

$\Delta g = \frac12\mathbb E\left[\frac{\epsilon^2_t}{\theta_t(1-\theta_t)}\right],$

and comes in the form of a penalty to growth rate. The structure of this informs us that error is most costly at the extremes where $\theta$ is near 0 or 1, which is where Kelly sizing would place the largest bets because of the high returns.

One way to mitigate this damage is to use what is called fractional Kelly, $f = \alpha f^*$ where $\alpha$ is the Kelly multiplier. It is a convex combination of the trader's estimate and market price, admitting that the market may be partly right. An optimal $\alpha$ will depend on that estimation variance. The growth rate should be recomputed as $\alpha$ changes.

Further Topics

All of these considerations will influence the fundamental wealth relation $W_T = W_0 \prod_{t=1}^T R_t$ Measuring this will allow us to consider returns in their proper geometric regime to arrive at an appropriate replacement to the Sharpe ratio: one that considers the log-returns and variance. Next time, we'll discuss similar metrics built off the geometric growth and the binary nature of returns.

A lot of these ideas are important, and will make appearances in future blog posts as we build on them, such as applications to portfolio management and capital allocation.