# Benchmarking Market Sentiment Classifiers

A very simple trading bot is one that buys when a market turns from bearish to
bullish and sells when it turns from bullish to bearish. The entity that informs
the bot about this sentiment is called a *sentiment classifier*.

Given your trading frequency, there is in fact a theoretical upper limit to how
much you can earn by trading, called the *peak return*. It can act as a
reference to benchmark different types of classifiers.

## Preliminaries

**Definition:** The **trading frequency** $f$ is the *maximum* frequency at which a trader
chooses to trade. For example, if a trader limits buy/sell operations for a
single asset to once a day, than the trading frequency is $1/\text{day}$. The
trader can choose not to trade for certain days—this doesn’t change the
trading frequency.

**Definition:** The **price** of an asset $p$ is a piecewise linear function
$p:\mathbb{R}\to\mathbb{R}$ where $p$ is linear in constant intervals $1/f$
and $p(t)>0$ for all $t\in\mathbb{R}$,

**Definition:** Given initial account balance $C_0$ and asset balance $A_0$,
a *purchase* of $\bar{A}$ assets with $\bar{C}$ capital at time $t_0$ renders
the balances

where the relation $\bar{C} = p(t_0) \bar{A}$ holds. Similarly, a sale of $\bar{A}$ assets for $\bar{C}$ capital at time $t_0$ renders the balances

Finally,
a *full purchase* at time $t_0$ renders the balances

and a *full sale* renders the balances

A purchase $B$ taking place at time $t_n$, spending $\bar{C}_n$ amount of capital is denoted by the 3-tuple $(t_n, p(t_n), \bar{C}_n)$.

Similarly, a sale $S$ taking place at time $t_n$, selling $\bar{A}_n$ amount of assets is denoted by the 3-tuple $(t_n, p(t_n), \bar{A}_n)$.

**Definition:** A **trade** refers to either a purchase or a sale. A
**trade cycle** $C$ is a 2-tuple of a consecutive full purchase and full sale
$(B_n, S_{n+1})$.
*Trade cycle* will be referred to as *cycle* for brevity.

**Definition:** The **return** $R$ of a cycle taking place between $t_n$ and
$t_{n+1}$ is calculated as $p(t_{n+1})/p(t_n)$. In other words, the return of a
cycle is how much it multiplies an initial capital, generally denoted by a
number followed by x, e.g. 3x.
Given a series of cycles
$\{C_1, C_2, \dots, C_n \}$ with returns
$\{R_1, R_2, \dots, R_n \}$ their combined return is equal to
$R_1 R_2 \cdots R_n$.

## Fundamental Theorem of Trading and Peak Return

Most traders are familiar with the saying *“buy the dip, sell the rally”*, which
is very reasonable advice if you can guess when the dips and rallies are. We
nevertheless know exactly when the **past** dips and rallies are, and can use them
to calculate the peak return for a given period in the past.

**Fundamental Theorem of Trading (FTT):** For a given price function $p$ and time
period $[a, b]$
the return is maximized when assets are fully bought at local minima (dips),
and fully sold at local maxima (rallies). This value is called the **peak
return** $PR$.

We assume that trading starts with a full purchase and ends with a full sale. If

the first purchase takes place at $a$; otherwise it takes place at the next local minimum. If the last extremum before $b$ is a local minimum, the last sale takes place at $b$.

For a given set of points, it is easy to calculate peak return algorithmically. It is also important to note the more general analytic definition that extends beyond the piecewise-linearity assumption:

where $H$ is the Heaviside step function, $p’$ is $p$’s first derivative, and $\rho(t)$ is defined as the multiplication of returns of cycles taking place before $t$.

To summarize, peak return is the maximum return possible for a given price function.

## Sentiment Classifiers

It is not possible to predict local minima and maxima of a price function precisely. However, it is possible to predict them to a degree of confidence.

Since we can’t know with certainty whether the current price value is a local
minimum or maximum, we name them differently; namely **buy signal** instead of
a possible local minimum and **sell signal** instead of a possible local
maximum.

We conceptualize a **sentiment classifier** that takes some input, and predicts the
market sentiment as bullish (the price will be higher at some point in the
future) or bearish (the price will be lower at some point in the future).

**Definition:** A **perfect sentiment classifier** (PSC) is a hypothetical classifier
that predicts with 100% accuracy whether

for a given time $\bar{t}$.

## A Simple Trading Bot

We conceptualize a trading bot using a sentiment classifier, which starts with some initial capital, fully buys when the sentiment turns from bearish to bullish and fully sells when the sentiment turns from bullish to bearish.

**Lemma:** A trading bot can achieve peak return with a PSC.

## Smoothing Sentiment Classifiers

The more a sentiment classifier diverges from a perfect one, the lower are the returns of the trading bot that uses it.

**Definition:** A **smoothing sentiment
classifier** (SSC) is one that can accurately predict whether

for a given time $\bar{t}$, a period of length $\alpha$ preceding $\bar{t}$ and a period of length $\beta$ following $\bar{t}$. In other words, the smoothing classifier predicts the relationship between means of the price function preceding and following current time.

For a piecewise constant price function consisting of a discrete set of values $\boldsymbol{p} = [p_0, p_1, \dots, p_n]$ and corresponding equidistant time values $\boldsymbol{t} = [t_0, t_1, \dots, t_n]$, an SSC can simply be rephrased as one that predicts whether

for time $t_i$, $X$ preceding points including $t_i$ and $Y$ following points excluding $t_i$. This classification is visualized in the following figure:

To better visualize the concept, we use 3.5 years of hourly historical Bitcoin price data from Coinbase:

We plot the classes for different values of $X$ and $Y$ to observe the smoothing effect:

Notice that smoothed sentiments make more sense visually in terms of reflecting long term trends.

### The Effect of Smoothing on the Number of Trade Cycles

Smoothing increases the overall length of contiguous bullish or bearish periods, and decreases the number of signals for a given price function. Therefore, the number of trade cycles also decreases. Below is a graph of the number of cycles for increasing $X$ and constant $Y$:

It can be observed from the graph that the relationship between smoothing factors and number of cycles is roughly hyperbolic, given by the following function

which ensures that $f(1)=1$. Here, $Y$ is kept constant at 1 and $X$ is allowed to increase.

One can do the same vice versa for $Y$ by keeping $X$ constant, and find the parameters

Then, the multi-variable function for the number of cycles can be approximated by

which can be used later in closed-form statistical derivations. Below is a plot of the relative error for the BTC-USD pair:

### The Effect of Smoothing on Final Return

We can observe the effect of smoothing on the final return of a trading bot, when we plot the returns for varying values of $X$ and $Y$.

Final return decreases super-exponentially with increasing smoothing factor.

## The Effect of Fuzzing Classifier Output

In the previous sections, we introduced a hypothetical classifier that work with 100% accuracy. However, a classifier having access to a finite amount of information can never achieve that kind of success rate. To see how decreased accuracy affects final returns, we can introduce errors retroactively to a PSC output. In other words, we “fuzz” the 100% correct case by introducing random errors. The process can be summarized as:

- Obtain correct sentiment classes.
- Iterate through array elements and change the value if a randomly generated number is smaller than Z%, that is, the fuzzing amount. This is analogous to a Poisson process.
- Calculate final return with the fuzzed classes.
- Repeat steps 1-3 many times to obtain a large enough distribution of final returns for Z, and take the average.
- Repeat steps 1-4 for different Z.

After running the process for the prices given in the previous section, we arrive at the following result:

On average, final return decreases exponentially with increasing fuzzing amount, given that the errors have a Poisson distribution. The rate of decay decreases with increasing smoothing factor. The lines intersect at 50% fuzzing with ~7.5x return. 0% return is obtained by fuzzing around 50~60% for different smoothing factors. These values are not universal and depend on the given prices.

This and the previous relations can be utilized in hyperparameter optimization when building sentiment classifiers with machine learning models.

## Conclusion

We introduced fundamental concepts such as peak return and sentiment classifier for a more mathematical understanding of trading. By breaking down the decision-making process of trading into different entities, we can reason more efficiently when choosing models. By introducing mathematical limits to trading performance, we can benchmark and compare different models. Overall, the results obtained should be helpful in developing statistically sound trading algorithms.