Sep

2

Counting: Seasonality

September 2, 2024 |

A lesson from the archives: Seasonality and changing cycles, by Victor Niederhoffer and Laurel Kenner, (04/26/2004)

A good part of the anomaly literature is devoted to studies of seasonality. A basic problem with these studies is that merely picking a season to study involves making guesses as to when and where the seasonality is. For example, is it in January or December, on Monday or Friday, in the United States or the Ukraine? (Yes, our Google search turned up a study of anomalies in the Ukraine.) Thus, the very choice of a subject might involve random luck.


Another aspect of seasonality studies that must be considered is whether the effects noted are sufficient to cover transaction costs. A retrospective study showing that you can make 2 cents more on Friday trades than Monday trades in your typical $50 stock would not be sufficient in practice to leave anyone but the broker and the market-maker richer.

Thus, it's essential to temper the conclusions of such studies with out-of-sample testing — in other words, with real trading.

[ … ]

Comment by Philip J. McDonnell, a former student of the Chairman at UC Berkeley: Dr. Niederhoffer points out that there is no a priori reason to believe that any one day of the week is stronger than any other. Thus when Y— collected the data (thank you!), presumably the reason was to find out if any days of the week behaved differently. Only after peeking at the data was it possible to say that Monday was the best and Tuesday the worst.

There are 10 such pairwise comparisons:
Mon with other 4 days 4
Tues with 3 last days 3
Wed with Thu & Fri 2
Thu with Fri 1
Total 10

In other words it is also possible that Tuesday could have been the best day and Monday the worst or any other pairwise comparison by chance alone. So when the one best and the one worst day shown by the data are compared and shown to have say a 5% significance we need to remember that we implicitly ruled out the other nine cases which weren't the best or worst. So we need to take our 5% number and multiply by 10 to get the correct significance of 50%. 50% is exactly consistent with randomness.

The problem is multiple comparisons are often subtle and remain unrecognized. Multiple comparisons are insidious because they dramatically reduce the power of the statistical tests we employ.

[ … ]

[More reading: Multiple comparisons problem]

William Huggins offers:

Bonferonni method suggests raising the confidence level proportional to the number of tested hypotheses. To get 95% confidence despite ten tests, he suggests 99.5 as a threshold.  It's a huge problem when testing which variables to include in a regression model.

Asindu Drileba writes:

The right way to do this type of thing is to form a specific hypothesis based on a single comparison and then to test it on the data. It is even possible to use data from a prior period to formulate our hypothesis. We then test our hypothesis on the subsequent period which excludes the period where we formed our hypothesis.

This is an approach used in machine learning. Datasets are always split into "training" and "test" datasets. "Training" datasets are exclusively used to build the components of the model. "Test" datasets are not used to build the model at all. They are excluded when building the model. The model built using the "training" dataset is then asked to make predictions on the "test" dataset. The accuracy on predictions made on the "test" datasets is then used to determine how accurate the model is (so it can be tuned for improvement or thrown away).

I found this particular statement from the full post so insightful because I didn't think of applying this approach to building models using other statistical methods (I thought it was something limited to machine learning).


Comments

Name

Email

Website

Speak your mind

1 Comment so far

  1. Emmanouel Xylouris on September 2, 2024 8:58 am

    We should not forget that the market is not a thing that is distinct from its participants.
    As soon as the market or the conscious mind discovers the pattern it dissolves.
    You can bet your silver everyday there are plenty of entities looking for such patterns. Thereby expediting their demise.

    You can not show up at the poker table and think by using scientific method you can gain an edge. This would presuppose that the other players are retarded.

    You can however do it on a game like blackjack where the dealer must play a predetermined strategy.

Archives

Resources & Links

Search