Ranks vs Sector Z-Scores

What is being compared

One model, one universe, one target, two feature transforms. Everything that could otherwise explain a difference in the result is held fixed: the same gradient boosting regressor, the same expanding-window cross-validation folds, the same hyperparameter grid, the same screen, and the same 15 raw signals. Only the map from a raw signal to a model input changes.

rank version     x  →  percentile of x among all stocks in that month
sector-z version x  →  (x − mean of x in that month × sector)
                       ────────────────────────────────────
                       (sd of x in that month × sector)

The target is identical in both cases and is not sector-relative: the cross-sectional percentile rank of the stock's return over the following month, computed across the whole universe. The question being asked stays "which stocks beat the market next month," never "which stocks beat their own sector."

The universe

• Monthly panel from Sharadar, 2001-01 through 2026-08, 308 months.
• Screen applied before any ranking or scoring: market-cap rank 1,001–3,000 in the month's full cross-section, and a prior-month unadjusted close above $5. That is approximately the Russell 2000 — mega caps dropped, penny stocks dropped.
• 566,602 stock-months survive. Training is 442,264 rows over 240 months; the test period is 124,113 rows over 68 months common to both panels.
• 15 features: momentum, ret_1m, volatility, volatility_12m, illiquidity, dollarvol, marketcap, pb, ps, roe, accruals, assetgrowth, issuance, grossprofitability, divyield.
• Industry is not a feature in either model. It enters the sector-z version only through the cells the z-scores are computed in.

Why rank the features at all

Trees are invariant to any monotone transform applied uniformly to a column, so neither transform is about scaling. Both are about drift. The median market cap of a $5 stock rose roughly fivefold over 2001–2026, so a split learned as "marketcap > 4,000" during the training years lands somewhere quite different in the test years. A percentile is stationary by construction: "top 30% by size" means the same thing in 2003 and in 2026.

Ranking also settles the missing-value question without arbitrariness. 6.6% of rows have at least one missing feature and the estimator rejects NaN outright, so a fill is unavoidable; 0.5 is the middle of the cross-section and is the least opinionated choice available.

What sector z-scores change

Within a single month × sector cell, the z-score is an affine function of the raw value — it cannot reorder two stocks in the same sector in the same month. What changes is what a split means when the model compares stocks across different cells.

1. The sector level is removed from every feature

A stock can no longer be cheap because energy is cheap, only cheap for an energy stock. Price-to-book is the clearest case: banks trade near book because their assets are financial claims carried close to fair value, and software firms trade far above it because they expense the R&D and sales spending that creates their value. A cross-sector price-to-book ranking is measuring accounting convention as much as cheapness. This is the same construction MSCI uses for its sector-neutral quality and enhanced value indexes.

2. The dispersion that ranks discard is kept

Two stocks one percentile apart are one percentile apart whether the month's cross-section is tight or wildly dispersed. In z-scores the same pair is far apart in a calm month and close together in a violent one. Whether that is information or noise is an empirical question, and it is one of the two things this test answers.

Winsorizing is not optional here

The raw features are severely right-skewed, and a mean and standard deviation computed on that is a description of one stock — every other name in the cell gets crushed toward zero. Pooled skewness before any transform:

So raw values are winsorized at the 1st and 99th percentile inside each month × sector cell before the moments are taken, and the resulting z is clipped at ±3 — which is MSCI's own recipe. Missing values fill to 0.0, the cell mean, exactly as the rank version fills to 0.5, the cross-sectional median.

The cells are large enough for this to be well behaved: 3,388 month × sector cells, smallest 22 firms, median 128, none below 10. 225 rows (0.04%) carry no sector at all — fewer than one per month, so there is no cell to score them against — and are dropped. After the transform the features are healthy:

Volatility went from a skewness of 745 to 1.08. Illiquidity, at 3.06, remains the ugliest column and would probably benefit from a log transform before scoring.

Cross-validation

Five expanding-window folds of whole months. The phrase bundles three separate decisions, and it is worth unpacking because the default — scikit-learn's KFold — is wrong here in a way that would have quietly inflated every number in this document.

The mechanics

The months are sorted, cut into six contiguous blocks, and each fold trains on every block up to some point and validates on the one immediately after:

um = np.array(sorted(pd.unique(months_tr)))   # every month, in order
blocks = np.array_split(um, 6)                # six contiguous blocks
for i in range(5):
    tr_m = np.concatenate(blocks[: i + 1])    # blocks 0..i
    va_m = blocks[i + 1]                      # the next block

Six blocks give five folds, because each fold needs a block after it to be scored on. For the 2001–2020 training period that is 240 months in blocks of 40:

block      1      2      3      4      5      6
fold 1   train  valid
fold 2   train  train  valid
fold 3   train  train  train  valid
fold 4   train  train  train  train  valid
fold 5   train  train  train  train  train  valid

Forward-only

Every validation month is later than every training month. KFold shuffles rows into random groups, so it would train on 2018 to predict 2006. That is not a subtle bias: factor premia are regime-dependent, and knowing how the later years went is real information about the earlier ones that no one standing in 2006 could have had. This is the decision that matters most.

Whole months, never split

A month is either entirely in the training set or entirely in the validation set. Stocks within one month share their factor realizations — if value won in March 2015, seeing half that month's stocks tells you something about the other half.

Worth being honest about how much this one buys. At the depths cross-validation actually selected — 1 to 3 — a tree has very little capacity to work out which month a row came from, since month is not a feature. The practical leakage from splitting a month is therefore small here. It costs nothing to prevent, and it would matter a great deal with a deeper model or with month fixed effects in the feature set.

Expanding, not rolling

The training set accumulates rather than sliding a fixed window along. That matches how the model is actually fit for deployment: you have all the history available, so the validation folds should too. A rolling window would be asking a different question — how quickly the strategy adapts to a change in regime — which is worth asking separately but is not what the hyperparameter search needs to know.

What the design costs

Fold 1 trains on 40 months and fold 5 on 200. Those are not the same model, and averaging their scores mixes an undertrained one with a mature one. The reported CV IC is therefore pessimistic relative to what the final fit achieves, since the final fit uses all 240 months.

It also tilts the hyperparameter comparison slightly toward settings that do well on little data, which is plausibly part of why depth kept winning at the bottom of its range. Scoring only folds 3 through 5 would show quickly whether the preference for shallow trees survives once the model has enough history to justify more capacity.

The score itself is the mean within-month Spearman correlation between prediction and realized rank — not R², which on a rank target is near zero and negative about as often as not.

Out-of-sample results

Both models are fit once on 2001–2020 and never refit. Scored on the 124,113 test rows common to both panels, so a difference cannot be a difference of population.

Ranks produce the larger raw signal. The gap is not comfortably significant: the paired difference in monthly IC is 0.0150 with t = 2.00 and p = 0.050, and ranks are better in 60% of months.

The dispersion of the signal matters more than its mean

The sector-neutral signal is smaller but steadier. Its mean IC is 22% lower and its standard deviation is 29% lower, so its t-statistic is higher. Judged on reliability rather than magnitude, sector z-scores win — and the ordering reverses relative to the headline comparison above. This recurs in every risk-adjusted measure in this document.

Neither decile profile is monotone, and both are visibly driven by the bottom decile rather than by a smooth gradient across the top — the first indication of where the performance actually lives.

How the two models differ

The two predictions have a mean monthly rank correlation of 0.737. They are related but genuinely different sorts, not the same model with cosmetic differences.

Dividend yield more than doubles and becomes the single most important feature; volatility more than halves. Both are heavily sector-determined — utilities and REITs anchor the yield distribution, biotech anchors the volatility one — so removing the sector level sharpens one and guts the other. Part of what made raw volatility useful in this universe simply was a sector bet.

Sector tilts of the long decile

Mean absolute tilt halves, from 3.0 to 1.5 percentage points. The rank model has been running a persistent eleven-point underweight in healthcare that nobody asked it for — a sector bet arriving through the back door, because the raw features correlate with sector membership.

A warning about diagnostics. The first measure tried here was the largest single sector share of the long decile, which came out 25.7% for ranks against 25.0% for sector z — nearly identical, and it would have supported the conclusion that the transform neutralized nothing. That statistic is useless: the biggest sector in the decile is big mostly because the biggest sector in the universe is big. Deviation from universe weights is the measure that answers the question.

Alphas and Sharpe ratios

Everything above says the sort works. None of it says the sort is worth anything. The model's features are factor proxies — momentum, gross profitability, price-to-book, market cap, volatility — so a long-short spread built from them could be a repackaging of premia available in an ETF for a few basis points. The regression is what separates the two cases.

Two portfolios per model, equal-weighted and rebalanced monthly. D10−D1 is zero-cost, so no risk-free subtraction and Sharpe is mean over standard deviation. D10 and D1 are reported in excess of the risk-free rate.

Three readings of that table

The benchmark decides the answer. Against the market alone the rank long-short earns 26.6% a year with t = 2.38 — significant, and a number a manager would put on a pitch deck. Add the other five factors and it falls to 10.1% and loses significance. Roughly sixteen points of apparent alpha are RMW and HML exposure: profitability at a loading of 1.26 and value at 0.67. The strategy is a leveraged profitability-and-value bet, which is the same bet a quality ETF and a value ETF make between them. The CAPM R² of 0.07 and 0.03 is the tell — a spread that is nearly market-neutral by construction leaves a single-factor regression with almost nothing to explain, and everything it cannot explain gets called alpha.

Sector z-scores win on risk-adjusted terms. Sharpe 0.89 against 0.82 on the long-short, despite earning 4.4 points a year less, because volatility falls from 26.6% to 19.5%. Removing the sector bet costs return and buys more than its cost in risk. The same ordering appeared in the IC t-statistics and in the D10−D1 t-statistics.

The long leg has no alpha. This is the result that matters for anyone building a long-only recommender. D10 earns 14.0% a year, but its FF6 alpha is −0.76% on an R² of 0.93 — entirely explained by factor exposure — and its CAPM alpha of 1.07% is indistinguishable from zero as well. Both benchmarks agree, so this is not an artifact of the model chosen. Every bit of the long-short alpha comes from the short leg, where D1's FF6 alpha is −10.8% for ranks and −11.3% for sector z, the latter at t = −2.22 — the only alpha in the table that survives the factor controls.

The model is good at identifying bad small-cap stocks and unremarkable at identifying good ones. A long-only portfolio built from its top decile would have delivered returns reproducible by buying a profitability fund. The value is in the leg that requires shorting roughly 180 small caps every month, which is the leg a student portfolio cannot run.

How much of this is sample size

Sixty-seven months is short for judging an alpha. Using each strategy's residual volatility — what the alpha's standard error actually depends on — the sample needed for t = 2 at the estimated effect size is:

Both point estimates are large and neither clears the bar, and at these effect sizes no 67-month sample could have. The correct statement is "not demonstrated," not "no alpha." It is also the reason the paired IC comparison between the two models, at p = 0.050, should be read as suggestive rather than decisive.

A note on model complexity

When the rank model was refit on the full 2001–2026 sample with the hyperparameter grid extended downward, cross-validation chose max_depth = 1 — a decision stump, the shallowest tree that exists — with a mean monthly IC of 0.0456. Depth was the only parameter that mattered, and less was better monotonically at every learning rate. Increasing the number of trees from 300 to 600 changed the score by nothing (0.0456 either way) and 1,200 made it slightly worse (0.0449), so this is not an undertrained model.

Boosted stumps are an additive model: they contain no feature interactions at all. Whatever the gradient boosting is buying over a linear regression on the same 15 inputs, it is not the ability to find interactions among them. That comparison — boosting against plain linear regression on identical features — is the one that decides whether the complexity is earning its keep.