One anomaly study finds little expected return after demanding adjustments; institutional trading research finds feasible implementations for selected familiar styles. Those results need not conflict. Validating factors before portfolio implementation means identifying whether the candidate fails because it was selected by chance, repeats an existing exposure or cannot retain its value when traded.
- The Federal Reserve anomaly estimate describes a defined study, not a customer return forecast.
- Execution redesign matters only if it preserves the exposure being validated.
- False discovery, redundancy and implementation cost require different responses.
- A surviving factor still needs evidence of incremental portfolio value and adverse outcome exposure.
Why the two cost studies can both be right
A Federal Reserve study examines 120 stock market anomalies after allowing for effective spreads, effects following publication and the modern trading era. It estimates an average expected return of 8 basis points per month under those adjustments. Price impact is omitted and combinations of anomalies are outside the experiment. The number is a finding about that study's portfolios and methods. It does not measure a customer's realised return, price a current strategy or establish the profitability of every factor combination. [3]
AQR's institutional trading study finds more favourable cost and exposure tradeoffs for value and momentum than for rapid reversal strategies. It uses actual trading evidence and portfolio optimisation to investigate feasible implementation. The apparent disagreement with the anomaly study is informative because strategy selection and construction differ. These are separate experiments, so their summaries cannot be averaged into one profitability estimate. They make the implementation definition part of the result a reader needs to understand. [4]
The constructive cost study prevents a lazy conclusion that any trading friction invalidates factor investing. The relevant analytical test is whether feasible construction preserves enough of the intended exposure to justify its realised cost. A cheaper portfolio that no longer expresses the mechanism is not an implementation victory; a more patient portfolio that retains the relationship may be. The research therefore motivates comparing alternative routes to the same intended exposure, with both tracking error and cost visible. That comparison must use the actual investor's instruments and scale before becoming evidence about their prospective results. [4]
A discovery needs an independent challenge
AQR's discussion of data mining describes researchers trying many variants while disproportionately reporting successes. Under that selection process, conventional significance claims can exaggerate what the winning backtest establishes. The discussion emphasises evidence outside the original sample and a credible economic explanation. It is a practitioner's methodological argument, rather than an estimated false discovery rate. Its relevance is the search history behind the factor, which a persuasive final chart can conceal. [2]
The FactorBench preprint evaluates automated factor mining through signal validity, generalisation through time, incremental prediction beyond measured exposures and portfolio results after costs. Its abstract reports no consistently dominant mining paradigm in the comparison. The work is an author preprint, with the repository providing access rather than independent institutional endorsement. Its useful contribution here is the evaluation structure: different discovery algorithms can be compared through common signal and portfolio procedures. The abstract's conclusion is bounded by its benchmark, and does not establish that automated discovery in general succeeds or fails. [1]
A discovery audit starts before the chosen statistic. Retain candidate families, transformations, sample choices and failed attempts, then keep validation data from becoming another repeatedly optimised discovery sample. This is a proposed response to AQR's selection argument. If the later test is repeatedly redesigned after disappointment, it no longer supplies the same independent challenge. Economic reasoning strengthens a testable hypothesis; it cannot turn a selected historical success into a result that has already replicated. [2]
Point in time backtesting adds an information contract to the benchmark idea. A defensible test would preserve when inputs became available, which version was used and when a feasible trade could follow the signal. This is an analytical requirement for the implementation being assessed, not a finding that FactorBench has verified every such issue. A historical value available only after the proposed decision should not receive credit for predicting that decision's outcome. The same contract should govern competing methods so a data advantage cannot be mistaken for superior discovery. [1]
Diagnose the failure before redesigning the strategy
AQR's full Academic Alpha paper distinguishes factors affected by overfitting, factors that resemble existing patterns and factors that cannot be implemented at useful scale after costs. It argues that assessing the body of evidence and designing positions requires skill. This is a manager's perspective, with a commercial interest in the strategy family. Its distinction remains useful because those failure modes call for different responses. A redundant signal may be unnecessary, an expensive one may need different construction and a selected statistical accident may have no implementable relationship to preserve. [7]
After discovery validation, execution needs a distinct test. Start with feasible positions and assess what remains after costs and after the original discovery period. The Federal Reserve anomaly findings make a small remaining margin consequential, especially when an omitted cost could consume it. Failure here identifies an implementation problem, not automatically a false discovery. Redesign deserves another test only if the intended signal survives the cheaper construction. No customer strategy is tested by this proposed procedure. [3]
A separate Federal Reserve study permits selected risk factors to change at previously unknown break dates. In its examined factor set, ignoring those breaks can retain factors that are no longer relevant, while selected factors price the omitted ones within regimes. This creates a failure mode distinct from excessive search. A relationship can have mattered historically without adding useful information in another regime. The result is specific to the studied factors and model, rather than a universal factor expiry rule. [5]
Does the surviving exposure add to the portfolio?
Finally, test incremental contribution. If a new factor mostly repeats a pattern already held, its stand alone chart can exaggerate the value of adding it. The AQR paper's discussion of correlated discoveries suggests comparing the proposed exposure with the existing portfolio and asking whether any benefit remains after shared drivers are recognised. Position sizing and unintended exposures belong in that comparison. The analytical goal is a useful contribution under feasible construction, rather than a count of individually attractive signals. More factors deserve more space only when they improve the portfolio's stated objective. [7]
BIS research on cumulant risk demonstrates that strategies can be exposed to features of return distributions beyond variance, particularly when leverage is substantial. It develops a measure using leveraged exchange traded funds and finds material stress related risk compensation across asset classes. The authors identify implications for factor models and momentum strategies. This evidence adds a question that an average return test cannot resolve on its own: what adverse distributional behaviour accompanies the premium? It does not show that every profitable factor is simply hidden leverage or that all factor portfolios share the same tail exposure. [6]
A factor's admission is provisional. If its incremental explanatory value disappears across plausible regime definitions, the historical case deserves reassessment. The structural break research motivates a review of which relationship changed, rather than an automatic rescue through new parameters. Record whether a revision is responding to new evidence or merely fitting the disappointing period. Otherwise a sequence of adaptations can erase the distinction between an accepted strategy and an ongoing search. The watchlist should preserve the original test alongside later revisions so the claim of replication remains meaningful. [5]
A deterioration concentrated in adverse outcomes may also expose a risk that ordinary performance measures understated. The BIS cumulant evidence motivates inspecting whether leverage or distributional sensitivity explains that deterioration, without assuming it does. A credible response would identify the mechanism, test the revised explanation and assess the combined portfolio consequence. The research case becomes stronger when it states what could refute it and survives that test. It becomes weaker when every loss is retrospectively renamed the premium investors were always meant to earn. [6]
Editorial assessment. The admission decision should follow the failure diagnosed. A selected statistical accident calls for rejection or genuinely new validation. A redundant factor calls for a contribution test against existing holdings. An expensive but credible exposure calls for an execution comparison that keeps costs and tracking error visible. The two cost studies favour this discriminating assessment over a universal verdict on factors. Better construction can address a trading problem, while leaving a discovery failure or an unwanted shared exposure untouched.
Evidence scope. The article does not replicate the estimates or identify a current profitable factor. The anomaly and institutional trading studies use different designs and cannot be combined into one profitability forecast. FactorBench is a preprint discussed through its abstract. Manager research has commercial context. The proposed admission process identifies separate failure modes, without certifying that a factor survives them or provides an investment edge.