Successful investment strategies are specific implementations of general theories. An investment strategy that lacks a theoretical justification is likely to be false. Hence, an asset manager should concentrate her efforts on developing a theory rather than on backtesting potential trading rules. The purpose of this Element is to introduce machine learning (ML) tools that can help asset managers discover economic and financial theories. ML is not a black box, and it does not necessarily overfit. ML tools complement rather than replace the classical statistical methods. Some of ML's strengths include (1) a focus on out-of-sample predictability over variance adjudication; (2) the use of computational methods to avoid relying on (potentially unrealistic) assumptions; (3) the ability to “learn” complex specifications, including nonlinear, hierarchical, and noncontinuous interaction effects in a high-dimensional space; and (4) the ability to disentangle the variable search from the specification search, robust to multicollinearity and other substitution effects.

Contents

1 Introduction

2 Denoising and Detoning

3 Distance Metrics

4 Optimal Clustering

5 Financial Labels 65

6 Feature Importance Analysis

7 Portfolio Construction

8 Testing Set Overfitting

Review

‘The book’s excellent introduction explains why machine learning techniques will benefit asset managers substantially and why traditional or classical linear techniques have limitations and are often inadequate in asset management. It makes a strong case that ML is not a black box but a set of data tools that enhance theory and improve data clarity. López de Prado focuses on seven complex problems or topics where applying new techniques developed by ML specialists will add value.’ Mark S. Rzepczynski, Enterprising Investor

About the Author

Marcos López de Prado is Professor of Practice at Cornell University's College of Engineering. He has helped modernize finance for the past 20 years, by advancing the adoption of machine learning and supercomputing, and by developing statistical tests that identify false investment strategies (false positives). In recognition of this work, Marcos has received various scientific awards, including the National Award for Academic Excellence (1999) by the Kingdom of Spain, the Quant of the Year Award (2019) by The Journal of Portfolio Management, and the Buy-Side Quant of the Year Award (2021) by Risk.