There are several reasons why probabilistic machine learning represents the next-generation ML framework and technology for finance and investing. This generative ensemble learns continually from small and noisy financial datasets while seamlessly enabling probabilistic inference, retrodiction, prediction, and counterfactual reasoning. Probabilistic ML also lets you systematically encode personal, empirical, and institutional knowledge into ML models.

Whether they're based on academic theories or ML strategies, all financial models are subject to modeling errors that can be mitigated but not eliminated. Probabilistic ML systems treat uncertainties and errors of financial and investing systems as features, not bugs. And they quantify uncertainty generated from inexact inputs and outputs as probability distributions, not point estimates. This makes for realistic financial inferences and predictions that are useful for decision-making and risk management.

Unlike conventional AI, these systems are capable of warning us when their inferences and predictions are no longer useful in the current market environment. By moving away from flawed statistical methodologies and a restrictive conventional view of probability as a limiting frequency, you'll move toward an intuitive view of probability as logic within an axiomatic statistical framework that comprehensively and successfully quantifies uncertainty. This book shows you how.

Why I Wrote This Book

There is a paucity of general probabilistic ML books, and none that is dedicated entirely to finance and investing problems. Because of the idiosyncratic complexities of these domains, any naive application of ML in general and probabilistic ML in particular is doomed to failure. A depth of understanding of the foundations of these domains is pivotal to having any chance of succeeding. This book is a primer that endeavors to give the thinking practitioner a solid grounding in the foundational concepts of probabilistic ML and how to apply it to finance and investing problems, using simple math and Python code.

There is another reason why I wrote this book. To this day, books are still a medium for serious discourse. I wanted to remind the readers about the continued grave flaws of modern financial theory and conventional statistical inference methodology. It is outrageous that these pseudoscientific methods are still taught in academia and practiced in industry despite their deep flaws and pathetic performance. They continue to waste billions of research dollars producing junk studies, tarnish the reputation of the scientific enterprise, and contribute significantly to economic disasters and human misery.

We are at a crossroads in the evolution of AI technologies, with most experts predicting exponential growth in its use, fundamentally transforming the way we live, work, and interact with one another. The danger that AI systems will take over humanity imminently is silly science fiction, because even the most advanced AI system lacks the common sense of a toddler. The real clear and present danger is that fools might end up developing and managing these powerful savants based on the spurious models of conventional finance and statistics. This will most likely lead to catastrophes faster and bigger than we have ever experienced before.

My criticisms are supported by simple math, common sense, data, and scholarly works that have been published over the past century. Perhaps one added value of this book is in retrieving many of those forgotten academic publications from the dusty archives of history and making readers aware of their insights in plain, unequivocal language using logic, simple math, or code that anyone with a high school degree can understand. Clearly, the conventional mode of expressing these criticisms hasn’t worked at all. The stakes for individuals, society, and the scientific enterprise are too high for us to care if plainly spoken mathematical and scientific truths might offend someone or tarnish a reputation built on authoring or supporting bogus theories.

Who Should Read This Book?

The primary audience of this book is the thinking practitioner in the finance and investing discipline. A thinking practitioner is someone who doesn’t merely want to follow instructions from a manual or cookbook. They want to understand the underlying concepts for why they must adopt a process, model, or technology. Generally, they are intellectually curious and enjoy learning for its own sake. At the same time, they are not looking for onerous mathematical proofs or tedious academic tomes. I have provided many scholarly references in each chapter for readers who are looking for the mathematical and technical details underlying the concepts and reasoning presented in this book.

A thinking practitioner could be an individual investor, analyst, developer, manager, project manager, data scientist, researcher, portfolio manager, or quantitative trader. These thinking practitioners understand that they need to learn new concepts and technologies continually to advance their careers and businesses. A practical depth of understanding gives them the confidence to apply what they learn to develop creative solutions for their unique challenges. It also gives them a framework to explore and learn related technologies and concepts more easily.

In this book, I am assuming that readers have a basic familiarity with finance, statistics, machine learning, and Python. I am not assuming that they have read any particular book or mastered any particular skill. I am only assuming that they have a willingness to learn, especially when ChatGPT, Bard, and Bing AI can easily explain any code or formula in this book.

Table of Contents

Chapter 1. The Need for Probabilistic Machine Learning

Chapter 2. Analyzing and Quantifying Uncertainty

Chapter 3. Quantifying Output Uncertainty with Monte Carlo Simulation

Chapter 4. The Dangers of Conventional Statistical Methodologies

Chapter 5. The Probabilistic Machine Learning Framework

Chapter 6. The Dangers of Conventional Al Systems

Chapter 7. Probabilistic Machine Learning with Generative Ensembles

Chapter 8. Making Probabilistic Decisions with Generative Ensembles

Review

"In his no-nonsense defiant style, Kanungo dismisses modern orthodoxies to deliver a superb analysis of probabilistic machine learning; not as a solution, but the most sensible way forward for FinTech." —Ian Angell, Professor Emeritus, London School of Economics.

"Explaining the flaws of conventional models, and the realistic predictions of probabilistic ML models for finance and investing, this book is a significant leap forward in minimizing the reliance on intuition." —Bruno Rignel, Chief Investment Officer, Alpha Key Capital Management.

"An enlightening book. It makes me think about the flaws of recent machine learning models. A probabilistic machine learning approach gives us another tool to apply." - Abdullah Karasan, Founder of Leveragai & Faculty, University of Maryland, Baltimore County.

"Finally, a lucid examination of the fallacies of classical statistics, particularly null hypothesis significance testing and confidence intervals. This book demonstrates the power of modern probabilistic ML and their generative ensembles through insightful applications to finance and investing." - Mike Shwe, former Product Manager, TensorFlow Probability, Google Inc.

"One of the best Generative Models books of all time" and "One of the best Probabilistic Algorithms books of all time" - BookAuthority

About the Author

Deepak Kanungo is an algorithmic derivatives trader, instructor, and CEO of Hedged Capital LLC, an AI-powered proprietary trading company. Since 2019, Deepak has taught tens of thousands of O'Reilly Media subscribers worldwide the concepts, processes, and machine learning technologies for algorithmic trading, investing and finance with Python.

In 2005, long before machine learning was an industry buzzword, Deepak invented a probabilistic machine learning method and software system for managing the risks and returns of project portfolios. It is a unique probabilistic framework that has been cited by IBM and Accenture, among others.

Previously, Deepak was a financial advisor at Morgan Stanley, a Silicon Valley fintech entrepreneur, a director in the Global Planning Department at Mastercard International and a senior analyst with Diamond Technology Partners. He was educated at Princeton University (astrophysics) and The London School of Economics (finance and information systems).