This book, together with specially prepared online material freely accessible to our readers, provides a complete introduction to Machine Learning, the technology that enables computational systems to adaptively improve their performance with experience accumulated from the observed data. Such techniques are widely applied in engineering, science, finance, and commerce.

This book is designed for a short course on machine learning. It is a short course, not a hurried course. From over a decade of teaching this material, we have distilled what we believe to be the core topics that every student of the subject should know. In addition, our readers are given free access to online e-Chapters that we update with the current trends in Machine Learning, such as deep learning and support vector machines. We chose the title `learning from data' that faithfully describes what the subject is about, and made it a point to cover the topics in a story-like fashion. Our hope is that the reader can learn all the fundamentals of the subject by reading the book cover to cover. Learning from data has distinct theoretical and practical tracks. In this book, we balance the theoretical and the practical, the mathematical and the heuristic. Theory that establishes the conceptual framework for learning is included, and so are heuristics that impact the performance of real learning systems. What we have emphasized are the necessary fundamentals that give any student of learning from data a solid foundation.

The authors are professors at California Institute of Technology (Caltech), Rensselaer Polytechnic Institute (RPI), and National Taiwan University (NTU), where this book is the text for their popular courses on machine learning. The authors also consult extensively with financial and commercial companies on machine learning applications, and have led winning teams in machine learning competitions.

Contents

1. The Learning Problem

2. Training versus Testing

3. The Linear Model

4. Overfitting

5. Three Learning Principles

About the Author

Dr. Yaser S. Abu-Mostafa is Professor at the California Institute of Technology. His areas of expertise are Machine Learning and Computational Finance. He received his PhD from Caltech where he was awarded the Clauser Prize for the most original doctoral thesis, and later received the Feynman Prize for excellence in teaching. In 2005, the Hertz Foundation established the Abu-Mostafa Fellowship in his honor. He has served as scientific advisor to several corporations and start-up companies in the US and abroad. He has travelled extensively and is fluent in 3 languages.

Malik Magdon Ismail obtained a B.S. in Physics from Yale University in 1993 (summa cum laude, phi beta kappa) and a Masters in Physics (1995) and a PhD in Electrical Engineering with a minor in Physics from the California Institute of Technology in 1998, winning the Wilts prize. He is currently a professor of Computer Science at Rensselaer Polytechnic Institute (RPI), where he is a member of the Theory group. His research interests have included the theory and applications of machine learning, social network algorithms, communication networks and computational finance. In particular, he is interested in the statistical, theoretical and algorithmic aspects of learning from data. He also has consulted in a variety of capacities in computational finance and data mining.

Hsuan-Tien Lin received a B.S. in Computer Science and Information Engineering from National Taiwan University in 2001, an M.S. and a Ph.D. in Computer Science from California Institute of Technology in 2005 and 2008, respectively. He joined the Department of Computer Science and Information Engineering at National Taiwan University as an assistant professor in 2008, and won the outstanding teaching award from the university in 2011. His research interests include theoretical foundations of machine learning, studies on new learning problems, and improvements on learning algorithms. He received the 2012 K.-T. Li Young Researcher Award from the ACM Taipei Chapter, and co-led the team that won the third place of KDDCup 2009 slow track, the champion of KDDCup 2010, and the double-champion of the two tracks in KDDCup 2011.