Gain the valuable skills and techniques you need to accelerate the delivery of machine learning solutions. With this practical guide, data scientists and ML engineers will learn how to bridge the gap between data science and Lean software delivery in a practical and simple way. David Tan and Ada Leung from Thoughtworks show you how to apply time-tested software engineering skills and Lean delivery practices that will improve your effectiveness in ML projects.

Based on the authors' experience across multiple real-world data and ML projects, the proven techniques in this book will help teams avoid common traps in the ML world, so you can iterate more quickly and reliably. With these techniques, data scientists and ML engineers can overcome friction and experience flow when delivering machine learning solutions.

This book shows you how to:

• Apply engineering practices such as writing automated tests, containerizing development environments, and refactoring problematic code bases
• Apply MLOps and CI/CD practices to accelerate experimentation cycles and improve reliability of ML solutions
• Design maintainable and evolvable ML solutions that allow you to respond to changes in an agile fashion
• Apply delivery and product practices to iteratively improve your odds of building the right product for your users
• Use intelligent code editor features to code more effectively

Table of Contents

Chapter 1. Challenges and Better Paths in Delivering ML Solutions

Part I. Product and Delivery

Chapter 2. Product and Delivery Practices for ML Teams

Part II. Engineering

Chapter 3. Effective Dependency Management: Principles and Tools

Chapter 4. Effective Dependency Management in Practice

Chapter 5. Automated Testing: Move Fast Without Breaking Things

Chapter 6. Automated Testing: ML Model Tests

Chapter 7. Supercharging Your Code Editor with Simple Techniques

Chapter 8. Refactoring and Technical Debt Management

Chapter 9. MLOps and Continuous Delivery for ML (CD4ML)

Part III. Teams

Chapter 10. Building Blocks of Effective ML Teams

Chapter 11. Effective ML Organizations

Whether you’re a ML practitioner in academia, an enterprise, a start-up, a scale-up, or consulting, the principles and practices in this book can help you and your team become more effective in delivering ML solutions. In line with the cross-functional nature of ML delivery techniques that we detail in this book, we address the concerns and aspirations of multiple roles in teams doing ML:

Data scientists and ML engineers: The job scope of a data scientist has evolved over the past few years. Instead of purely focusing on modeling techniques and data analysis, we’re seeing expectations (implicit or explicit) that one needs to possess the capabilities of a full-stack data scientist: data wrangling, ML engineering, MLOps, and business case formulation, among others. This book elaborates on the capabilities necessary for data scientists and ML engineers to design and deliver ML solutions in the real world.

In the past, we’ve presented the principles, practices, and hands-on exercises in this book to data scientists, ML engineers, PhD students, software engineers, quality analysts, and product managers, and we’ve consistently received positive feedback. The ML practitioners we’ve worked with in the industry have said that they benefited from improvement in feedback cycles, flow, and reliability that comes from practices such as automated testing and refactoring. Our takeaway is that there is a desire from the ML community to learn these skills and practices, and this is our attempt to scale the sharing of this knowledge.

Software engineers, infrastructure and platform engineers, architects: When we run workshops on the topics we cover in this book, we often come across software engineers, infrastructure and platform engineers, and architects working in the ML space. While capabilities from the software world (e.g., infrastructure-as-code, deployment automation, automated testing) are necessary in designing and delivering ML solutions in the real world, they are also insufficient. To build reliable ML solutions, we need to widen the software lens and look at other principles and practices—such as ML model tests, dual-track delivery, continuous discovery, and ML governance—to handle challenges that are unique to ML.

Product managers, delivery managers, engineering managers: We set ourselves up for failure if we think that we need only data scientists and ML engineers to build an ML product. In contrast, our experience tells us that teams are most effective when they are cross-functional and equipped with the necessary ML, data, engineering, product, and delivery capabilities.

In this book, we elaborate on how you can apply Lean delivery practices and systems thinking to create structures that help teams to focus on the voice of the customer, shorten feedback loops, experiment rapidly and reliably, and iterate toward building the right thing. As W. Edwards Deming once said, “A bad system will beat a good person every time.” So, we share principles and practices that will help teams create structures that optimize information flow, reduce waste (e.g., handoffs, dependencies), and improve value.

If we’ve done our job right, this book will invite you to look closely at how things have “always been done” in ML and in your teams, to reflect on how well they are working for you, and to consider better alternatives. Read this book with an open mind, and—for the engineering-focused chapters—with an open code editor. As Peter M. Senge said in his book The Fifth Discipline (Doubleday), “Taking in information is only distantly related to real learning. It would be nonsensical to say, ‘I just read a great book about bicycle riding—I’ve now learned that.’” We encourage you to try out the practices in your teams, and we hope you’ll experience firsthand the value that they bring in real-world projects.

Approach this book with a continuous improvement mindset, not a perfectionist mindset. There is no perfect project where everything works perfectly without challenges. There will always be complexity and challenges (and we know a healthy amount of challenge is essential for growth), but the practices in this book will help you minimize accidental complexity so that you can focus on the essential complexity of your ML solutions and on delivering value responsibly.

About the Author

David Tan is a Senior ML Engineer at Thoughtworks. He has worked on multiple data and machine learning projects and applied time-tested software engineering practices to help teams iterate more quickly and reliably in the machine learning development lifecycle.

Ada Leung is a Senior Business Analyst at Thoughtworks. She has technology delivery experience across several industries and her experience includes breaking down complex problems in varying domains, including customer facing applications, scaling of ML solutions, and more recently, data strategy and delivery of data platforms. She has been part of exemplar cross-functional delivery teams, both in-person and remotely, and is an advocate of cultivation as a way to build high performing teams.

David "Dave" Colls is a technology leader with broad experience helping software and data teams deliver great results. David's technical background is in engineering design, simulation, optimization, and large-scale data-processing software. At Thoughtworks, he has led numerous agile and lean transformation projects, and most recently he established the Data and AI practice in Australia. In his practice leadership role, he develops new ML services, consults on ML strategy, and provides leadership to the delivery of ML initiatives.