Do your product dashboards look funky? Are your quarterly reports stale? Is the data set you're using broken or just plain wrong? These problems affect almost every team, yet they're usually addressed on an ad hoc basis and in a reactive manner. If you answered yes to these questions, this book is for you.

Many data engineering teams today face the "good pipelines, bad data" problem. It doesn't matter how advanced your data infrastructure is if the data you're piping is bad. In this book, Barr Moses, Lior Gavish, and Molly Vorwerck, from the data observability company Monte Carlo, explain how to tackle data 
quality and trust at scale by leveraging best practices and technologies used by some of the world's most innovative companies.

• 
  •  Build more trustworthy and reliable data pipelines
• •  Write scripts to make data checks and identify broken pipelines with data observability
• •  Learn how to set and maintain data SLAs, SLIs, and SLOs
• •  Develop and lead data quality initiatives at your company
• •  Learn how to treat data services and systems with the diligence of production software
• •  Automate data lineage graphs across your data ecosystem
• •  Build anomaly detectors for your critical data assets
  
  If you’ve experienced any of the following scenarios, raise your hand (or, you can just nod in solidarity):
• •  Five thousand rows in a critical table suddenly turns into five hundred, with no rhyme or reason.
• •  A broken dashboard causes an executive dashboard to spit null values.
• •  A hidden schema change breaks a downstream pipeline.
• •  And the list goes on.
  
  From the Preface

This book is for everyone who has suffered from unreliable data, silently or with muffled screams, and wants to do something about it. We expect that these individuals will come from data engineers, data analytics, or data science backgrounds, and be actively involved in building, scaling, and managing their company’s data pipelines.
 

On the surface, it may seem like Data Quality Fundamentals is a manual about how to clean, wrangle, and generally make sense of data—and it is. But more so, this book tackles best practices, technologies, and processes around building more reliable data systems and, in the process, cultivating data trust with your team and stakeholders.

Editorial Reviews

About the Author

Barr Moses is the CEO and co-founder of Monte Carlo, a data reliability company. In her decade-long career in data, Barr has served as commander of a data intelligence unit in the Israeli Air Force, a consultant at Bain & Company, and VP of Operations at Gainsight, where she built and led their data and analytics team. The instructor of O’Reilly first course on Data Observability, an emerging discipline in data engineering, Barr has worked with hundreds of data teams struggling with these problems. Inspired by her time in the analytics trenches, she is building a product literally dedicated to identifying, resolving, and preventing what she calls “data downtime,” periods of time when data is missing, erroneous, or otherwise inaccurate. In other words: bad data. In this book, she shares her experiences and learnings on how today’s data organizations can achieve high data quality at scale through technological, organization, and cultural best practices.

Lior Gavish is CTO and Co-Founder of Monte Carlo, a data reliability company backed by Accel, Redpoint, GGV, and other top Silicon Valley investors. Prior to Monte Carlo, Lior co-founded cybersecurity startup Sookasa, which was acquired by Barracuda in 2016. At Barracuda, Lior was SVP of Engineering, launching award-winning ML products for fraud prevention. Lior holds an MBA from Stanford and an MSC in Computer Science from Tel-Aviv University.

Molly Vorwerck is the Head of Content at Monte Carlo, a data reliability company. Prior to joining Monte Carlo, Molly served as editor-in-chief of the Uber Engineering Blog and lead program manager for Uber’s Technical Brand team, where she spent countless hours helping engineers, data scientists, and analysts write and edit content about their technical work and experiences. She also led internal communications for Uber’s Chief Technology Officer and strategy for Uber AI’s Research Review Program. In her spare time, she freelances for USA Today, reads up on all the latest trends in data, and volunteers for the California Historical Society.