With the shift from data warehouses to data lakes, data now lands in repositories before it's been transformed, enabling engineers to model raw data into clean, well-defined datasets. dbt (data build tool) helps you take data further. This practical book shows data analysts, data engineers, BI developers, and data scientists how to create a true self-service transformation platform through the use of dynamic SQL.

Authors Rui Machado from Monstarlab and Hélder Russa from Jumia show you how to quickly deliver new data products by focusing more on value delivery and less on architectural and engineering aspects. If you know your business well and have the technical skills to model raw data into clean, well-defined datasets, you'll learn how to design and deliver data models without any technical influence.

With this book, you'll learn:

• What dbt is and how a dbt project is structured
• How dbt fits into the data engineering and analytics worlds
• How to collaborate on building data models
• The main tools and architectures for building useful, functional data models
• How to fit dbt into data warehousing and laking architecture
• How to build tests for data transformations

Table of Contents

Chapter 1. Analytics Engineering

Chapter 2. Data Modeling for Analytics

Chapter 3. SQL for Analytics

Chapter 4. Data Transformation with dbt

Chapter 5. dbt Advanced Topics

Chapter 6. Building an End-to-End Analytics Engineering Use Case

Who This Book Is For

This book is designed for professionals, students, and enthusiasts dealing with the complex world of data management and analytics. Whether you’re an experienced veteran reminiscing about the basic principles of data modeling or an aspiring analyst keen to understand the transformation from business intelligence to contemporary analytics engineering, our storytelling assures clearness and direction.

Organizations seeking to strengthen their data processes will discover immense value in the combination of well-proven principles and modern tools discussed in this book. In summary, if you wish to take full advantage of your data by combining the strengths of the past with the innovations of the present, this book will guide you.

Why We Wrote This Book

In today’s era of abundant information, it is not uncommon for vital knowledge, concepts, and techniques to become obscured amid the rapid growth of technology and the relentless pursuit of innovation. During this dynamic transformation, several essential concepts can sometimes be inadvertently overlooked. This oversight doesn’t stem from their diminishing relevance but rather from the swift pace of progress.

One such fundamental concept that often falls by the wayside is data modeling in the context of data management. It’s worth noting that data modeling encompasses various approaches, including Kimball, conceptual, logical, and physical modeling, among others. We recognize the pressing need to emphasize the significance of data modeling in this diverse landscape, and that’s one of the key reasons we’ve crafted this book. Within these pages, we aim to shed light on the intricacies and various dimensions of data modeling and how it underpins the broader field of analytics engineering.

Over time, the importance of data modeling in guaranteeing a solid data management system has gradually faded from general awareness. This is not because it became outdated but rather due to a shift in the industry’s focus. New words, tools, and methods have emerged, making the fundamental principles less important. A transition occurred from traditional practices to modern solutions that promised quickness and efficiency, sometimes resulting in a loss of foundational strength.

The rise of analytics engineering led to a resurgence. It was not just a trend filled with fancy words but also a return to the basics, echoing the principles of the business intelligence sector. The difference is that modern tools, infrastructure, and techniques are now available to implement these principles more efficiently.

So, why did we feel the need to document our thoughts? There are two primary reasons. First and foremost, it is crucial to underscore the enduring value and significance of well-established concepts like data modeling. While these methodologies may have been around for a while, they provide a robust foundation for the development of modern techniques. Our second intention is to emphasize that analytics engineering is not a standalone entity but rather a natural progression from the legacy of business intelligence. By integrating the two, organizations can construct a more resilient data value chain, ensuring that their data is not just extensive but also actionable, ultimately enhancing its utility.

This book is not just a sentimental trip down memory lane or a commentary on the present. It’s a blueprint for the future. Our goal is to help organizations revisit their foundations, appreciate the advantages of old and new technologies, and integrate them for a comprehensive data management approach. We’ll dig deeper into data modeling and transformation details, explain its importance, and examine how it interacts with modern analytics engineering tools. We aim to provide our readers with a complete understanding, enabling them to strengthen their data management processes and utilize the full potential of their data.

About the Author

Rui Machado is a Director of Data Engineering at Monstarlab and has a background in Information Technologies and Data Science. Has over a decade of relevant experience in the architecture and implementation of data warehouses, data lakes, and decision support systems in industries such as Retail, Ecommerce, Supply Chain, Healthcare, and Social Networks. Has led Engineering and Analytics teams at Jumia, Nike, and Facebook. He is also co-founder and CEO of ShopAI.co. He has previously collaborated with Synfusion in publishing three technical books on Powershell, SSIS, and BizTalk Server.

HR. Helder Russa is a Data Engineering Lead at Jumia with a background in Information Technologies and Data Science. Has over 10 years of professional experience in computer science, with an emphasis on evolving and maintaining data solutions applied to decision making. Nowadays, he works as a lead data engineer at Jumia where he contributes to the strategy definition, design, and implementation of multiple Jumia data platforms. In similitude, and since 2018, he is a co-founder and data architect of ShopAI, a company specialized in deep learning, that leverages the capabilities of the image for optimization of search channels inside webshops. LinkedIn profile: https://www.linkedin.com/in/hrussa/