You know how to program in Python but are interested in what goes on under the covers of the interpreter? Well, fasten your seat-belts as this book will take you on a tour of the virtual machine that runs your Python code.

It will describe how Python code is compiled and run, how the language itself can be modified and will demystify the mysterious bytecodes that run on the Python virtual machine.

Inside the Python Virtual Machine provides a guided tour under the covers of the Python interpreter for the curious pythonista. It attempts to show the user what happens from the moment the user executes a piece of Python code to the point when the interpreter returns the result of executing the piece of code.

This book will provide the readers with an understanding of the various processes that go into compiling and executing a python program removing most of the mystery surrounding how the python interpreter executes source code. The books starts out with a description of the compilation phase with emphasis on the less generic parts of the compilation phase. It then proceeds to discuss python objects and their implementation in CPython. This is followed by a discussion of various objects types that are central to the interpreter such as frame objects and code objects. The process of evaluating code objects by the interpreter loop is also discussed as well as how to extend the Python programming language with your own constructs.

About the Author

Obi has more than five years of experience using Python. His interests include but are not limited to algorithms, mathematics, computer security, writing, developmental economics, business and problem solving in general.

Table of Contents

1. Introduction

2. The View From 30,000ft

3. Compiling Python Source Code

3.1 From Source To Parse Tree

3.2 Python tokens

3.3 From Parse Tree To Abstract Syntax Tree

3.4 Building The Symbol Table

3.5 From AST To Code Objects

4. Python Objects

4.1 PyObject

4.2 Dissecting Types

4.3 Type Object Case Studies

4.4 Minting type instances

4.5 Objects and their attributes

4.6 Method Resolution Order (MRO)

5. Code Objects

5.1 Exploring code objects

5.2 Code Objects within other code objects

5.3 Code Objects in the VM

6. Frame Objects

6.1 Allocating Frame Objects

7. Interpreter and Thread States

7.1 The Interpreter state

7.2 The Thread state

8. Intermezzo: The abstract.c Module

9. The evaluation loop, ceval.c

9.1 Putting names in place

9.2 The parts of the machine

9.3 The Evaluation loop

9.4 A sampling of opcodes

10. The Block Stack

10.1 A Short Note on Exception Handling

11. From Class code to bytecode

12. Generators: Behind the scenes.

12.1 The Generator object

12.2 Running a generator