O'Reilly logo
live online training icon Live Online training

Scaling Python with generators

Writing expressive, highly scalable code

Aaron Maxwell

Whether we like it or not, we’re in the age of big data. But big data is not just a buzzword; from now on, many of the programs you write will need to be able to process more data than ever before. So what's the best way to ensure that your code is scalable and high performing? The answer lies in a rich, powerful, and surprisingly underused feature of Python: the generator function.

In this hands-on three-hour course, expert Pythonista Aaron Maxwell walks you through Python's generator function and teaches you how to use generators to create powerful, highly expressive code. You'll discover how generators can profoundly improve the scalability and performance of all the code you write, from scripts to large applications, and learn how to improve the organization, composability, and expressiveness of your code. Along the way, you'll explore rarely taught generator design patterns that you can put into use immediately.

What you'll learn-and how you can apply it

By the end of this live, online course, you’ll understand:

  • How Python’s mysterious generators really work
  • The key design patterns made possible by generator functions
  • How to create more responsive and scalable Python software

And you’ll be able to:

  • Write code that automatically and gracefully handles increasing magnitudes of data
  • Create interlocking Python components
  • Write Python code that’s concise, readable, and highly maintainable
  • Advise your teammates on powerful Python patterns and crucial best practices

This training course is for you because...

  • You're a web developer using Python frameworks, and you want to expand your knowledge.
  • You’re a QA engineer using Python for scripting and writing tests, and you want to explore Python's generator function in more detail.
  • You’re a data scientist using Python, and you want to get more done, faster.
  • You’re a software engineer aiming to be more productive and write more robust, reliable, and maintainable Python code.
  • You want to improve your Python knowledge to ace an interview or land that dream job.
  • You want to deeply understand the Python language and ecosystem and how to use it to its fullest potential.


  • Complete the Python Labs Prep
  • An intermediate understanding of Python (e.g., the ability to write simple Python programs using lists and dictionaries)
  • Familiarity with basic types (int, str, float, etc.)
  • Experience using modules in the standard library

Materials or downloads needed in advance:

A machine with Python 3 (3.6+) or Python 2.7 installed along with the Python-aware IDE or editor of your choice (e.g., PyCharm)—You will be able to do the programming exercises in either version of Python. Since almost everything in the course applies to both versions, the class will be taught primarily using 3, pointing out where 2.7 is different as we go along. You should be able to open a Python interactive interpreter on the command line and run simple programs ("python3 helloworld.py") either on the command line or in your IDE. On Windows, you will need to set your system path.

Recommended preparation:

Introduction to Python (video)

Intermediate Python: Introduction (video)

Pythonic Object-Oriented Programming (live online training)

Recommended follow-up:

Python Cookbook, 3rd edition (book)

Become Fluent in Python (Learning Path)

Magically Crafting Your Own Python Syntax (Oriole online training)

Fluent Python: The Power of Special Methods (Oriole online training)

Python: The Next Level (live online training)

About your instructor

  • Aaron Maxwell is author of the book "Powerful Python: The Most Impactful Patterns, Features, and Development Strategies Modern Python Provides." As a software engineer, he has worked in devops, test automation, and machine learning, and now divides his time between coding, writing, and teaching


The timeframes are only estimates and may vary according to how the class is progressing

This course will cover:

  • Pythonic scalability overview
  • Generators for efficient, scalable, well-encapsulated code
  • Demystifying Python’s iterator protocol
  • Understanding views, iterators, and iterables
  • Patterns for scalable composability
  • Rich and expressive data structures overview
  • List comprehensions for expressive, readable list creation
  • Comprehensions of dicts, sets, and more
  • Generator comprehensions