O'Reilly logo
live online training icon Live Online training

Deep Reinforcement Learning

enter image description here

Superhuman Performance on Complex Problems with Deep Learning

Jon Krohn

Relatively obscure a few short years ago, Deep Learning is ubiquitous today across data-driven applications as diverse as machine vision, super-human game-playing, and natural language processing (NLP). This Live Training builds on the fundamentals of Deep Learning to develop a specialization in the advanced topic of Deep Reinforcement Learning.

Deep Reinforcement Learning accounts for the lion’s share of widely publicised “Artificial Intelligence” breakthroughs in recent years. Achievements on this list include AlphaGo defeating world-leading boardgame players, a single algorithm passing human-level performance on a broad range of Atari games, and robots becoming capable of subtle manipulation tasks previously reserved for human hands alone.

To facilitate an intuitive understanding of Deep Reinforcement Learning, essential theory will be introduced visually and pragmatically. Theory will immediately be brought to life with interactive demos and hands-on exercises featuring Keras, the wildly-popular Python API for TensorFlow, and OpenAI Gym, the leading Reinforcement Learning toolkit.

What you'll learn-and how you can apply it

  • Understand the high-level theory and key language around Reinforcement Learning
  • Build a Deep Reinforcement Learning algorithm that gradually becomes adept at playing a video game
  • Appreciate how to apply these principles to other complex applications such as board games, self-driving cars, and inventory management

This training course is for you because...

  • You already have a working understanding of the fundamentals of Deep Learning
  • You want to architect Deep Reinforcement Learning agents that specialize in a task by learning from their environment
  • You'd like to grasp how Reinforcement Learning approaches have developed superhuman capability on such a broad range of complex tasks


  • Experience with an object-oriented programming language, e.g., Python (all code demos during the training will be in Python)
  • A working understanding of the fundamentals of Deep Learning would make it a lot easier to follow along with the training

Course Set-up

  • If you’d like to work along with this Live Training’s Jupyter notebooks interactively, a link will be provided to you at the beginning of class.

  • Alternatively, if you’d simply likely to view the notebooks as we work through them, they’re available online here

Video Resources

About your instructor

  • Jon Krohn is Chief Data Scientist at the machine learning company untapt. He is the presenter of a popular series of tutorials on artificial neural networks, including Deep Learning with TensorFlow, and is the author of Deep Learning Illustrated, the acclaimed book released by Pearson in 2019. Jon holds a doctorate in neuroscience from Oxford University and has been publishing on machine learning in leading academic journals since 2010. He teaches his deep learning curriculum at the NYC Data Science Academy as well as Columbia University. Along with researchers at Columbia’s medical center, Dr. Krohn holds a National Institutes of Health grant to automate medical image processing with deep learning.


The timeframes are only estimates and may vary according to how the class is progressing

Segment 1: The Foundations of Artificial Intelligence (60 minutes)

Training Overview (5 minutes)

The Contemporary State of A.I. (20 minutes)

Break + Q&A (5 minutes)

Applications of Deep Reinforcement Learning (15 minutes)

Review of Prerequisite Deep Learning Theory (10 minutes)

Break + Q&A (5 minutes)

Segment 2: Deep Q-Learning Networks (DQNs)

Length (60 minutes)

The Cartpole Game (10 minutes)

Essential Deep Reinforcement Learning Theory (15 minutes)

Break + Q&A (5 minutes)

Defining a DQN Agent (15 minutes)

Interacting with an OpenAI Gym Environment (10 minutes)

Break + Q&A (5 minutes)

Segment 3: Advanced Deep Reinforcement Learning Agents (60 minutes)

OpenAI Lab for Agent Experimentation and Optimization (25 minutes)

Break + Q&A (5 minutes)

Policy Gradients, the REINFORCE Algorithm, and the Actor-Critic Algorithm (10 minutes)

Software 2.0 (5 minutes)

Approaching Artificial General Intelligence (10 minutes)

Break + Q&A (5 minutes)