O'Reilly logo
live online training icon Live Online training

Deep Reinforcement Learning: Complete Artificial Intelligence Series

State-of-the-Art Approaches for Complex Decision-Making Tasks

Jon Krohn

Relatively obscure a few short years ago, Deep Learning is ubiquitous today across data-driven applications as diverse as machine vision, super-human game-playing, and natural language processing (NLP). This Live Training builds on the fundamentals of Deep Learning to develop a specialization in the advanced topic of Deep Reinforcement Learning.

Deep Reinforcement Learning accounts for the lion’s share of widely publicised “Artificial Intelligence” breakthroughs in recent years. Achievements on this list include AlphaGo defeating world-leading boardgame players, a single algorithm passing human-level performance on a broad range of Atari games, and robots becoming capable of subtle manipulation tasks previously reserved for human hands alone. To facilitate an intuitive understanding of Deep Reinforcement Learning, essential theory will be introduced visually and pragmatically. Theory will immediately be brought to life with interactive demos and hands-on exercises featuring Keras, TensorFlow 2.0, and OpenAI Gym, the leading Reinforcement Learning toolkit.

This is part of Jon Krohn’s Complete Artificial Intelligence Series, a collection of interactive trainings that together comprehensively cover the foundations of modern AI approaches. The recommended progression through the Series is to take one of these two introductory sessions:

  • [Introduction to Deep Learning (with TensorFlow 2.0)](https://learning.oreilly.com/search/?query=%22Introduction%20to%20Deep%20Learning%20(with%20TensorFlow%22&extended_publisher_data=true&highlight=true&include_assessments=false&include_case_studies=true&include_courses=true&include_orioles=true&include_playlists=true&include_collections=true&include_notebooks=true&is_academic_institution_account=false&source=user&sort=relevance&facet_json=true&page=0)
  • Introduction to Deep Learning with PyTorch

Following either of the introductory sessions (or if you’re familiar with the content covered in Chapters 1 and 5-9 of Jon Krohn’s Deep Learning Illustrated book), you’re well-prepared to specialize in any of the other Live Trainings in the Complete Artificial Intelligence Series, which can be undertaken in any order you fancy:

What you'll learn-and how you can apply it

  • Understand the high-level theory and key language around Reinforcement Learning
  • Build a Deep Reinforcement Learning algorithm that gradually becomes adept at playing a video game
  • Appreciate how to apply these principles to other complex applications such as board games, self-driving cars, and maximizing investment returns

This training course is for you because...

  • You already have a working understanding of the fundamentals of Deep Learning
  • You want to architect Deep Reinforcement Learning agents that specialize in a task by learning from their environment
  • You'd like to grasp how Reinforcement Learning approaches have developed superhuman capability on such a broad range of complex tasks

Prerequisites

  • Experience with an object-oriented programming language, e.g., Python (all code demos during the training will be in Python)
  • A working understanding of the fundamentals of Deep Learning would make it a lot easier to follow along with the training

Course Set-up

  • During class, we’ll work on Jupyter notebooks interactively in the cloud via Google Colab. This requires nearly zero setup and instructions will be provided in class. If you’d like to take a sneak peak at the notebooks we’ll be using, check out github.com/jonkrohn/DLTFpT

Resources

About your instructor

  • Jon Krohn is Chief Data Scientist at the machine learning company untapt. He is the presenter of a popular series of tutorials on artificial neural networks, including Deep Learning with TensorFlow, and is the author of Deep Learning Illustrated, the acclaimed book released by Pearson in 2019. Jon holds a doctorate in neuroscience from Oxford University and has been publishing on machine learning in leading academic journals since 2010. He teaches his deep learning curriculum at the NYC Data Science Academy as well as Columbia University. Along with researchers at Columbia’s medical center, Dr. Krohn holds a National Institutes of Health grant to automate medical image processing with deep learning.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

Segment 1: The Foundations of Artificial Intelligence (90 min)

  • Training Overview
  • The Contemporary State of A.I.
  • Applications of Deep Reinforcement Learning
  • The (Cart)Pole Before the Horse: Hands-on Jupyter Demo
  • Review of Prerequisite Deep Learning Theory

Segment 2: Deep Q-Learning Networks (DQNs) (60 min)

  • The Cartpole Game
  • Essential Deep Reinforcement Learning Theory
  • Defining a DQN Agent: Hands-on Jupyter Demo
  • Interacting with an OpenAI Gym Environment: Hands-on Jupyter Demo

Segment 3: Advanced Deep Reinforcement Learning Agents (30 min)

  • SLM-Lab for Agent Experimentation and Optimization
  • Policy Gradients, the REINFORCE Algorithm, and the Actor-Critic Algorithm
  • Software 2.0
  • Approaching Artificial General Intelligence