O'Reilly logo
live online training icon Live Online training

Practical Deep Learning with PyTorch

A Practical Guide to Produce an End-to-End Deep Learning System

Jeffrey Yau

PyTorch is an open source deep learning platform with a rich ecosystem that enables seamless integration from research prototyping to production deployment. PyTorch is grabbing the attention of deep learning researchers and data science practitioners due to its simplicity of use, accessibility, efficiency, and being more native to Python way of development.

This course will get you up and running with PyTorch, one of the most cutting-edge deep learning libraries. You'll start off with the motivation for using PyTorch, it’s unique features that make it an indispensable deep learning platform, and the fundamental blocks of building deep learning frameworks that power the applications of modern deep learning, such as various dimensional tensors, tensor operations, and tensor operations on GPU.

You will also learn how to build your first neural network using PyTorch and progressively move to other network architectures, such as Convolutional Neural Network (CNN), various forms of Recurrent Neural Network (RNN), and GPU computing to solve real-world problems.

What you'll learn-and how you can apply it

  • Tensor operations
  • The fundamentals of building neural network models
  • PyTorch fundamentals and building blocks for deep learning models development
  • PyTorch ecosystem
  • GPU-accelerated tensor computations with PyTorch
  • Distributed computing in the context of deep learning model development in PyTorch

This training course is for you because...

You are a machine learning engineer, data analyst, data scientists, Python programmer interested in deep learning and are looking to explore implementing deep learning algorithms in one of the most popular and fastest growing open-source deep learning platform, PyTorch.

Prerequisites

  • Working knowledge of Python, especially numpy, pandas, and matplotlib
  • An understanding of the standard densely connected neural network
  • An understanding and working knowledge of the fundamentals of machine learning and popular machine learning algorithms, such as decision tree, random forest, gradient boosting machine, linear regression, logistic regression, and k-nearest neighborhood

Preparation:

  • Students need to have PyTorch installed before the class.
  • The official documentation from PyTorch provides a step-by-step installation process: https://pytorch.org/
  • Installation: https://pytorch.org/get-started/locally/#mac-installation
  • Getting Started: https://pytorch.org/get-started/locally/

Recommended Preparation:

Deep Learning with PyTorch (book)

About your instructor

  • Jeffrey is the Chief Data Scientist at AllianceBernstein, a global investment firm managing over $500 billions. In this role, he is responsible for leading the data science team, partnering with investment professionals to create investment signals, and collaborating with sales and marketing teams to optimize sales. Graduated with a Ph.D. in economics from the University of Pennsylvania, he has also taught statistics, econometrics, and machine learning courses at UC Berkeley, Cornell, NYU, the University of Pennsylvania, and Virginia Tech. Previously, Jeffrey held data science and analytic leadership positions at Silicon Valley Data Science, Charles Schwab Corporation, and KPMG. Jeffrey also is a frequent speaker at leading data science conferences such as ODSC, Spark & AI Summit, and Strata.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

DAY 1

Section 1: An Introduction to PyTorch (80 minutes)

  • Welcome, self-introduction, and an introduction to the training course
  • Motivation: Why PyTorch?
  • Unique features of PyTorch
  • Fundamental building blocks
  • Lab: Using PyTorch for tensor operations, gradient computation, and loading data into PyTorch
  • Lab Solution Discussion
  • Break: 10 minutes

Section 2: Densely-Connected Neural Network for Classification I (80 minutes)

  • Standard Feed-forward neural network architecture
  • Linear activation function
  • Non-linear activation function
  • Forward Propagation
  • Backward Propagation
  • Network weights and bias
  • Feature Scaling
  • Loss function
  • Optimization
  • Training and various forms of Gradient Descent
  • Lab: Building our first neural network model in PyTorch for a classification problem
  • Lab Solution Discussion
  • Break: 10 minutes

Section 3: Practical Aspects of Developing Deep Learning Model (80 minutes)

  • Practical considerations of deep learning model development
  • Recap the key topics covered in Day 1
  • Lab: Tune (or improve) our neural network model developed in the previous section
  • Lab Solution Discussion
  • Discussion of optional (but encouraged) take-home readings to prepare Day 2 + Questions & Answers

DAY 2

Section 4: Convolutional Neural Network (CNN) “convnets” (~80 minutes)

  • An introduction to convets
  • The key concepts of convets
  • What do convnets learn?
  • CNN Architecture and Convolutional layers
  • Feature engineering
  • Visualize the weights of CNN layers
  • Touchvision
  • An introduction to transfer learning
  • Walk-through an example
  • Lab: Building a convolutional neural network model in PyTorch for an image classification problem
  • Lab Solution Discussion
  • Break: 10 minutes

Section 5: Recurrent Neural Network I for sequence data and text (~80 minutes)

  • Tokenization
  • NLTK
  • Vectorization and word embedding
  • Torchtext
  • Building a vocabulary
  • Pre-trained word embedding
  • Recurrent neural network (RNN)
  • Long Short Term memory (LSTM) network
  • An example walk-through
  • Lab: Building a recurrent neural network model in PyTorch for language modeling
  • Lab Solution Discussion
  • Break: 10 minutes

Section 6: Recurrent Neural Network II for numerical time series (~50 minutes)

  • Time series problem formulation
  • Fundamentals of time series analysis
  • Examples of time series analysis using recurrent neural network
  • An example walk-through
  • Lab: Building a recurrent neural network model in PyTorch for time series forecasting
  • Lab solution discussion

Section 7: Distributed Computing in the context of deep learning (30 minutes)

  • Why distributed learning?
  • GPU usage and cloud computing
  • Using distributed computing in deep learning model development using PyTorch
  • An example walk-through

Wrap-up (15 minutes)

  • Summary of what we have learned in the last 2 days
  • Discussion of resources to continue to expand your knowledge and experience in deep learning model development using PyTorch
  • References