O'Reilly logo
live online training icon Live Online training

Deep Learning for Natural Language Processing (NLP): Complete Artificial Intelligence Series

enter image description here

Powerful, Efficient Processing of Natural Language with Deep Neural Networks

Jon Krohn

Relatively obscure a few short years ago, Deep Learning is ubiquitous today across data-driven applications as diverse as machine vision, super-human game-playing, and natural language processing (NLP).

This Live Training builds on the fundamentals of Deep Learning to develop a specialization in handling natural language data and building powerful, efficient, broadly-applicable predictive models that have sequences of words as inputs.

To facilitate an intuitive understanding of NLP and neural-network layers particularly well-suited to processing natural language data (e.g., word vectors, RNNs, GRUs, LSTMs), essential theory will be introduced visually and pragmatically. Theory will immediately be brought to life with interactive demos and hands-on exercises within Jupyter notebooks that feature Python and Keras, the high-level TensorFlow API.

What you'll learn-and how you can apply it

  • Preprocess natural language data and create word vectors for use in machine learning applications
  • Leverage Keras and its TensorFlow backend to make predictions with Deep Learning models trained on natural language
  • Improve Deep Learning model performance by tuning hyperparameters

This training course is for you because...

  • You already have a working understanding of the fundamentals of Deep Learning
  • You want to apply state-of-the-art Deep Learning models to natural language data
  • You want to be able to transform natural language into quantitative representations that can be used as inputs into a broad range of machine learning models

Prerequisites

  • Experience with an object-oriented programming language, e.g., Python (all code demos during the training will be in Python)
  • A working understanding of the fundamentals of Deep Learning would make it a lot easier to follow along with the training

Materials, downloads, or Supplemental Content needed in advance::

  • During class, we’ll work on Jupyter notebooks interactively in the cloud via the Safari JupyterHub platform. A link will be provided at the beginning of class.
  • Alternatively, if you’d like to run this Live Training’s Jupyter notebooks on your own local machine or server, cross-platform step-by-step instructions are available here.

Resources:

About your instructor

  • Jon Krohn is Chief Data Scientist at the machine learning company untapt. He is the presenter of a popular series of tutorials on artificial neural networks, including Deep Learning with TensorFlow, and is the author of Deep Learning Illustrated, the acclaimed book released by Pearson in 2019. Jon holds a doctorate in neuroscience from Oxford University and has been publishing on machine learning in leading academic journals since 2010. He teaches his deep learning curriculum at the NYC Data Science Academy as well as Columbia University. Along with researchers at Columbia’s medical center, Dr. Krohn holds a National Institutes of Health grant to automate medical image processing with deep learning.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

Segment 1: The Power and Elegance of Deep Learning for NLP (45 min)

  • Training Overview
  • Introduction to Deep Learning for Natural Language Processing
  • Easy, Intermediate, and Complex NLP Applications
  • Deep Learning vs Traditional Machine Learning
  • Review of Prerequisite Deep Learning Theory, including Artificial Neurons, Activation Functions, Cost Functions, Gradient Descent, Backpropagation, Weight Initialization, Dense Layers, Convolutional Layers, Max-Pooling, and Dropout
  • Word Vectors: Representing Language as Embeddings
  • Word Vector Arithmetic
  • An Interactive Visualization of Vector-Space Embeddings
  • Vector-Based Representations vs One-Hot Encodings
  • Break + Q&A (5 minutes)

Segment 2: Modeling Natural Language Data (90 min)

  • Hands-on Demo: Best Practices for Preprocessing Natural Language Data for Machine Learning Applications
  • Hands-on Demo: Using word2vec to Create Word Vectors
  • Hands-on Demo: Document Classification with a Dense Neural Network
  • Hands-on Demo: Document Classification with a Convolutional Neural Network
  • Break + Q&A (5 minutes)

Segment 3: Recurrent and Advanced Neural Networks (45 min)

  • Hands-on Demo: Recurrent Neural Networks (RNNs)
  • Hands-on Demo: Long Short-Term Memory Units (LSTMs)
  • Hands-on Demo: Gated Recurrent Units (GRUs)
  • Hands-on Demo: Bi-Directional LSTMs
  • Hands-on Demo: Stacked LSTMs
  • Hands-on Demo: Parallel Network Architectures
  • Break + Q&A (5 minutes)