Natural Language Processing with TensorFlow : Teach Language to Machines Using Python's Deep Learning Library.

Cover -- Copyright -- Packt Upsell -- Contributors -- Table of Contents -- Preface -- Chapter 1: Introduction to Natural Language Processing -- What is Natural Language Processing? -- Tasks of Natural Language Processing -- The traditional approach to Natural Language Processing -- Understanding the traditional approach -- Example - generating football game summaries -- Drawbacks of the traditional approach -- The deep learning approach to Natural Language Processing -- History of deep learning -- The current state of deep learning and NLP -- Understanding a simple deep model - a Fully Connected Neural Network -- The roadmap - beyond this chapter -- Introduction to the technical tools -- Description of the tools -- Installing Python and scikit-learn -- Installing Jupyter Notebook -- Installing TensorFlow -- Summary -- Chapter 2: Understanding TensorFlow -- What is TensorFlow? -- Getting started with TensorFlow -- TensorFlow client in detail -- TensorFlow architecture - what happens when you execute the client? -- Cafe Le TensorFlow - understanding TensorFlow with an analogy -- Inputs, variables, outputs, and operations -- Defining inputs in TensorFlow -- Feeding data with Python code -- Preloading and storing data as tensors -- Building an input pipeline -- Defining variables in TensorFlow -- Defining TensorFlow outputs -- Defining TensorFlow operations -- Comparison operations -- Mathematical operations -- Scatter and gather operations -- Neural network-related operations -- Reusing variables with scoping -- Implementing our first neural network -- Preparing the data -- Defining the TensorFlow graph -- Running the neural network -- Summary -- Chapter 3: Word2vec - Learning Word Embeddings -- What is a word representation or meaning? -- Classical approaches to learning word representation.

WordNet - using an external lexical knowledge base for learning word representations -- Tour of WordNet -- Problems with WordNet -- One-hot encoded representation -- The TF-IDF method -- Co-occurrence matrix -- Word2vec - a neural network-based approach to learning word representation -- Exercise: is queen = king - he + she? -- Designing a loss function for learning word embeddings -- The skip-gram algorithm -- From raw text to structured data -- Learning the word embeddings with a neural network -- Formulating a practical loss function -- Efficiently approximating the loss function -- Implementing skip-gram with TensorFlow -- The Continuous Bag-of-Words algorithm -- Implementing CBOW in TensorFlow -- Summary -- Chapter 4: Advanced Word2vec -- The original skip-gram algorithm -- Implementing the original skip-gram algorithm -- Comparing the original skip-gram with the improved skip-gram -- Comparing skip-gram with CBOW -- Performance comparison -- Which is the winner, skip-gram or CBOW? -- Extensions to the word embeddings algorithms -- Using the unigram distribution for negative sampling -- Implementing unigram-based negative sampling -- Subsampling - probabilistically ignoring the common words -- Implementing subsampling -- Comparing the CBOW and its extensions -- More recent algorithms extending skip-gram and CBOW -- A limitation of the skip-gram algorithm -- The structured skip-gram algorithm -- The loss function -- The continuous window model -- GloVe - Global Vectors representation -- Understanding GloVe -- Implementing GloVe -- Document classification with Word2vec -- Dataset -- Classifying documents with word embeddings -- Implementation - learning word embeddings -- Implementation - word embeddings to document embeddings -- Document clustering and t-SNE visualization of embedded documents -- Inspecting several outliers.

Implementation - clustering/classification of documents with K-means -- Summary -- Chapter 5: Sentence Classification with Convolutional Neural Networks -- Introducing Convolution Neural Networks -- CNN fundamentals -- The power of Convolution Neural Networks -- Understanding Convolution Neural Networks -- Convolution operation -- Standard convolution operation -- Convolving with stride -- Convolving with padding -- Transposed convolution -- Pooling operation -- Max pooling -- Max pooling with stride -- Average pooling -- Fully connected layers -- Putting everything together -- Exercise - image classification on MNIST with CNN -- About the data -- Implementing the CNN -- Analyzing the predictions produced with a CNN -- Using CNNs for sentence classification -- CNN structure -- Data transformation -- The convolution operation -- Pooling over time -- Implementation - sentence classification with CNNs -- Summary -- Chapter 6: Recurrent Neural Networks -- Understanding Recurrent Neural Networks -- The problem with feed-forward neural networks -- Modeling with Recurrent Neural Networks -- Technical description of a Recurrent Neural Network -- Backpropagation Through Time -- How backpropagation works -- Why we cannot use BP directly for RNNs -- Backpropagation Through Time - training RNNs -- Truncated BPTT - training RNNs efficiently -- Limitations of BPTT - vanishing and exploding gradients -- Applications of RNNs -- One-to-one RNNs -- One-to-many RNNs -- Many-to-one RNNs -- Many-to-many RNNs -- Generating text with RNNs -- Defining hyperparameters -- Unrolling the inputs over time for Truncated BPTT -- Defining the validation dataset -- Defining weights and biases -- Defining state persisting variables -- Calculating the hidden states and outputs with unrolled inputs -- Calculating the loss.

Resetting state at the beginning of a new segment of text -- Calculating validation output -- Calculating gradients and optimizing -- Outputting a freshly generated chunk of text -- Evaluating text results output from the RNN -- Perplexity - measuring the quality of the text result -- Recurrent Neural Networks with Context Features - RNNs with longer memory -- Technical description of the RNN-CF -- Implementing the RNN-CF -- Defining the RNN-CF hyperparameters -- Defining input and output placeholders -- Defining weights of the RNN-CF -- Variables and operations for maintaining hidden and context states -- Calculating output -- Calculating the loss -- Calculating validation output -- Computing test output -- Computing the gradients and optimizing -- Text generated with the RNN-CF -- Summary -- Chapter 7: Long Short-Term Memory Networks -- Understanding Long Short-Term Memory Networks -- What is an LSTM? -- LSTMs in more detail -- How LSTMs differ from standard RNNs -- How LSTMs solve the vanishing gradient problem -- Improving LSTMs -- Greedy sampling -- Beam search -- Using word vectors -- Bidirectional LSTMs (BiLSTM) -- Other variants of LSTMs -- Peephole connections -- Gated Recurrent Units -- Summary -- Chapter 8: Applications of LSTM - Generating Text -- Our data -- About the dataset -- Preprocessing data -- Implementing an LSTM -- Defining hyperparameters -- Defining parameters -- Defining an LSTM cell and its operations -- Defining inputs and labels -- Defining sequential calculations required to process sequential data -- Defining the optimizer -- Decaying learning rate over time -- Making predictions -- Calculating perplexity (loss) -- Resetting states -- Greedy sampling to break unimodality -- Generating new text -- Example generated text -- Comparing LSTMs to LSTMs with peephole connections and GRUs -- Standard LSTM -- Review.

Example generated text -- Gated Recurrent Units (GRUs) -- Review -- The code -- Example generated text -- LSTMs with peepholes -- Review -- The code -- Example generated text -- Training and validation perplexities over time -- Improving LSTMs - beam search -- Implementing beam search -- Examples generated with beam search -- Improving LSTMs - generating text with words instead of n-grams -- The curse of dimensionality -- Word2vec to the rescue -- Generating text with Word2vec -- Examples generated with LSTM-Word2vec and beam search -- Perplexity over time -- Using the TensorFlow RNN API -- Summary -- Chapter 9: Applications of LSTM - Image Caption Generation -- Getting to know the data -- ILSVRC ImageNet dataset -- The MS-COCO dataset -- The machine learning pipeline for image caption generation -- Extracting image features with CNNs -- Implementation - loading weights and inferencing with VGG-16 -- Building and updating variables -- Preprocessing inputs -- Inferring VGG-16 -- Extracting vectorized representations of images -- Predicting class probabilities with VGG-16 -- Learning word embeddings -- Preparing captions for feeding into LSTMs -- Generating data for LSTMs -- Defining the LSTM -- Evaluating the results quantitatively -- BLEU -- ROUGE -- METEOR -- CIDEr -- BLEU-4 over time for our model -- Captions generated for test images -- Using TensorFlow RNN API with pretrained GloVe word vectors -- Loading GloVe word vectors -- Cleaning data -- Using pretrained embeddings with TensorFlow RNN API -- Defining the pretrained embedding layer and the adaptation layer -- Defining the LSTM cell and softmax layer -- Defining inputs and outputs -- Processing images and text differently -- Defining the LSTM output calculation -- Defining the logits and predictions -- Defining the sequence loss -- Defining the optimizer -- Summary.

Chapter 10: Sequence-to-Sequence Learning - Neural Machine Translation.

TensorFlow is the leading framework for deep learning algorithms critical to artificial intelligence, and natural language processing (NLP) makes much of the data used by deep learning applications accessible to them. This book brings the two together and teaches deep learning developers how to work with today's vast amount of unstructured data.

Description based on publisher supplied metadata and other sources.

Electronic reproduction. Ann Arbor, Michigan : ProQuest Ebook Central, 2024. Available via World Wide Web. Access may be limited to ProQuest Ebook Central affiliated libraries.