C2 - Advanced Learning Algorithms (Neural Networks and Decision Trees)

In the second course of the Machine Learning Specialization, you will:

• Build and train a neural network with TensorFlow to perform multi-class classification
• Apply best practices for machine learning development so that your models generalize to data and tasks in the real world
• Build and use decision trees and tree ensemble methods, including random forests and boosted trees

It provides a broad introduction to modern machine learning, including supervised learning (multiple linear regression, logistic regression, neural networks, and decision trees), unsupervised learning (clustering, dimensionality reduction, recommender systems), and some of the best practices used in Silicon Valley for artificial intelligence and machine learning innovation (evaluating and tuning models, taking a data-centric approach to improving performance, and more.)

Week 1 - Neural Networks

This week, you’ll learn about neural networks and how to use them for classification tasks. You’ll use the TensorFlow framework to build a neural network with just a few lines of code. Then, dive deeper by learning how to code up your own neural network in Python, “from scratch”. Optionally, you can learn more about how neural network computations are implemented efficiently using parallel processing (vectorization).

Learning Objectives

• Get familiar with the diagram and components of a neural network
• Understand the concept of a “layer” in a neural network
• Understand how neural networks learn new features.
• Understand how activations are calculated at each layer.
• Learn how a neural network can perform classification on an image.
• Use a framework, TensorFlow, to build a neural network for classification of an image.
• Learn how data goes into and out of a neural network layer in TensorFlow
• Build a neural network in regular Python code (from scratch) to make predictions.
• (Optional): Learn how neural networks use parallel processing (vectorization) to make computations faster.

Week 2 - Neural Networks Training

This week, you’ll learn how to train your model in TensorFlow, and also learn about other important activation functions (besides the sigmoid function), and where to use each type in a neural network. You’ll also learn how to go beyond binary classification to multiclass classification (3 or more categories). Multiclass classification will introduce you to a new activation function and a new loss function. Optionally, you can also learn about the difference between multiclass classification and multi-label classification. You’ll learn about the Adam optimizer, and why it’s an improvement upon regular gradient descent for neural network training. Finally, you will get a brief introduction to other layer types besides the one you’ve seen thus far.

Learning Objectives

• Train a neural network on data using TensorFlow
• Understand the difference between various activation functions (sigmoid, ReLU, and linear)
• Understand which activation functions to use for which type of layer
• Understand why we need non-linear activation functions
• Understand multiclass classification
• Calculate the softmax activation for implementing multiclass classification
• Use the categorical cross entropy loss function for multiclass classification
• Use the recommended method for implementing multiclass classification in code
• (Optional): Explain the difference between multi-label and multiclass classification

Week 3 - Advice for applying machine learning

This week you’ll learn best practices for training and evaluating your learning algorithms to improve performance. This will cover a wide range of useful advice about the machine learning lifecycle, tuning your model, and also improving your training data.

Learning Objectives

• Evaluate and then modify your learning algorithm or data to improve your model’s performance
• Evaluate your learning algorithm using cross validation and test datasets.
• Diagnose bias and variance in your learning algorithm
• Use regularization to adjust bias and variance in your learning algorithm
• Identify a baseline level of performance for your learning algorithm
• Understand how bias and variance apply to neural networks
• Learn about the iterative loop of Machine Learning Development that’s used to update and improve a machine learning model
• Learn to use error analysis to identify the types of errors that a learning algorithm is making
• Learn how to add more training data to improve your model, including data augmentation and data synthesis
• Use transfer learning to improve your model’s performance.
• Learn to include fairness and ethics in your machine learning model development
• Measure precision and recall to work with skewed (imbalanced) datasets

Week 4 - Decision trees

This week, you’ll learn about a practical and very commonly used learning algorithm the decision tree. You’ll also learn about variations of the decision tree, including random forests and boosted trees (XGBoost).

Learning Objectives

• See what a decision tree looks like and how it can be used to make predictions
• Learn how a decision tree learns from training data
• Learn the “impurity” metric “entropy” and how it’s used when building a decision tree
• Learn how to use multiple trees, “tree ensembles” such as random forests and boosted trees
• Learn when to use decision trees or neural networks