MNIST Series

MNIST

• Image of handwritten digit
• 60000 training examples
• 10000 test examples
• 28*28 grayscale images
• 10 classes

ICCV 2017
@vt.edu @gatech.edu

Contents

• slide
• pdf

Introduction

• Input: Images
• Output: Visualization of discrimination regions
• Interpretability
• Without architectural changes or re-training
• To give explaination to the prediction of some models
• General version of CAM

CVPR 2016
@csail.mit.edu

Contents

pdf

Introduction

Problem

In some CNN models, the objects localization ability is lost when fully-connected layers are used for classification

Goal

To both classify the image and localize class-specific image regions in a single forward-pass

NIPS 2016
@google.com

Contents

pdf

Introduction

• Generate pictures pixel by pixel
• Related Works
  • PixelRNN: better performance
  • PixelCNN: faster to train (easier to parallelize)
• Gated PixelCNN
• Conditional variant of the Gated PixelCNN

NIPS 2017
@google.com

Contents

pdf

Introduction

• Recurrent Models
  • sequence operations
  • hard to do parallelization
• the Transformer
  • no recurrence
  • relying entirely on an attention mechanism
  • global dependencies
  • parallelization with 8 GPUs

EMNLP 2015
@stanford.edu

Contents

pdf

Introduction

• Attentional mechanism jointly translate and align words
• 2 types of attention-based models
  • global
    • all source words
  • local
    • subset of source words

ICLR 2014
@google.com

Contents

pdf

Introduction

• Variational Bayesian (VB)
• Approximate posterior using MLP
• Stochatic Gradient Variational Bayes (SGVB)
• Auto-Encoding VB (AEVB) algorithm
• Variational auto-encoder (VAE)

Method

problem

• the integral of the marginal likelihood $p_\theta(x) = \int p_\theta(z)p_\theta(x|z) dz$ is intractable
• a large dataset: the need of updating using small minibatches

NIPS 2014
@google.com

Contents

pdf

Abstract

• Task: an English to French translation task from the WMT-14 dataset
• Method:
  • a Deep LSTM: maps theinput sequence to a vector of a fixed dimensionality
  • another Deep LSTM: decodes the target sequence from the vector
• Result: BLEU score 34.8
• Additional Founds: reversing the order of the words in all source sentences (but not target sentences) improved the LSTM’s performance markedly

CVPR 2016
@microsoft.com

Contents

pdf

Abstract

• residual networks
• VGG nets
• COCO object detection dataset