Section 1: Classical Image Classification Methods

Presentation Transcript

Section 1: Classical Image Classification Methods Kai Yu Dept. of Media Analytics NEC Laboratories America Andrew Ng Computer Science Dept. Stanford University

Outline of Part 2 Local Features, Sampling, Visual Words Discriminative Methods Bag-of-Words (BoW) portrayal Spatial pyramid coordinating (SPM) Generative Methods P workmanship based strategies Topic models

Outline of Part 2 Local Features, Sampling, Visual Words Discriminative Methods Bag-of-Words (BoW) portrayal Spatial pyramid coordinating (SPM) Generative Methods P craftsmanship based techniques Topic models

Local components Distinctive descriptors of nearby picture patches Invariant to neighborhood interpretation, scale, … and in some cases revolution or general relative changes The most celebrated decision is the SIFT highlight

Sampling nearby elements from pictures An arrangement of focuses Image credits: F-F. Li, E. Nowak, J. Sivic

Visual words Similar focuses are assembled into one visual word Algorithms: k-implies, agglomerative bunching, … Points from various pictures are then more effectively thought about. Slide credit: Kristen Grauman

Outline of Part 2 Local Features, Sampling, Visual Words, … Discriminative Methods Bag-of-Words (BoW) portrayal Spatial pyramid coordinating (SPM) Generative Methods P workmanship based strategies Topic models

Bag-of-words (BoW) portrayal Analogy to records Adapted from instructional exercise slides by Fei-Fei et al.

BoW for protest order Works entirely well for entire picture grouping Csurka et al. (2004), Willamowski et al. (2005), Grauman & Darrell (2005), Sivic et al. (2003, 2005) Slide credit: Svetlana Lazebnik

Unsupervised Dictionary Learning SIFT space R1 R2 R3 picture database Sample nearby components from pictures Run k-mean or other bunching calculation to get lexicon Dictionary is likewise called "codebook"

Compute BoW histogram for each picture R1 R2 Assign filter highlights into groups R3 Compute the recurrence of each bunch inside a picture BoW histogram portrayals

Indication of BoW histogram Summarize whole picture in view of its circulation of visual word events Turn packs of various sizes into a settled length vector Analogous to sack of words portrayal generally utilized for content classification.

Image characterization in view of BoW histogram BoW histogram vector space fowl Decision limit puppy Learn a grouping model to decide the choice limit Nonlinear SVMs are generally connected.

Issues Sampling methodology Learning codebook: measure? directed?, … Classification: which technique? adaptability? Adaptability: how to deal with a large number of information? How to utilize spatial data?

Spatial data The BoW expels spatial design. This builds the invariance to scale, interpretation, and twisting, B ut penances discriminative power, particularly when the spatial format is essential . Slide adjusted from Bill Freeman

Spatial pyramid coordinating Compute BoW for picture districts at various areas in different scales Figure credit: Svetlana Lazebnik

A typical pipeline for discriminative picture grouping utilizing BoW Dictionary Learning Image Classification Dense/Sparse SIFT VQ Coding Dense/Sparse SIFT Spatial Pyramid Pooling K-implies lexicon Nonlinear SVM

Combining numerous descriptors Multiple Feature Detectors Multiple Descriptors: SIFT, shape, shading, … VQ Coding and Spatial Pooling Nonlinear SVM Diagram from SurreyUVA_SRKDA, champ group in PASCAL VOC 2008

Outline of Part 2 Local Features, Sampling, Visual Words, … Discriminative Methods Bag-of-Words (BoW) portrayal Spatial pyramid coordinating (SPM) Generative Methods P workmanship based strategies Topic models

"shoreline"  z c w N D Topic models for pictures Latent Dirichlet Allocation (LDA) Fei-Fei et al. ICCV 2005 Slide credit Fei-Fei Li

Part-based Model Rob Fergus ICCV09 Tutorial Fischler & Elschlager 1973

For an extensive scope of question classification models , please visit Recognizing and Learning Object Categories Li Fei-Fei (Stanford), Rob Fergus (NYU), Antonio Torralba (MIT) http://people.csail.mit.edu/torralba/shortCourseRLOC/