Free Programming Books

# Pattern Recognition and Machine Learning

Written by

Machine learning

Machine learning is an area of study that deals with the making predictions using algorithms. It analyses data to automates analytical model building. It aims to guesses to be useful.

Pattern recognition

It is a process of recognition of patterns using a Machine Learning algorithm. It may be defined as is the ability to trace arrangements of characteristics or data that produce information for a  given system or data set.it is a ramification of machine learning that enforce  the recognition of data in a particular situation

Course outline

Introduction

Example: Polynomial Curve Fitting

Probability Theory

Probability densities

Expectations and covariances

Bayesian probabilities

The Gaussian distribution

Curve ﬁtting re-visited

Bayesian curve ﬁtting

Model Selection

The Curse of Dimensionality

Decision Theory

Minimizing the misclassiﬁcation rate

Minimizing the expected loss

The reject option and decision

Loss functions for regression

Information Theory

Relative entropy and mutual information

Probability Distributions

Binary Variables

The beta distribution

Multinomial Variables

The Dirichlet distribution

The Gaussian Distribution

Conditional Gaussian distributions

Marginal Gaussian distributions

Bayes’ theorem for Gaussian variables

Maximum likelihood for the Gaussian

Sequential estimation

Bayesian inference for the Gaussian

Student’s t-distribution

Mixtures of Gaussians

The Exponential Family

Maximum likelihood and sufﬁcient statistics

Conjugate priors

Noninformative priors

Linear Models for Regression

Linear Basis Function Models

Maximum likelihood and least squares

Geometry of least squares

Sequential learning

Regularized least squares

Multiple outputs

The Bias-Variance Decomposition

Bayesian Linear Regression

Predictive distribution

Equivalent kernel

Bayesian Model Comparison

Evaluation of the evidence function

­­­­­Maximizing the evidence function

Effective number of parameters

Limitations of Fixed Basis Functions

Linear Models for Classiﬁcation

Discriminant Functions

Two classes

Multiple classes

Least squares for classiﬁcation

Fisher’s linear discriminant

Relation to least squares

Fisher’s discriminant for multiple classes

The Percheron algorithm

Probabilistic Generative Models

Continuous inputs

Maximum likelihood solution

Discrete features

Exponential family

Probabilistic Discriminative Models

Fixed basis functions regression

Iterative re weighted least squares

Multi class logistic regression

Pr obit regression

The Laplace Approximation

Model comparison and BIC

Bayesian Logistic Regression

Laplace approximation

Predictive distribution

Neural Networks

Feed-forward Network Functions

Network Training

Error Back propagation

The Hessian Matrix

Regularization in Neural Networks

Mixture Density Networks

Bayesian Neural Networks

Kernel Methods

Dual Representations

Constructing Kernels

Gaussian Processes

Linear regression revisited

Gaussian processes for regression

Learning the hyper parameters

Automatic relevance determination

Gaussian processes for classiﬁcation

Laplace approximation

Connection to neural networks

Sparse Kernel Machines

Maximum Margin Classiﬁers

Overlapping class distributions

Relation to logistic regression

Multi class SVMs

SVMs for regression

Computational learning theory

Relevance Vector Machines

RVM for regression

Analysis of sparsity

Graphical Models

Bayesian Networks

Conditional Independence

Markov Random Fields

Inference in Graphical Models

Mixture Models and EM

K-means Clustering

Image segmentation and compression

Mixtures of Gaussians

Maximum likelihood

EM for Gaussian mixtures

An Alternative View of EM

Gaussian mixtures revisited

Relation to K-means

Mixtures of Bernoulli distributions

EM for Bayesian linear regression

The EM Algorithm in General