 1st Edition

# Mathematical Theory of Bayesian Statistics

By

## Sumio Watanabe

ISBN 9781482238068
Published April 23, 2018 by Chapman and Hall/CRC
320 Pages 50 B/W Illustrations

USD \$180.00

Prices & shipping based on shipping country

## Book Description

Mathematical Theory of Bayesian Statistics introduces the mathematical foundation of Bayesian inference which is well-known to be more accurate in many real-world problems than the maximum likelihood method. Recent research has uncovered several mathematical laws in Bayesian statistics, by which both the generalization loss and the marginal likelihood are estimated even if the posterior distribution cannot be approximated by any normal distribution.

Features

• Explains Bayesian inference not subjectively but objectively.
• Provides a mathematical framework for conventional Bayesian theorems.
• Introduces and proves new theorems.
• Cross validation and information criteria of Bayesian statistics are studied from the mathematical point of view.
• Illustrates applications to several statistical problems, for example, model selection, hyperparameter optimization, and hypothesis tests.

This book provides basic introductions for students, researchers, and users of Bayesian statistics, as well as applied mathematicians.

Author

Sumio Watanabe is a professor of Department of Mathematical and Computing Science at Tokyo Institute of Technology. He studies the relationship between algebraic geometry and mathematical statistics.

Definition of Bayesian Statistics

Bayesian Statistics

Probability distribution

True Distribution

Statistical model, prior, and posterior

Examples of Posterior Distributions

Estimation and Generalization

Marginal Likelihood or Partition Function

Conditional Independent Cases

Statistical Models

Normal Distribution

Multinomial Distribution

Linear regression

Neural Network

Finite Normal Mixture

Nonparametric Mixture

Basic Formula of Bayesian Observables

Formal Relation between True and Model

Normalized Observables

Cumulant Generating Functions

Basic Bayesian Theory

Regular Posterior Distribution

Division of Partition Function

Asymptotic Free Energy

Asymptotic Losses

Proof of Asymptotic Expansions

Point Estimators

Standard Posterior Distribution

Standard Form

State Density Function

Asymptotic Free Energy

Renormalized Posterior Distribution

Conditionally Independent Case

General Posterior Distribution

Bayesian Decomposition

Resolution of Singularities

General Asymptotic Theory

Maximum A Posteriori Method

Markov Chain Monte Carlo

Metropolis Method

Basic Metropolis Method

Hamiltonian Monte Carlo

Parallel Tempering

Gibbs Sampler

Gibbs Sampler for Normal Mixture

Nonparametric Bayesian Sampler

Numerical Approximation of Bayesian Observables

Generalization and Cross Validation Losses

Numerical Free Energy

Information Criteria

Model Selection

Criteria for Generalization Loss

Comparison of ISCV with WAIC

Discussion for Model Selection

Hyperparameter Optimization

Criteria for Generalization Loss

Discussion for Hyperparameter Optimization

Topics in Bayesian Statistics

Formal Optimality

Bayesian Hypothesis Test

Bayesian Model Comparison

Phase Transition

Discovery Process

Hierarchical Bayes

Basic Probability Theory

Delta Function

Kullback-Leibler Distance

Probability Space

Empirical Process

Convergence of Expected Values

Mixture by Dirichlet Process

...

## Author(s)

### Biography

Sumio Watanabe is a professor in the Department of Computational Intelligence and Systems Science at Tokyo Institute of Technology, Japan.

## Reviews

"Information criteria are introduced from the two viewpoints, model selection and hyperparameter optimization. In each viewpoint, the properties of the generalization loss and the free energy or the minus log marginal likelihood are investigated. The book is very nicely written with well-defined concepts and contexts. I recommend to all students and researchers." ~Rozsa Horvath-Bokor, Zentralblatt MATH