Title: Matching randomly generated proposals to kernels in independence
Metropolis chains.

Authors: Maarten Marsman, Gunter Maris, Timo Bechger and Cees Glas

Affiliation: University of Amsterdam

Abstract: A common problem in statistical inference is sampling from not
one, but many posterior distributions (say n).  This is most convenient
when the data are modelled as conditionally independent given the random
effects, since then the posterior distributions of the random effects given
the data are also conditionally independent.  This makes the problem of
sampling from the posterior distributions of the random effects ideal for
parallel processing.
 
We consider the situation where it is not possible to produce a direct
sample from the posterior distribution of any of the n random effects, but
where it is easy to generate data from the model.  That is, if we consider
the factorization of the joint distribution of the parameter and the data
then we assume that we are able to generate i.i.d. from both the marginal
distribution of the random effects and the conditional distribution of the
data.  From the composite sample drawn from these two distributions, we
obtain a draw from a random posterior distribution (of a random effect).
This is not necessarily the posterior distribution that we need a sample
from.
 
An independence Metropolis-Hastings (iMH) algorithm is used to sample from
the posterior distributions of the random effects, since its proposal
distribution does not depend on the current state of the parameter, which
will become important.  Choosing the iMH algorithm raises the issue of
formulating an efficient proposal distribution for each of the n random
effects, a problem that has not been solved in general.  In this
presentation, a general procedure is presented that automatically generates
more efficient proposal distributions for each of the random effects'
posterior distributions, as the number of effects (n) increase (i.e. the
procedure scales).  Another convenient feature is that the algorithm allows
for parallel processing due to the structure of sampling problem (posterior
distribution factors) and the use of the iMH algorithm.
 
We illustrate our approach with sampling from the posterior ability
distributions in the context of item response theory models, of which we
usually have many.