A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
A Generalized Mean Field Algorithm for Variational Inference in Exponential Families
[article]
2012
arXiv
pre-print
Preserved Fulltext
We present a class of generalized mean field (GMF) algorithms for approximate inference in complex exponential family models, which entails limiting the optimization over the class of cluster-factorizable ...
In this paper, we discuss a generalized mean field theory on variational approximation to a broad class of intractable distributions using a rich set of tractable distributions via constrained optimization ...
Acknowledgments We thank Yair Weiss and colleagues for their generos ity in sharing their code for exact inference and GBP on grids. ...
arXiv:1212.2512v1
fatcat:hbdgnfrza5d5dkgle3op4a2kiy
Variational Inference: A Review for Statisticians
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive ...
In this paper, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization. ...
Variational inference with exponential families We described mean-field variational inference and derived CAVI, a general coordinate-ascent algorithm for optimizing the ELBO. ...
arXiv:1601.00670v8
fatcat:opqknuo6t5cezfvluwhmu4cg7e
Composing graphical models with neural networks for structured representations and fast inference
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
All components of these models are learned simultaneously with a single objective, giving a scalable algorithm that leverages stochastic variational inference, natural gradients, graphical model message ...
Our model family augments graphical structure in latent variables with neural network observation models. ...
The SVAE algorithm computes stochastic gradients of a mean field variational inference objective. ...
arXiv:1603.06277v5
fatcat:kkackdtwkrcenhj36csyvorlwq
Hierarchical Variational Models
[article]
2016
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Black box variational inference allows researchers to easily prototype and evaluate an array of models. Recent advances allow such algorithms to scale to high dimensions. ...
HVMs augment a variational approximation with a prior on its parameters, which allows it to capture complex structure for both discrete and continuous latent variables. ...
We demonstrate HVMs with a study of approximate posteriors for several variants of deep exponential families (Ranganath et al., 2015) ; HVMs generally outperform mean-field variational inference. ...
arXiv:1511.02386v2
fatcat:cyu73a35fbcdlbo57pynmxkrxe
Variational Bayes on Monte Carlo Steroids
2016
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We demonstrate empirical improvements on benchmark datasets in vision and language for sigmoid belief networks, where a neural network is used to approximate the posterior. ...
In contrast to standard variational methods, our bounds are guaranteed to be tight with high probability. ...
using a mean-field approximation. ...
dblp:conf/nips/GroverE16
fatcat:hp2rmlxuira3pbiurpggpiyldi
7 A Variational Principle for Graphical Models
[chapter]
2006
New Directions in Statistical Signal Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
Indeed, graphical models provide a natural framework for formulating variations on these classical architectures, and for exploring entirely new families of statistical models. ...
For suitably sparse graphs, the junction tree algorithm provides a systematic and practical solution to the general problem of computing likelihoods and other statistical quantities associated with a graphical ...
This observation suggests a general class of variational EM algorithms, in which the approximation provided by a variational inference algorithm is substituted for the mean parameters in the E step. ...
doi:10.7551/mitpress/4977.003.0009
fatcat:q3avgd2tsfbhno7hx7ckyuuevq
A Review of Inference Algorithms for Hybrid Bayesian Networks
2018
The Journal of Artificial Intelligence Research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
In this paper we provide an overview of the main trends and principled approaches for performing inference in hybrid Bayesian networks. ...
However, this extra feature also comes at a cost: inference in these types of models is computationally more challenging and the underlying models and updating procedures may not even support closed-form ...
AMIDST has received funding from the European Union's Seventh Framework Programme for research, technological development and demonstration under grant agreement no 619209. ...
doi:10.1613/jair.1.11228
fatcat:vhmuf44ftbg73mjygiyrwrdw44
Variational Inference in Nonconjugate Models
[article]
2013
arXiv
pre-print
Preserved Fulltext
The Internet Archive has a preservation copy of this work in our general collections.
The file type is application/pdf.
Other Versions
Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. ...
In this paper, we develop two generic methods for nonconjugate models, Laplace variational inference and delta method variational inference. ...
This technique applies to a subset of models described in this paper. 1 In this paper we develop generic approaches to mean-field variational inference for a large class of nonconjugate models. ...
arXiv:1209.4360v4
fatcat:6lgnxdwa2nbhvghzt5agdh5phe
Stochastic Variational Inference
[article]
2013
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. ...
Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. ...
distribution p(β k |x, z, β −k ) is in a tractable exponential family: We will assign each β k an independent variational distribution so that q(z, β) = ( n,j q(z n,j )) k q(β k ). ...
arXiv:1206.7051v3
fatcat:hsdtdfvs6ne2dnhefqokmtfua4
Variational Inference over Combinatorial Spaces
2010
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Since the discovery of sophisticated fully polynomial randomized algorithms for a range of #P problems [1, 2, 3] , theoretical work on approximate inference in combinatorial spaces has focused on Markov ...
Simulations on a range of matching models show that the algorithm is more general and empirically faster than a popular fully polynomial randomized algorithm. ...
[9, 10] for computing the permanent of a matrix-we are not aware of a general treatment of variational inference in combinatorial spaces. ...
dblp:conf/nips/Bouchard-CoteJ10
fatcat:hl7jp7pt5vdv3h22gjjmlbahke
Variational inference for Dirichlet process mixtures
2006
Bayesian Analysis
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
In this paper, we present a variational inference algorithm for DP mixtures. ...
Thus far, variational methods have mainly been explored in the parametric setting, in particular within the formalism of the exponential family (Attias, 2000; Ghahramani and Beal, 2001; Blei et al., 2003 ...
We now use the general expression in Equation (21) to derive a mean-field coordinate ascent algorithm. ...
doi:10.1214/06-ba104
fatcat:u3utwh7t3bdttl6nksk4g3km5m
Bayesian Nonparametric Matrix Factorization for Recorded Music
2010
International Conference on Machine Learning
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We derive a mean-field variational inference algorithm and evaluate GaP-NMF on both synthetic data and recorded music. ...
These methods require that the number of sources be specified in advance, which is not always possible. ...
Acknowledgments We thank the reviewers for their helpful observations and suggestions. David M. Blei is supported by ONR 175-6343, NSF CAREER 0745520. ...
dblp:conf/icml/HoffmanBC10
fatcat:mzdldcwfkjavfigtnjsjbg4buu
Optimization of Structured Mean Field Objectives
[article]
2012
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
If not, optimization is harder, but we show a new algorithm based on the construction of an auxiliary exponential family that can be used to make inference possible in this case as well. ...
In intractable, undirected graphical models, an intuitive way of creating structured mean field approximations is to select an acyclic tractable subgraph. ...
We also presented a novel algorithm for computing the gradient and bound on the log-partition function in the b-acyclic case. ...
arXiv:1205.2658v1
fatcat:yf4ck67i7fc3tjrwmab4vt73c4
Embarrassingly Parallel Variational Inference in Nonconjugate Models
[article]
2015
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating ...
This type of "embarrassingly parallel" procedure has recently been developed for MCMC inference algorithms; however, in many cases it is not possible to directly extend this procedure to VI methods without ...
Recently, progress has been made toward this goal for mean field variational inference methods limited to models with certain exponential family restrictions on the likelihood and prior distribution [ ...
arXiv:1510.04163v1
fatcat:ylpi75e2zbgx3eqxssjnegtbwm
Patterns of Scalable Bayesian Inference
[article]
2016
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
As a result, there is a zoo of ideas with few clear overarching principles. In this paper, we seek to identify unifying principles, patterns, and intuitions for scaling Bayesian inference. ...
Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. ...
E.A. is supported by the Miller Institute for Basic Research in Science, University of California, Berkeley. M.J. is supported by a fellowship from the Harvard/MIT Joint Grants program. ...
arXiv:1602.05221v2
fatcat:ksgpb3vhgbdszbzna5fgyelske
« Previous
Showing results 1 — 15 out of 41,691 results