A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
A Neural Network MCMC Sampler That Maximizes Proposal Entropy
2021
Entropy
Preserved Fulltext
To optimize proposal entropy directly, we devised a neural network MCMC sampler that has a flexible and tractable proposal distribution. ...
Here we propose to maximize proposal entropy for adapting the proposal to distributions of any shape. ...
Here, we employed the entropy-based objective in a neural network MCMC sampler for optimizing exploration speed. ...
doi:10.3390/e23030269
pmid:33668743
pmcid:PMC7996279
fatcat:onabxdjgbzanheax43usnqmsjq
Sequential Likelihood-Free Inference with Neural Proposal
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This paper introduces a new sampling approach, called Neural Proposal (NP), of the simulation input that resolves the biased data collection as it guarantees the i.i.d. sampling. ...
As the likelihood evaluation is inaccessible, previous papers train the amortized neural network to estimate the ground-truth posterior for the simulation of interest. ...
Instead, Neural Proposal is a neural sampler that replaces the unnormalized proposal distribution, and we select the next batch of simulation inputs from this neural sampler. ...
arXiv:2010.07604v3
fatcat:yhg6tf4sgjahvdhne3gny4rlbe
Learning to Draw Samples with Amortized Stein Variational Gradient Descent
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference. ...
Our method is based on iteratively adjusting the neural network parameters so that the output changes along a Stein variational gradient direction (Liu & Wang, 2016) that maximally decreases the KL divergence ...
CONCLUSION We propose a new method to train neural samplers for given distributions, together with various applications to learning to draw samples using neural samplers. ...
arXiv:1707.06626v2
fatcat:czlv76pwdfdwddhvwwvwanv334
Maximum Entropy Generators for Energy-Based Models
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2019 pre-print arXiv:1901.08508v1 fatcat:cue2pmou7rcgrmyicio27l2usu
In this work, we propose learning both the energy function and an amortized approximate sampling mechanism using a neural generator network, which provides an efficient approximation of the log-likelihood ...
The resulting objective requires maximizing entropy of the generated samples, which we perform using recently proposed nonparametric mutual information estimators. ...
We also thank NVIDIA for donating a DGX-1 computer used for certain experiments in this work. ...
arXiv:1901.08508v2
fatcat:llwr2536rnexrffys2ji3jn4nu
Approximate Inference with Amortised MCMC
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler. ...
produced by warping a source of randomness through a deep neural network. ...
Bayesian neural network classification Next we apply amortised MCMC to classification using Bayesian neural networks. ...
arXiv:1702.08343v2
fatcat:t7igg5ix7bdgljvz7i6s6iwov4
Variationally Inferred Sampling through a Refined Bound
2021
Entropy
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
In this work, a framework to boost the efficiency of Bayesian inference in probabilistic models is introduced by embedding a Markov chain sampler within a variational posterior approximation. ...
Its strengths are its ease of implementation and the automatic tuning of sampler parameters, leading to a faster mixing time through automatic differentiation. ...
First, in a refinement phase, the sampler parameters are learned in an optimization loop that maximizes the ELBO with the new posterior. ...
doi:10.3390/e23010123
pmid:33477766
pmcid:PMC7832329
fatcat:3g4tc6zxabdgjctqfl4wgjs47m
Towards quantum gravity with neural networks: Solving the quantum Hamilton constraint of U(1) BF theory
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2024; you can also visit the original URL.
The file type is application/pdf.
We show that the Neural Network Quantum State (NNQS) ansatz can be used to numerically solve the constraints efficiently and accurately. ...
To make the problem amenable for numerical simulation we fix a graph and introduce a cutoff on the kinematical degrees of freedom, effectively considering U_q(1) BF theory at a root of unity. ...
Therefore, the Metropolis type samplers struggle to propose different proposals which will not be rejected by the MCMC. ...
arXiv:2402.10622v1
fatcat:7swj5lqkl5fqhcsti5nxoblvee
Challenges in Markov chain Monte Carlo for Bayesian neural networks
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Nevertheless, this paper shows that a non-converged Markov chain, generated via MCMC sampling from the parameter space of a neural network, can yield via Bayesian marginalization a valuable posterior predictive ...
Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). ...
Lee (2003) proposes a restricted flat prior for feedforward neural networks by bounding some of the parameters and by imposing constraints that guarantee layer-wise linear independence between activations ...
arXiv:1910.06539v6
fatcat:a7yyjtpsxvcd5okxwclt5gm3xe
Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
A framework to boost the efficiency of Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation. ...
Its strength lies both in ease of implementation and automatically tuning of the sampler parameters to speed up mixing time using automatic differentiation. ...
The initial variational distribution q 0,φ (z|x) is a Gaussian parameterized by a deep neural network (NN). Then, T iterations of a sampler Q parameterized by η are applied leading to q φ,η . ...
arXiv:1908.09744v4
fatcat:wsfd2feowbe2hhv6ybecyyli2m
Meta-Learning for Stochastic Gradient MCMC
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Experiments validate the proposed approach on both Bayesian fully connected neural network and Bayesian recurrent neural network tasks, showing that the learned sampler out-performs generic, hand-designed ...
This paper presents the first meta-learning algorithm that allows automated design for the underlying continuous dynamics of an SG-MCMC sampler. ...
Once trained, the sampler can generalize to different datasets and architectures. • Extensive evaluation of the proposed sampler on Bayesian fully connected neural networks and Bayesian recurrent neural ...
arXiv:1806.04522v1
fatcat:b3zfzd3kzjezvlfmykfd2kfvdy
Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We demonstrate SNEP and the posterior server on distributed Bayesian learning of logistic regression and neural networks. ...
SNEP is a black box variational algorithm, in that it does not require any simplifying assumptions on the distribution of interest, beyond the existence of some Monte Carlo sampler for estimating the moments ...
It is generally accepted that, in high-dimensional settings, MCMC samplers often have lower variance than naive Monte Carlo and as a result work better, with the tradeoff being that MCMC samplers need ...
arXiv:1512.09327v4
fatcat:mt4d7wujqbcztpb5rp7zngfzs4
Exponential Family Estimation via Adversarial Dynamics Embedding
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks. ...
To represent this sampler, we introduce a novel neural architecture, dynamics embedding, that generalizes Hamiltonian Monte-Carlo (HMC). ...
., 2019) , where the sampler is parametrized via a neural network and learned through certain objectives. ...
arXiv:1904.12083v3
fatcat:ksk36lkbf5ftzirf7j4yet3rra
Generative Modeling by Inclusive Neural Random Fields with Applications in Image Generation and Anomaly Detection
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Neural random fields (NRFs), referring to a class of generative models that use neural networks to implement potential functions in random fields (a.k.a. energy-based models), are not new but receive less ...
In this paper we propose a new approach, the inclusive-NRF approach, to learning NRFs for continuous data (e.g. images), by introducing inclusive-divergence minimized auxiliary generators and developing ...
However, 5 Minimizing the inclusive-divergence tends to drive the generator (the proposal) to have higher entropy than the target density, which is a desirable property for proposal design in MCMC. ...
arXiv:1806.00271v5
fatcat:kvo3vg3ayjfcjjpmntveptk6pm
Learning Deep Generative Models with Doubly Stochastic MCMC
[article]
2016
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
a neural adaptive importance sampler, where the proposal distribution is parameterized by a deep neural network and learnt jointly. ...
We present doubly stochastic gradient MCMC, a simple and generic method for (approximate) Bayesian inference of deep generative models (DGMs) in a collapsed continuous parameter space. ...
To address that, we develop a neural adaptive importance sampler (NAIS), where the adaptive proposal is parameterized by a recognition network and the parameters are optimized by descending inclusive KL-divergence ...
arXiv:1506.04557v4
fatcat:nfvl5xhxzfcydgzj5h7skwcfmy
Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis–Hastings
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We theoretically and empirically justify our sampling algorithm by showing that the masked conditionals on their own do not yield a Markov chain whose stationary distribution is that of our target distribution ...
We validate the effectiveness of the proposed parametrizations by exploring the quality of samples drawn from these energy-based models for both open-ended unconditional generation and a conditional generation ...
Wang and Ou (2017) train text-based energy networks directly via MCMC sampling with a CNN-LSTM based energy network and a backbone of autoregressive proposal distribution, but find it to be computationally ...
arXiv:2106.02736v2
fatcat:yykdqxtjq5bdzjrtfecykie3ty
« Previous
Showing results 1 — 15 out of 728 results