A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives
[article]
2021
arXiv
pre-print
Preserved Fulltext
Other Versions
We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. ...
In addition, the article rigorously establishes why symplectic discretization schemes are important for momentum-based optimization algorithms, and provides a characterization of algorithms that exhibit ...
Acknowledgments We thank the Branco Weiss Fellowship, administered by ETH Zurich, for the generous support and the Office of Naval Research under grant number N00014-18-1-2764. ...
arXiv:2002.12493v2
fatcat:dmvyrq3dxrdtvlfrjjendrwvtu
Momentum Stiefel Optimizer, with Applications to Suitably-Orthogonal Attention, and Optimal Transport
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
Other Versions
It leads to a gradient-based optimizer with intrinsically added momentum. ...
Yet, a new approach is proposed based on, for the first time, an interplay between thoughtfully designed continuous and discrete dynamics. ...
We are grateful for partial support by NSF DMS-1847802 (LK, YW and MT), NSF ECCS-1936776 (MT), and Cullen-Peck Scholar Award (LK, YW and MT). ...
arXiv:2205.14173v3
fatcat:b7qytvqfjzgkxnfgue7osgvtmq
On dissipative symplectic integration with applications to gradient-based optimization
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Recently, continuous-time dynamical systems have proved useful in providing conceptual and quantitative insights into gradient-based optimization, widely used in modern machine learning and statistics. ...
Our arguments rely on a combination of backward error analysis with fundamental results from symplectic geometry. ...
Acknowledgements We wish to thank Jelena Diakonikolas and Michael Muehlebach for helpful discussions. This work was supported by grant ARO MURI W911NF-17-1-0304. ...
arXiv:2004.06840v3
fatcat:tovnb5pjwbgglc5a5sciw7w6ay
Variational Principles for Optimal Control of Left-Invariant Multi-Agent Systems with Asymmetric Formation Constraints
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
agents, that break the symmetry of the individual agents and the cost functions, and render the optimal control problem partially invariant by a Lie group of symmetries. ...
We study an optimal control problem for a multi-agent system modeled by an undirected formation graph with nodes describing the kinematics of each agent, given by a left invariant control system on a Lie ...
in previous developments for reduction of optimal control [4] , [22] , [23] by studying optimal control problems for multi-agent formations whose dynamics evolves on a Lie group of symmetries and ...
arXiv:1802.01224v1
fatcat:qu6hhfx44jd6hhrrfp56nogtp4
A Physical Perspective on Control Points and Polar Forms: Bézier Curves, Angular Momentum and Harmonic Oscillators
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
with polar forms -- in the context of quantum spin systems. ...
We relate B\'ezier curves to the theory of angular momentum in both classical and quantum mechanics, and describe physical analogues of various properties of B\'ezier curves -- such as their connection ...
I would also like to thank Ron Goldman, Malcolm Sabin, Kestutis Karčiauskas, Alyn Rockwood and Gábor Etesi for illuminating discussions. ...
arXiv:1809.07287v1
fatcat:yarbu5hvgveb3leprludkuqmky
Zigzag path connects two Monte Carlo samplers: Hamiltonian counterpart to a piecewise deterministic Markov process
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2022 pre-print arXiv:2104.07694v2 fatcat:5evs2digwvhmrla53bzwhioyb4Other Versions
This theoretical insight suggests that, when retaining full momentum information, Hamiltonian zigzag can better explore target distributions with highly correlated parameters by suppressing the diffusive ...
The position and velocity component of the corresponding Hamiltonian dynamics travels along a zigzag path paralleling the Markovian zigzag process; however, the dynamics is non-Markovian in this position-velocity ...
Moreover, the dynamics is time-reversible and symplectic on R 2d \ Ω. ...
arXiv:2104.07694v3
fatcat:vvadizuxdzdcvjuami553ugp34
Implicit regularization and momentum algorithms in nonlinear adaptive control and prediction
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Stable concurrent learning and control of dynamical systems is the subject of adaptive control. ...
We illustrate our analyses with simulations demonstrating our theoretical results. ...
Diakonikolas and Jordan (2019) develop momentum algorithms from the perspective of Hamiltonian dynamics, while Maddison et al. (2018) use Hamiltonian dynamics to prove linear convergence of new optimization ...
arXiv:1912.13154v6
fatcat:7bs5d63sfbh7dbkxbqzcujhdde
Losing momentum in continuous-time stochastic optimisation
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Theoretically, this combination of stochasticity and momentum is badly understood. In this work, we propose and analyse a continuous-time model for stochastic gradient descent with momentum. ...
We then propose a stable, symplectic discretisation scheme to construct an algorithm from our continuous-time dynamical system. ...
Momentum with and without subsampling. First, we study the interplay between subsampling and not subsampling in the momentum dynamic. ...
arXiv:2209.03705v1
fatcat:ozxhxzpxmzgl3mwhhxpvxhuevq
Angular momentum and rotational energy of meanflows in toroidal magnetic fields
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Further, we identify the magnetic shear as a source of poloidal ExB angular momentum and discuss the mirror and the Lorentz force. ...
We find that the components of angular momentum are given by the covariant poloidal and toroidal components of ExB and parallel flow velocities and we separately identify all relevant stress tensors, torques ...
Acknowledgements We acknowledge fruitful discussions with N. Tronko, P. Strand, V. Naulin, and J.J. Rasmussen. ...
arXiv:2003.02707v2
fatcat:c3ryk2firnaspf4ayrfjps2pwm
Phase space geometry and optimal state preparation in quantum metrology with collective spins
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We illustrate our results with the paradigmatic examples of the two-axis counter-twisting and twisting-and-turning Hamiltonians, where we provide analytical expressions for all the relevant optimal time ...
We revisit well-known protocols in quantum metrology using collective spins and propose a unifying picture for optimal state preparation based on a semiclassical description in phase space. ...
Volkoff and Jason Twamley for engaging discussions. This work was supported by NSF Grant No. PHY-1606989, and Quantum Leap Challenge Institutes program, Award No. 2016244. ...
arXiv:2211.01250v1
fatcat:z6acuz27bvhbtntmenb6r2ltk4
Angular momentum and rotational energy of mean flows in toroidal magnetic fields
2020
Nuclear Fusion
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Further, we identify the magnetic shear as a source of poloidal E × B angular momentum and discuss the mirror and the Lorentz force. ...
We find that the components of angular momentum are given by the covariant poloidal and toroidal components of E × B and parallel flow velocities and we separately identify all relevant stress tensors, ...
Acknowledgments We acknowledge fruitful discussions with N. Tronko, P. Strand, V. Naulin, and J.J. Rasmussen. ...
doi:10.1088/1741-4326/ab9fa8
fatcat:sswofkqpurdv5pp2q6hiuek3ke
Symmetry Reduction and Control of the Dynamics of a 2D Rigid Circular Cylinder and a Point Vortex: Vortex Capture and Scattering
2007
European Journal of Control
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2010; you can also visit the original URL.
The file type is application/pdf.
On this reduced space, both non-optimal and optimal controllers, the latter using Pontryagin's maximum principle, are investigated with the control objective of changing the vortex orbit from a bound to ...
Symmetry reduction and control of the Hamiltonian system of a 2D rigid circular cylinder dynamically interacting with a point vortex external to it is presented. ...
In particular, a finite-dimensional model to which dynamical systems and control theoretic ideas can be applied would be desirable. ...
doi:10.3166/ejc.13.641-655
fatcat:ouokgrfw5rbfhnyy4oykb6sr5q
Negative curvature obstructs acceleration for geodesically convex optimization, even with exact first-order oracles
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
and strongly geodesically convex functions, in the regime where the condition number scales with the radius of the optimization domain. ...
This cements a surprising gap between the complexity of convex optimization and geodesically convex optimization: for hyperbolic spaces, Riemannian gradient descent is optimal on the class of smooth and ...
Acknowledgments We thank David Martínez-Rubio for helpful discussions and feedback on a version of this paper. ...
arXiv:2111.13263v2
fatcat:lse2vmmub5hfrczsvlwcn26uyy
Conformal Symplectic and Relativistic Optimization
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Such connections with continuous-time dynamical systems have been instrumental in demystifying acceleration phenomena in optimization. ...
Moreover, we propose a new algorithm based on a dissipative relativistic system that normalizes the momentum and may result in more stable/faster optimization. ...
be conformal symplectic besides being adaptive in the momentum which may help controlling divergences. ...
arXiv:1903.04100v6
fatcat:o6o6jhbi7bcqhmfz7xdbm5x2y4
Practical Perspectives on Symplectic Accelerated Optimization
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
From this paper emerge symplectic accelerated optimization algorithms whose computational efficiency, stability and robustness have been improved, and which are now much simpler to use and tune for practical ...
Geometric numerical integration has recently been exploited to design symplectic accelerated optimization algorithms by simulating the Lagrangian and Hamiltonian systems from the variational framework ...
Acknowledgments The authors were supported in part by NSF under grants DMS-1411792, DMS-1345013, DMS-1813635, CCF-2112665, by AFOSR under grant FA9550-18-1-0288, and by the DoD under grant HQ00342010023 ...
arXiv:2207.11460v1
fatcat:vfa45nf2srhuxipkf6byj7axam
« Previous
Showing results 1 — 15 out of 1,014 results