A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
When Are Nonconvex Problems Not Scary?
[article]
2016
arXiv
pre-print
Preserved Fulltext
Other Versions
In this note, we focus on smooth nonconvex optimization problems that obey: (1) all local minimizers are also global; and (2) around any saddle point or local maximizer, the objective has a negative directional ...
Finally we highlight alternatives, and open problems in this direction. ...
Introduction General nonconvex optimization problems (henceforth "NCVX problems" for brevity) are NPhard, even the goal is computing only a local minimizer [MK87, Ber99] . ...
arXiv:1510.06096v2
fatcat:r2jzsjmhfzgufprx3aklv3ofde
When Are Nonconvex Optimization Problems Not Scary?
2017
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This class of nonconvex problems has two distinctive features: (i) All local minimizer are also global. ...
In applied disciplines, however, nonconvex problems abound, and simple algorithms, such as gradient descent and alternating direction, are often surprisingly effective. ...
For nonconvex problems, the Hessian ∇ 2 f (x) is not always definite. ...
doi:10.7916/d8251j7h
fatcat:aridvancfvfq5acwn2or3lxazy
Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
for solving the associated nonconvex optimization problem, to applications in machine intelligence, representation learning, and imaging sciences. ...
However, in contrast to the classical sparse recovery problem, the most natural formulation for finding the sparsest vector in a subspace is usually nonconvex. ...
Wright, “When are nonconvex problems not scary?,” arXiv preprint arXiv:1510.06096,
2015.
[106] Y. Zhai, Z. Yang, Z. Liao, J. Wright, and Y. ...
arXiv:2001.06970v1
fatcat:zluhhl3635bzrnnk7fjw5tvi7a
Convergence of Cubic Regularization for Nonconvex Optimization under KL Property
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. ...
However, existing understandings of the convergence rate of CR are conditioned on special types of geometrical properties of the objective function. ...
When Are Nonconvex Problems Not Scary? ArXiv:1510.06096v2. [Sun et al., 2017] Sun, J., Qu, Q., and Wright, J. (2017). A geometrical analysis of phase retrieval. ...
arXiv:1808.07382v1
fatcat:7eutnjrplfckzbt5na2satbbbe
Entropic metric alignment for correspondence problems
2016
ACM Transactions on Graphics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
These applications expand the scope of entropic GW correspondence to major shape analysis problems and are stable to distortion and noise. ...
Furthermore, we expand entropic GW correspondence to a framework for other matching problems, incorporating partial distance matrices, user guidance, shape exploration, symmetry detection, and joint analysis ...
That said, γ is meaningful even when Σ0 and Σ are not isometric, measuring the optimal deviation from preserving the distance structure of a surface. ...
doi:10.1145/2897824.2925903
fatcat:u22rmttwwfcypdkxzpgniyjurq
Generalized Orthogonal Procrustes Problem under Arbitrary Adversaries
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2106.15493v1 fatcat:pxmy7vgmn5atvdfqyfjyq3judu
The highlight of our work is that the theoretical guarantees are purely algebraic and do not assume any statistical priors of the additive adversaries, and thus it applies to various interesting settings ...
Despite its tremendous practical importance, it is generally an NP-hard problem to find the least squares estimator. ...
Regarding the optimization landscape, many works have shown that seemingly nonconvex objective function is not as "scary" as expected [11, 50, 51] : the landscape is usually benign and contains only one ...
arXiv:2106.15493v2
fatcat:4fpcshkosrhpbgcroheu6adoti
Analysis of the Optimization Landscapes for Overcomplete Representation Learning
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Despite the empirical success of simple nonconvex algorithms, theoretical justifications of why these methods work so well are far from satisfactory. ...
In this work, we show these problems can be formulated as ℓ^4-norm optimization problems with spherical constraint, and study the geometric properties of their nonconvex optimization landscapes. ...
When are nonconvex problems not scary? arXiv preprint arXiv:1510.06096,
2015.
[SQW16a] Ju Sun, Qing Qu, and John Wright. ...
arXiv:1912.02427v2
fatcat:fb3x6iimyjcjdpworgsqkne5w4
Efficient Dictionary Learning with Gradient Descent
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Randomly initialized first-order optimization algorithms are the method of choice for solving many high-dimensional nonconvex problems in machine learning, yet general theoretical guarantees cannot rule ...
For some highly structured nonconvex problems however, the success of gradient descent can be understood by studying the geometry of the objective. ...
When are nonconvex problems not scary? arXiv preprint arXiv:1510.06096, 2015. [37] Ju Sun, Qing Qu, and John Wright. A geometric analysis of phase retrieval. ...
arXiv:1809.10313v1
fatcat:2iopwmq3p5hxxkzux7c3cww4ha
On Gradient Descent Algorithm for Generalized Phase Retrieval Problem
[article]
2016
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Although the cost function is nonconvex, the global convergence of gradient descent algorithm from a random initialization is studied, when m is large enough. ...
The problem can be reformulated as a least-squares minimization problem. ...
ACKNOWLEDGMENTS The authors are indebted to Stefano Marchesini for providing us with the gold balls data set used in numerical simulations. The first author would like to thank Ms. ...
arXiv:1607.01121v1
fatcat:xb3va5dj4ng25asiiw5yhve6zu
Sharp Analysis for Nonconvex SGD Escaping from Saddle Points
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2019 pre-print arXiv:1902.00247v1 fatcat:soddjrulwbarfedn47fyioq2vu
optimization problems, when the objective function satisfies gradient-Lipschitz, Hessian-Lipschitz, and dispersive noise assumptions. ...
Such SGD rate matches, up to a polylogarithmic factor of problem-dependent parameters, the rate of most accelerated nonconvex stochastic optimization algorithms that adopt additional techniques, such as ...
When are nonconvex problems not scary? arXiv preprint arXiv:1510.06096. Sun, J., Qu, Q., & Wright, J. (2017). Complete dictionary recovery over the sphere i: Overview and the geometric picture. ...
arXiv:1902.00247v2
fatcat:q2olwny57revbl5z7vytcn5gfq
Stochastic Approximation for Online Tensorial Independent Component Analysis
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we present a convergence analysis for an online tensorial ICA algorithm, by viewing the problem as a nonconvex stochastic approximation problem. ...
When are nonconvex problems not scary? arXiv preprint arXiv:1510.06096, 2015. (Cited on pages 2 and 12.) Ju Sun, Qing Qu, and John Wright. ...
Recall that when α ∈ (0, 1), ψ α (x) does not satisfy convexity when x is around 0. ...
arXiv:2012.14415v2
fatcat:c2yhwj3mifcazgymxpmsvwjgxi
Escaping Saddle Points Faster with Stochastic Momentum
2020
International Conference on Learning Representations
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Stochastic gradient descent (SGD) with stochastic momentum is popular in nonconvex stochastic optimization and particularly for the training of deep neural networks. ...
In standard SGD, parameters are updated by improving along the path of the gradient at the current iterate on a batch of examples, where the addition of a "momentum" term biases the update in the direction ...
When are nonconvex problems not scary? NIPS Workshop on Non-convex Optimization for Machine Learning: Theory and Practice, 2015. Ju Sun, Qing Qu, and John Wright. ...
dblp:conf/iclr/WangLA20
fatcat:53xyuzycvbb3lbwmwxpdhtl7zq
Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2024; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2024 pre-print arXiv:2312.05720v2 fatcat:jvydbe2bq5avtep5fcub6wurjeOther Versions
However, when the new metric is used in discrete and continuous optimization together, the results are not always two-win. ...
that i mind ugly ; the problem is he has no character , loveable or otherwise . he no problem is here i really love cute, not ugly the mind or no character ; the loveable love factor cute has. of softheaded ...
arXiv:2312.05720v4
fatcat:u7cm2ts7lzhtvpa6g23o5pnale
Recovery and Generalization in Over-Realized Dictionary Learning
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2020 pre-print arXiv:2006.06179v1 fatcat:5ab3l5zotbgyxou3mt7sgwpynu
In over two decades of research, the field of dictionary learning has gathered a large collection of successful applications, and theoretical guarantees for model recovery are known only whenever optimization ...
When are nonconvex problems not scary? arXiv preprint
arXiv:1510.06096, 2015.
Yuandong Tian. Over-parameterization as a catalyst for better generalization of deep relu network. ...
When the dictionary is not known, the dictionary learning problem attempts to recover an estimate as close as possible to the ground-truth model given a set of n training samples x i from it. ...
arXiv:2006.06179v2
fatcat:lvku4p56szcslnim3hgvy4pvpi
An Unconstrained Layer-Peeled Perspective on Neural Collapse
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Empirically, we show that our results also hold during the training of neural networks in real-world tasks when explicit regularization or weight decay is not used. ...
We prove that gradient flow on this model converges to critical points of a minimum-norm separation problem exhibiting neural collapse in its global minimizer. ...
ACKNOWLEDGMENTS We are grateful to Qing Qu and X.Y. ...
arXiv:2110.02796v2
fatcat:rpnczdoqhvff3g6y7smk4spkzu
« Previous
Showing results 1 — 15 out of 25 results