Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method.

In this paper, we take advantage of the particular geometric structure, and describe a Riemannian trust region algorithm that provably converges to a local minimizer with from arbitrary initializations ... The rows are then recovered by linear programming rounding and deflation. ... This work was partially supported by grants ONR N00014-13-1-0492, NSF 1343282, NSF CCF 1527809, NSF IIS 1546411, and funding from the Moore and Sloan Foundations. ...

doi:10.1109/tit.2016.2632149 fatcat:vfu2lrtvifb75bqddft4sl6hwe

Multiple Versions

This particular geometric structure allows us to design a Riemannian trust region algorithm over the sphere that provably converges to one local minimizer with an arbitrary initialization, despite the ... This recovery problem is central to the theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals, and finds numerous applications in modern ... We derive an algorithm based on the Riemannian trust region method (TRM) [ABG07, AMS09] over the sphere for this purpose. ...

doi:10.1109/sampta.2015.7148922 fatcat:oygnuyi5s5gehiiokqjoo4hbju

This particular geometric structure allows us to design a Riemannian trust region algorithm over the sphere that provably converges to one local minimizer with an arbitrary initialization, despite the ... This recovery problem is central to the theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals, and finds numerous applications in modern ... We derive an algorithm based on the Riemannian trust region method (TRM) [ABG07, AMS09] over the sphere for this purpose. ...

arXiv:1504.06785v3 fatcat:r2bxpqxeqnhxxj7ng47otd3rc4

Multiple Versions

In a companion paper (arXiv:1511.04777), we design a second-order trust-region algorithm over the sphere that provably converges to a local minimizer from arbitrary initializations, despite the presence ... This recovery problem is central to theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals and finds numerous applications in modern signal ... This work was partially supported by grants ONR N00014-13-1-0492, NSF 1343282, NSF CCF 1527809, NSF IIS 1546411, and funding from the Moore and Sloan Foundations. ...

doi:10.1109/tit.2016.2632162 fatcat:4dnbtv2725bhxddmguarwug2b4

Multiple Versions

recovery, dictionary learning, sparse blind deconvolution, and many other problems in signal processing and machine learning. ... However, in contrast to the classical sparse recovery problem, the most natural formulation for finding the sparsest vector in a subspace is usually nonconvex. ... Wright, “Complete dictionary recovery over the sphere ii: Recovery by riemannian trust-region method,” IEEE Transactions on Information Theory, vol. 63, no. 2, pp. 885–914, 2016. [69] D. ...

arXiv:2001.06970v1 fatcat:zluhhl3635bzrnnk7fjw5tvi7a

We describe a second-order trust-region algorithm that provably converges to a global minimizer efficiently, without special initializations. ... Concrete applications such as dictionary learning, generalized phase retrieval, and orthogonal tensor decomposition are known to induce such structures. ... is called the Riemannian trust-region subproblem. ...

arXiv:1510.06096v2 fatcat:r2jzsjmhfzgufprx3aklv3ofde

Multiple Versions

The purpose of this paper is to illustrate, by means of nine concrete examples, that nonsmooth Riemannian optimization finds numerous applications in engineering and the sciences. ... Nonsmooth Riemannian optimization is a still scarcely explored subfield of optimization theory that concerns the general problem of minimizing (or maximizing), over a domain endowed with a manifold structure ... Fairly recently, other versions of smooth optimization algorithms such as the steepest descent method, Newton's method, trust-region methods and conjugate gradient methods, have been extended to solving ...

doi:10.1007/978-3-030-11370-4_1 fatcat:3lv7d727zje5jbttdhpzjjrale

We study one such problem -- complete orthogonal dictionary learning, and provide converge guarantees for randomly initialized gradient descent to the neighborhood of a global optimum. ... For some highly structured nonconvex problems however, the success of gradient descent can be understood by studying the geometry of the objective. ... Exact recovery of sparsely-used dictionaries. In Conference on Learning Theory, pages 37-1, 2012. [35] Ju Sun, Qing Qu, and John Wright. Complete dictionary recovery over the sphere. ...

arXiv:1809.10313v1 fatcat:2iopwmq3p5hxxkzux7c3cww4ha

This area is rich with observed phenomena and open problems; we close by highlighting directions for future research. ... Nevertheless, simple methods (e.g., gradient descent) often perform surprisingly well in practice. ... Methods in this class include trust-region methods [19] , cubic regularization [25] , and curvilinear search [24] . ...

arXiv:2007.06753v4 fatcat:l3kursnwwjc23l4opu235a3reu

Multiple Versions

We study nonconvex optimization landscapes for learning overcomplete representations, including learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictionaries, where these unsupervised ... either in the entire space or within a sufficiently large region. ... Complete dictionary recovery over the sphere ii: Recovery by riemannian trust-region method. IEEE Transactions on Information Theory, 63(2):885-914, 2016. ...

arXiv:1912.02427v2 fatcat:fb3x6iimyjcjdpworgsqkne5w4

Multiple Versions

Our theoretical findings are complemented by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods. ... Our nonconvex optimization formulation solves for a filter h on the unit sphere that produces sparse output y_i h. ... Similarly, optimization methods over Riemannian manifolds that can escape saddle points include manifold gradient descent [43] , the trust region method [40, 42] , and the negative curvature method ...

arXiv:1805.10437v2 fatcat:z56lc7bxbfbqvhankyhls6x7di

Multiple Versions

We formulate the task as a nonconvex optimization problem over the sphere. ... Our theoretical results are corroborated by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods on both synthetic and real datasets. ... We would like to thank the National Science ...

arXiv:1908.10776v3 fatcat:ku5mxr7ppfhejlh6kdyuuppspm

Multiple Versions

To the best of our knowledge, these are the first convergence guarantees for using Riemannian subgradient-type methods to optimize a class of nonconvex nonsmooth functions over the Stiefel manifold. ... Finally, we discuss the sharpness properties of various formulations of the robust subspace recovery and orthogonal dictionary learning problems and demonstrate the convergence performance of the algorithms ... We also thank the Associate Editor and two anonymous reviewers for their detailed and helpful comments. ...

arXiv:1911.05047v4 fatcat:bmxpaqankvbkdijjd4sgcmaad4

Multiple Versions

by disseminating ideas through both specific oral/poster presentations and free discussions. ... ; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? ... Acknowledgements This work is funded by the Belgian Fonds National de la Recherche Scientifique (FNRS). ...

arXiv:1410.0719v2 fatcat:4y3drgk3ujh5hopfn2p2runlzu

Multiple Versions

Citation

L. Jacques, C. De Vleeschouwer, Y. Boursier, P. Sudhakar, C. De Mol, A. Pizurica, S. Anthoine, P. Vandergheynst, P. Frossard, C. Bilen, S. Kitic, N. Bertin, R. Gribonval, N. Boumal, B. Mishra, P.-A. Absil, R. Sepulchre, S. Bundervoet, C. Schretter, A. Dooms, P. Schelkens, O. Chabiron, F. Malgouyres, J.-Y. Tourneret, N. Dobigeon, P. Chainais, C. Richard, B. Cornelis, I. Daubechies, D. Dunson, M. Dankova, P. Rajmic, K. Degraux, V. Cambareri, B. Geelen, G. Lafruit, G. Setti, J.-F. Determe, J. Louveaux, F. Horlin, A. Drémeau, P. Heas, C. Herzet, V. Duval, G. Peyré, A. Fawzi, M. Davies, N. Gillis, S. A. Vavasis, C. Soussen, L. Le Magoarou, J. Liang, J. Fadili, A. Liutkus, D. Martina, S. Gigan, L. Daudet, M. Maggioni, S. Minsker, N. Strawn, C. Mory, F. Ngole, J.-L. Starck, I. Loris, S. Vaiter, M. Golbabaee, D. Vukobratovic. "Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)." arXiv (2014)

An advantage is that we can implement it quickly with minimal effort by directly using the existing standard smooth Riemannian solvers, such as Manopt. ... Numerical experiments show the efficiency of our method especially for large-scale CP factorizations. ... Acknowledgments This work was supported by the Research Institute for Mathematical Sciences, an International Joint Usage/Research Center, at Kyoto University, JSPS KAKENHI Grant, number (B)19H02373, and ...

arXiv:2107.01538v4 fatcat:rjz3dj5vebconkgc25csuexn4q

Open Access Multiple Versions

Complete Dictionary Recovery Over the Sphere II: Recovery by Riemannian Trust-Region Method

Preserved Fulltext

Other Versions

Complete dictionary recovery over the sphere

Preserved Fulltext

Complete Dictionary Recovery over the Sphere [article]

Preserved Fulltext

Other Versions

Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture

Preserved Fulltext

Other Versions

Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications [article]

Preserved Fulltext

When Are Nonconvex Problems Not Scary? [article]

Preserved Fulltext

Other Versions

A Collection of Nonsmooth Riemannian Optimization Problems [chapter]

Preserved Fulltext

Efficient Dictionary Learning with Gradient Descent [article]

Preserved Fulltext

From Symmetry to Geometry: Tractable Nonconvex Problems [article]

Preserved Fulltext

Other Versions

Analysis of the Optimization Landscapes for Overcomplete Representation Learning [article]

Preserved Fulltext

Other Versions

Multichannel Sparse Blind Deconvolution on the Sphere [article]

Preserved Fulltext

Other Versions

A Nonconvex Approach for Exact and Efficient Multichannel Sparse Blind Deconvolution [article]

Preserved Fulltext

Other Versions

Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods [article]

Preserved Fulltext

Other Versions

Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14) [article]

Preserved Fulltext

Other Versions

Completely Positive Factorization by a Riemannian Smoothing Method [article]

Preserved Fulltext

Other Versions