A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Risk Bounds for Randomized Sample Compressed Classifiers
2008
Neural Information Processing Systems
Preserved Fulltext
We derive risk bounds for the randomized classifiers in Sample Compression setting where the classifier-specification utilizes two sources of information viz. the compression set and the message string ...
By extending the recently proposed Occam's Hammer principle to the data-dependent settings, we derive point-wise versions of the bounds on the stochastic sample compressed classifiers and also recover ...
Acknowledgments The author would like to thank John Langford for interesting discussions. ...
dblp:conf/nips/Shah08
fatcat:qvcvommcbbdp7gdinx3oass7hm
PAC-Bayesian Compression Bounds on the Prediction Error of Learning Algorithms for Classification
2005
Machine Learning
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
We consider bounds on the prediction error of classification algorithms based on sample compression. ...
Also, we extend known results on compression to the case of non-zero empirical risk. ...
Also, we would like to thank the anonymous reviewers for their useful suggestions and Shai Ben-David for handling the editorial process.
Notes ...
doi:10.1007/s10994-005-0462-7
fatcat:tmh6vwgllfekzdegszhggq2gxy
PAC-Bayes Risk Bounds for Stochastic Averages and Majority Votes of Sample-Compressed Classifiers
2007
Journal of machine learning research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
For posteriors having all their weights on a single sample-compressed classifier, the general risk bound reduces to a bound similar to the tight sample-compression bound proposed in . ...
We propose a PAC-Bayes theorem for the sample-compression setting where each classifier is described by a compression subset of the training data and a message string of additional information. ...
For posteriors having all their weights on a single sample-compressed classifier, the general risk bound reduces to a bound similar to the tight sample-compression bound of . ...
dblp:journals/jmlr/LavioletteM07
fatcat:3nwnikirq5bvjgvqfgjwbzyrma
PAC-Bayes risk bounds for sample-compressed Gibbs classifiers
2005
Proceedings of the 22nd international conference on Machine learning - ICML '05
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2013; you can also visit the original URL.
The file type is application/pdf.
We extend the PAC-Bayes theorem to the sample-compression setting where each classifier is represented by two independent sources of information: a compression set which consists of a small subset of the ...
The new PAC-Bayes theorem states that a Gibbs classifier defined on a posterior over samplecompressed classifiers can have a smaller risk bound than any such (deterministic) samplecompressed classifier ...
The PAC-Bayes risk bound of Theorem 4 is, however, valid for sample-compressed Gibbs classifiers with arbitrary posteriors. ...
doi:10.1145/1102351.1102412
dblp:conf/icml/LavioletteM05
fatcat:p46ngski3veelb34lo3666oipa
Generalization Error Analysis of Quantized Compressive Learning
2019
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Generalization error bounds are derived for three models: nearest neighbor (NN) classifier, linear classifier and least squares regression. ...
In this paper, we consider the learning problem where the projected data is further compressed by scalar quantization, which is called quantized compressive learning. ...
Concluding Remarks This paper studies the generalization error of various quantized compressive learning models, including nearest neighbor classifier, linear classifier and linear regression. ...
dblp:conf/nips/LiL19a
fatcat:yhhhn7rxerg6nmpi74oasmxici
Error Bounds for Transductive Learning via Compression and Clustering
2003
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
The technique is applied to derive error bounds for compression schemes such as (transductive) SVMs and for transduction algorithms based on clustering. ...
We present explicit error bounds for transduction and derive a general technique for devising bounds within this setting. ...
We also thank anonymous referees for their useful comments. ...
dblp:conf/nips/DerbekoEM03
fatcat:zj4g4m5455dklgx37arqlsoipu
Towards a Unified Information-Theoretic Framework for Generalization
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2111.05275v1 fatcat:4vlw7shlqfedriywiz54vj5pzu
We further study the CMI of empirical risk minimizers (ERMs) of class H and show that it is possible to output all consistent classifiers (version space) with bounded CMI if and only if H has a bounded ...
We prove that the CMI framework yields the optimal bound on the expected risk of Support Vector Machines (SVMs) for learning halfspaces. ...
Acknowledgments The authors would like to thank Blair Bilodeau, Mufan Bill Li, and Jeffery Negrea for feedback on drafts of this work. ...
arXiv:2111.05275v2
fatcat:ji25ndu3kfatbocbuirlq3sbwy
Explicit Learning Curves for Transduction and Application to Clustering and Compression Algorithms
2004
The Journal of Artificial Intelligence Research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This characterization is obtained using concentration inequalities for the tail of sums of random variables obtained by sampling without replacement. ...
We then derive error bounds for compression schemes such as (transductive) support vector machines and for transduction algorithms based on clustering. ...
We also thank anonymous referees for their useful comments. ...
doi:10.1613/jair.1417
fatcat:4rxuyoqwsncyfnlqt7cpx64gs4
Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Our main technical result is a generalization bound for compressed networks based on the compressed size. ...
Combined with off-the-shelf compression algorithms, the bound leads to state of the art generalization guarantees; in particular, we provide the first non-vacuous generalization guarantees for realistic ...
We train for 30000 steps using the ADAM optimizer. We quantize all the weights using a 4 bit codebook per layer initialized using k-means. ...
arXiv:1804.05862v3
fatcat:tgisbjtj7rbixf4yrgpjmbl3pu
Structure-aware error bounds for linear classification with the zero-one loss
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In particular, we prove upper bounds for both 'compressive learning' by empirical risk minimization (ERM) (that is when the ERM classifier is learned from data that have been projected from high-dimensions ...
We prove risk bounds for binary classification in high-dimensional settings when the sample size is allowed to be smaller than the dimensionality of the training set observations. ...
Both authors thank Dehua Xu for early work on the implementation of the algorithm in Section 3.4.2. ...
arXiv:1709.09782v1
fatcat:4wwfi2474nhs5izhbwvevaadgq
Revised Loss Bounds for the Set Covering Machine and Sample-Compression Loss Bounds for Imbalanced Data
2007
Journal of machine learning research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We then propose a loss bound, valid for any sample-compression learning algorithm (including the set covering machine), that depends on the observed fraction of positive examples and on what the classifier ...
Marchand and Shawe-Taylor (2002) have proposed a loss bound for the set covering machine that has the property to depend on the observed fraction of positive examples and on what the classifier achieves ...
Acknowledgments We would like to thank the anonymous reviewers for their helpful comments. ...
dblp:journals/jmlr/HussainLMSBM07
fatcat:fhovmibd7rdnnnxeyoxigr3g3u
Sample compression bounds for decision trees
2007
Proceedings of the 24th international conference on Machine learning - ICML '07
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
In particular, we propose Sample Compression and Occam's Razor bounds. ...
Potentially, these risk bounds can also guide the model selection process and replace traditional pruning strategies. ...
Acknowledgements The author would like to thank the anonymous reviewers and the ICML senior program committee for their comments and suggestions that helped improve the paper significantly. ...
doi:10.1145/1273496.1273597
dblp:conf/icml/Shah07
fatcat:zdsoqs7h75gcti2lxjxr7zfaym
Learning the set covering machine by bound minimization and margin-sparsity trade-off
2009
Machine Learning
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
We investigate classifiers in the sample compression framework that can be specified by two distinct sources of information: a compression set and a message string of additional information. ...
Finally, we show how these bounds are able to guide the model selection for the set covering machine algorithm enabling it to learn by bound minimization. ...
The authors would like to thank the action editor and the anonymous reviewers for their comments and suggestions that helped to improve the manuscript significantly. ...
doi:10.1007/s10994-009-5137-3
fatcat:bor4vlf54vchngptn5vorfltlu
Loss-Proportional Subsampling for Subsequent ERM
[article]
2013
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We propose a sampling scheme suitable for reducing a data set prior to selecting a hypothesis with minimum empirical risk. ...
The sampling only considers a subset of the ultimate (unknown) hypothesis set, but can nonetheless guarantee that the final excess risk will compare favorably with utilizing the entire original data set ...
Sample compression algorithms (Floyd & Warmuth, 1995) learn classifiers that can be described by only a small fraction of the training data. ...
arXiv:1306.1840v2
fatcat:3dcimlhkrvcgxaapflcyjovghm
Structure from Randomness in Halfspace Learning with the Zero-One Loss
2020
The Journal of Artificial Intelligence Research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We prove risk bounds for halfspace learning when the data dimensionality is allowed to be larger than the sample size, using a notion of compressibility by random projection. ...
In particular, we give upper bounds for the empirical risk minimizer learned efficiently from randomly projected data, as well as uniform upper bounds in the full high-dimensional space. ...
Acknowledgements AK is funded by EPSRC Fellowship EP/P004245/1 FORGING: Fortuitous Geometries and Compressive Learning. ...
doi:10.1613/jair.1.11506
fatcat:o3vhta7urzflxpabcqbzkxqhay
« Previous
Showing results 1 — 15 out of 25,815 results