Maximizing classifier utility when there are data acquisition and modeling costs.

In this article we analyze the relationship between the number of acquired training examples and the utility of the data mining process and, given the necessary cost information, we determine the number ... While our cost model does not take into account all possible costs, our analysis provides some useful insights and a template for future analyses using more sophisticated cost models. ... One of the challenges of Utility-Based Data Mining is to maximize the utility of the data mining process when there are competing costs and benefits. ...

doi:10.1007/s10618-007-0082-x fatcat:mjg55dzqufeilh44eabjscg37m

and Data Mining. ... This workshop was geared toward researchers with an interest in how economic utility factors affect data mining (e.g., researchers in costsensitive learning and active learning) and practitioners who have ... We are indebted to Mohammad Zaki, the KDD-05 Workshop Chair, and to the SIGKDD for organizational and funding assistance. ...

doi:10.1145/1117454.1117477 fatcat:4yab7ibdrvggfptuqib5qhuyya

Considering data acquisition costs explicitly can allow the building of predictive models at significantly lower costs, and a modeler may be able to improve performance via new sources of information that ... Electronic commerce is revolutionizing the way we think about data modeling, by making it possible to integrate the processes of (costly) data acquisition and model induction. ... Acknowledgments We thank Mikhail Bilenko, Panos Ipeirotis, and Ronny Kohavi for enlightening discussions of predictive modeling and data acquisition for electronic commerce. ...

doi:10.1145/1282100.1282172 dblp:conf/ACMicec/ProvostMS07 fatcat:2xokdpbl7zhz3d2rceojl3n7x4

Keywords Cost-sensitive learning, active learning, active information acquisition, utility-based data mining ... and Data Mining. ... We are indebted to Sunita Sarawagi, the KDD-06 Workshop Chair, and to the SIGKDD for organizational and funding assistance. ...

doi:10.1145/1233321.1233338 fatcat:knxiyv57rfedvluk3cqyljrtsm

The goal of active feature-value acquisition is to incrementally select feature values that are most cost-effective for improving the model's accuracy. ... We present two policies, Sampled Expected Utility and Expected Utility-ES, that acquire feature values for inducing a classification model based on an estimation of the expected improvement in model accuracy ... Acknowledgments Prem Melville and Raymond Mooney were supported by DARPA grant HR0011-04-1-007. ...

doi:10.1145/1089827.1089828 fatcat:b5urxwnigzcobfyi2aphiv34zi

We present Confidence-based Feature Acquisition (CFA), a novel supervised learning method for acquiring missing feature values when there is missing data at both training and test time. ... At training time, CFA constructs a cascaded ensemble of classifiers, starting with the zero-cost features and adding a single feature for each successive model. ... and Space Administration. ...

doi:10.1137/1.9781611972801.45 dblp:conf/sdm/desJardinsMW10 fatcat:smdou777v5dgvi2kqiw2ai5wgq

We first formally define a utility metric which captures both the performance and the delay of a binary filtering classifier system. ... Owing to dynamic changes in characteristics of the data streams, these classifiers need to be configured dynamically to maximize the performance of the system. ... Mihaela van der Schaar for her constructive and precious comments toward reviewing this paper. ...

doi:10.1007/s42452-019-0565-6 fatcat:fys6t7q5grbwbh7a4gybcl7dke

DOAJ

Finally, we propose a method, which we refer to as reflect and correct, that can learn and predict when the classification system is likely to make mistakes and suggests acquisitions to correct those mistakes ... Simple models like label propagation and iterative classification can aggravate a misclassification by propagating mistakes in the network, while more complex models that define and optimize a global objective ... There are nodes (customers) that we need to classify and we have Market a product to a subset of customers Acquire labels for a subset of nodes Objective Maximize the number of customers Maximize the number ...

doi:10.1145/1631162.1631168 fatcat:3m7cwu2qwjehpalxbegquuvjea

In real-world machine learning applications, there is a cost associated with sampling of different features. ... We propose two types of feature value acquisition policies based on the multi-armed bandit problem: random and adaptive. ... In many real world applications, the acquisition of these features can be expensive, and thus there is a trade-off between model performance and resource consumption. ...

arXiv:1903.05382v1 fatcat:iea5rkzod5acnng3qaj6p4jwhu

step based on model improvements and acquisition costs and 3) simultaneously optimizing model improvement and acquisition costs with multi-objective Monte Carlo Tree Search. ... Feature acquisition algorithms address the problem of acquiring informative features while balancing the costs of acquisition to improve the learning performances of ML models. ... We are also grateful to Dr. Plamen Petrov for his initiation of the project. ...

arXiv:2212.11360v1 fatcat:yf2se33yjzhmzilsftsumnuasu

Are deep convolutional neural networks (CNNs) for image classification explainable by utility maximization with information acquisition costs? ... We demonstrate that deep CNNs behave equivalently (in terms of necessary and sufficient conditions) to rationally inattentive utility maximizers, a generative model used extensively in economics for human ... ACKNOWLEDGEMENT This research was supported in part by the Army Research Office under grants W911NF-21-1-0093 and W911NF-19-1-0365, and the National Science Foundation under grant CCF-2112457. ...

arXiv:2102.04594v3 fatcat:vkqscdtuobdavfjdyuuowt47pe

Multiple Versions

Here, we present a novel technique, which we refer to as reflect and correct,that can learn and predict when the underlying classification system is likely to make mistakes and it suggests acquisitions ... A number of methods that exploit both local and relational information have been developed for this task. ... We assume we are given a cost for misclassifying a node; when we classify a node as y k whereas the correct assignment is y l , we incur a cost of c kl . ...

doi:10.1609/aaai.v24i1.7704 fatcat:q572kfauo5g73bpdkcnduzm6sy

Interesting experiments could be executed in the future utilizing estimators from robust statistics inside the AL acquisition function framework. ... Comparisons with results from random sampling (RS) on AL are carried out. ... As far as "Scene 2" is concerned, the model with BALD acquisition function becomes able of 100% test accuracy when 114, 152 and 190 sample training images are utilized. ...

doi:10.5194/isprs-annals-v-3-2021-175-2021 fatcat:qxmydmntljfypb6tb2klcdbgnm

DOAJ

However, it is still not easy-to-use technology because of its heavy installation cost. When an indoor localization system is installed, it needs to collect RSSI data for training classifiers. ... This algorithm can remove the need to collect data of all location labels and select location labels to acquire new data efficiently. ... ACKNOWLEDGEMENT This work was partly supported by CREST, JST and by JSPS KAKENHI Grant Number 25700026. ...

doi:10.1145/2971648.2971710 dblp:conf/huc/ShimosakaS16 fatcat:o5rudcenffgzxdi6cyr5m46idm

Many real-world situations allow for the acquisition of additional relevant information when making an assessment with limited or uncertain data. ... In this work, we consider models that perform active feature acquisition (AFA) and query the environment for unobserved features to improve the prediction assessments at evaluation time. ... There is typically a cost associated with features and the goal is to maximize the task performance while minimizing the acquisition cost, i.e., minimize L(ŷ(x o ), y) + αC(o), (1) where L(ŷ(x o ), y) ...

arXiv:2010.02433v2 fatcat:xld5palabrdkvdoaodzjruxaw4

Multiple Versions

Maximizing classifier utility when there are data acquisition and modeling costs

Preserved Fulltext

Report on UBDM-05

Preserved Fulltext

Data acquisition and cost-effective predictive modeling

Preserved Fulltext

UBDM 2006

Preserved Fulltext

Economical active feature-value acquisition through Expected Utility estimation

Preserved Fulltext

Confidence-Based Feature Acquisition to Minimize Training and Test Costs [chapter]

Preserved Fulltext

A stream-sensitive distributed approach for configuring cascaded classifier topologies in real-time large-scale stream mining systems

Preserved Fulltext

Reflect and correct

Preserved Fulltext

Online Budgeted Learning for Classifier Induction [article]

Preserved Fulltext

Feature Acquisition using Monte Carlo Tree Search [article]

Preserved Fulltext

Rationally Inattentive Utility Maximization for Interpretable Deep Image Classification [article]

Preserved Fulltext

Other Versions

Active Inference for Collective Classification

Preserved Fulltext

DEEP BAYESIAN ACTIVE LEARNING IN HIGH-RESOLUTION SATELLITE IMAGES FOR CHANGE DETECTION IN URBAN AND SUBURBAN AREAS

Preserved Fulltext

Efficient calibration for rssi-based indoor localization by bayesian experimental design on multi-task classification

Preserved Fulltext

Active Feature Acquisition with Generative Surrogate Models [article]

Preserved Fulltext