Zhixin Li
ABSTRACT
To learn a well-performed image annotation model, a large number of labeled samples are usually required. In this paper, we propose a novel semi-supervised approach based on adaptive weighted fusion for automatic image annotation, which can utilize the labeled data and unlabeled data simultaneously. Firstly, two different classifiers, namely the CNN (convolutional neural network) and the LDA-SVM, are constructed by all the labeled data. These two classifiers are independently represented as different feature views. Then, the most confident data with relevant pseudo-labels are chosen and amalgamated with the whole labeled dataset. After that, the two classifiers are retrained with the new labeled dataset until a stop condition is reached. In each iteration process, the unlabeled samples are labeled by high confidence pseudo-labels that are estimated by an adaptive weighted fusion strategy. Finally, we conduct experiments on two datasets, namely IAPR TC12 and NUS-WIDE, and measure the performance of the model with standard criteria, including precision, recall, F-measure, N+ and mAP. The experimental results show that our approach outperforms many state-of-the-art automatic image annotation approaches.
- D. M. Blei and M. I. Jordan. 2003. Modeling annotated data. In Proceedings of the 26th annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'03). ACM, pp.127--134. Google Scholar
Digital Library
- A. Blum and T. Mitchell. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the 11th annual Conference on Computational Learning Theory (COLT'98). ACM, pp. 92--100. Google ScholarDigital Library
- G. Carneiro, A. B. Chan, P. J. Moreno and N. Vasconcelos. 2007. Supervised learning of semantic classes for image annotation and retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3), 394--410.Google ScholarDigital Library
- X. Chen, Y. Mu, S. Yan and T.S. Chua. 2010. Efficient large-scale image annotation by probabilistic collaborative multi-label propagation. In Proceedings of the 18th ACM International Conference on Multimedia (MM'10). ACM, pp. 35--44. Google ScholarDigital Library
- T. S. Chua, J. Tang, R. Hong, H. Li, Z. Luo and Y. Zheng. 2009. NUS-WIDE: A real-world web image database from National University of Singapore. In Proceedings of ACM International Conference on Image and Video Retrieval (CIVR'09). ACM, article 48. Google ScholarDigital Library
- S. L. Feng, R. Manmatha and V. Lavrenko. 2004. Multiple Bernoulli relevance models for image and video annotation. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'04). IEEE, pp. 1002--1009.Google Scholar
- K. S. Goh, E. Y. Chang and B. Li. 2005. Using one-class and two-class SVMs for multiclass image annotation. IEEE Transactions on Knowledge and Data Engineering, 17(10), 1333--1346. Google ScholarDigital Library
- Y. Gong, Y. Jia, T. K. Leung, A. Toshev and S. Loffe. 2014. Deep convolutional ranking for multilabel image annotation. arXiv preprint arXiv:1312.4894.Google Scholar
- M. Guillaumin, T. Mensink, J. Verbeek and C. Schmid. 2009. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In Proceedings of IEEE International Conference on Computer Vision (ICCV'09). IEEE, pp. 309--316.Google Scholar
- J. Jeon, V. Lavrenko and R. Manmatha. 2003. Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'03). ACM, pp. 119--126. Google ScholarDigital Library
- X. Jing, F. Wu, Z. Li, R. Hu and D. Zhang. 2016. Multi-label dictionary learning for image annotation. IEEE Transactions on Image Processing, 25(6), 2712--2725. Google ScholarDigital Library
- J. Johnson, L. Ballan and Fei-Fei Li. 2015. Love thy neighbors: Image annotation by exploiting image metadata. In Proceedings of IEEE International Conference on Computer Vision (ICCV'15). IEEE, pp. 4624--4632. Google ScholarDigital Library
- X. Ke, M. Zhou, Y Niu and W. Guo. 2017. Data equilibrium based automatic image annotation by fusing deep model and semantic propagation. Pattern Recognition, 71, 60--77.Google ScholarCross Ref
- A. Krizhevsky, I. Sutskever and G. Hinton. 2012. ImageNet classification using deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS'12). MIT, pp. 1097--1105. Google ScholarDigital Library
- V. Lavrenko, R. Manmatha and J. Jeon. 2004. A model for learning the semantics of pictures. In Advances in Neural Information Processing Systems (NIPS'04). MIT, pp.553--560.Google Scholar
- Z. Li, J. Liu, X. Zhu, T. Liu and H. Lu. 2010. Image annotation using multi-correlation probabilistic matrix factorization. In Proceedings of the 18th ACM International Conference on Multimedia (MM'10). ACM, pp. 1187--1190. Google ScholarDigital Library
- Z. Li, J. Liu, C. Xu and H. Lu. 2013. MLRank: Multi-correlation learning to rank for image annotation. Pattern Recognition, 46(10): 2700--2710. Google ScholarDigital Library
- Z. Li, Z. Shi, X. Liu and Z. Shi. 2011. Modeling continuous visual features for semantic image annotation and retrieval. Pattern Recognition Letters, 32(3): 516--523. Google ScholarDigital Library
- Z. Li, Z. Shi, W. Zhao, Z. Li and Z. Tang. 2013. Learning semantic concepts from image database with hybrid generative/discriminative approach. Engineering Applications of Artificial Intellegence, 26(9): 2143--2152. Google ScholarDigital Library
- A. Makadia, V. Pavlovic and S. Kumar. 2010. Baselines for image annotation. International Journal of Computer Vision, 90(1): 88--105. Google ScholarDigital Library
- F. Monay and D. Gatica-Perez. 2007. Modeling semantic aspects for cross-media image indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(10): 1802--1817. Google ScholarDigital Library
- V. N. Murthy, S. Maji and R. Manmatha. 2015. Automatic image annotation using deep learning representations. In Proceedings of ACM International Conference on Multimedia Retrieval (ICMR'15). ACM, pp. 603--606. Google ScholarDigital Library
- X. Qi and Y. Han. 2007. Incorporating multiple SVMs for automatic image annotation. Pattern Recognition, 40(2): 728--741. Google ScholarDigital Library
- K. Simonyan and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409.1556.Google Scholar
- A. Tariq and H. Foroosh. 2018. Designing a symmetric classifier for image annotation using multi-layer sparse coding. Image and Vision Computing, 69: 33--43.Google ScholarCross Ref
- T. Uricchio, L. Ballan, L. Seidenari and A. Del Bimbo. 2017. Automatic image annotation via label transfer in the semantic space. Pattern Recognition, 71: 144--157.Google ScholarCross Ref
- Y. Verma and C. V. Jawahar. 2012. Image annotation using metric learning in semantic neighbourhoods. In Proceedings of European Conference on Computer Vision (ECCV'12). Springer, pp. 836--849.Google Scholar
- S. Zhang, J. Huang, Y. Huang, Y. Yu, H. Li and D. N. Metaxas. 2010. Automatic image annotation using group sparsity. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'10). IEEE, pp.3312--3319.Google Scholar
- D. Zhang, M. M. Islam and G. Lu. 2012. A review on automatic image annotation techniques. Pattern Recognition, 45(1): 346--362. Google ScholarDigital Library
- X. Zhu. 2007. Semi-supervised learning literature survey. Technical Report, University of Wisconsin-Madison.Google Scholar
Index Terms
- Collaborating CNN and SVM for Automatic Image Annotation
Recommendations
A Semi-supervised Learning Approach Based on Adaptive Weighted Fusion for Automatic Image Annotation
To learn a well-performed image annotation model, a large number of labeled samples are usually required. Although the unlabeled samples are readily available and abundant, it is a difficult task for humans to annotate large numbers of images manually. In ...
A Novel Region-based Image Annotation Using Multi-instance Learning
WKDD '09: Proceedings of the 2009 Second International Workshop on Knowledge Discovery and Data MiningIn this paper, we formulate image annotation as a semi-supervised learning problem under multi-instance learning framework. A novel graph based semi-supervised learning approach to image annotation using multiple instances is presented, which extends ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningIn multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Comments