Lei Bao
ABSTRACT
The major scientific problem for content-based video retrieval is the semantic gap. Generally speaking, there are two appropriate ways to bridge the semantic gap: the first one is from human perspective (top-down) and the other one is from computer perspective (bottom-up). The top-down method defines a concept lexicon from human perspective, trains the detector for each concept based on supervised learning, and then indexes the corpus with concept detectors. Since each concept has an explicit semantic meaning, we call this concept as an explicit concept. The bottom-up approach directly discovers the underlying latent topics from video corpus by machine perspective using an unsupervised learning. The video corpus is indexed subsequently by these latent topics. As opposite to explicit concepts, we name latent topics as implicit concepts. Given the explicit concept set is pre-defined and independent of the corpus, it is impossible to completely describe corpus and users' queries. On the other hand, the implicit concepts are dynamic and dependent on the corpus, which is able to fully describe corpus and users' queries. Therefore, combining explicit and implicit concepts could be a promising way to bridge the semantic gap effectively. In this paper, a Bipartite Graph Propagation Model (BGPM) is applied to automatically balance influences from explicit and implicit concepts. Concept nodes with strong connections to queries are reinforced no matter explicit or implicit. Demonstrated by the experiments on TREVID 2008 video dataset, BGPM successfully fuses explicit and implicit concepts to achieve a significant improvement on 48 search tasks.
- C. G.M. Snoek and M. Worring. Concept-Based Video Retrieval. Foundations and Trends in Information Retrieval: 2(4), 215--322, 2009. Google Scholar
Digital Library
- A. G. Hauptmann, et al. Video Retrieval Based on Semantic Concepts. Proceedings of the IEEE : 96(4), 602--622, 2008.Google ScholarCross Ref
- A. G. Hauptmann, R. Yan, W. H. Lin, and H. Wactlar. Can high-level concepts fill the semantic gap in video retrieval? IEEE Transactions on Multimedia: 9(5), 958--966, 2007. Google ScholarDigital Library
- Y.G. Jiang, C.W. Ngo, and S.F. Chang. Semantic Context Transfer across Heterogeneous Sources for Domain Adaptive.Google Scholar
- X.G. Rui, M.J. Li, Z.W. Li, W.Y. Ma, Bipartite graph reinforcement model for web image annotation, In. Proc. of ACM Multimedia, 585--594, 2008. Google ScholarDigital Library
- D. Blei, A. Ng, and M. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3, 2003. Google ScholarDigital Library
- R. Zhao, et al. Narrowing the semantic gap-improved text-based web document retrieval using visual features. IEEE Transactions on Multimedia: 4(2), 189--200, 2002 Google ScholarDigital Library
- J. Cao, Y.D. Zhang, B.L. Feng, X.F Hua, L. Bao, X. Zhang and J.T. Li. MCG-ICT-CAS TRECVID2008 Search Task Report. In. Proc. of TRECVID Workshop, 2008.Google Scholar
- R. Yan, A. G. Hauptmann. Probabilistic Latent Query Analysis for Combining Multiple Retrieval Sources. In Proc. of ACM SIGIR, 324--331, 2006 Google ScholarDigital Library
- Y.G. Jiang, A. Yanagawa, S.F. Chang, and C.W. Ngo. CU-VIREO374: Fusing Columbia374 and VIREO374 for Large Scale Semantic Concept Detection, Columbia University ADVENT Technical Report #223-2008-1, Aug. 2008.Google Scholar
Index Terms
- Explicit and implicit concept-based video retrieval with bipartite graph propagation model
Recommendations
Explicit versus latent concept models for cross-language information retrieval
IJCAI'09: Proceedings of the 21st International Joint Conference on Artificial IntelligenceThe field of information retrieval and text manipulation (classification, clustering) still strives for models allowing semantic information to be folded in to improve performance with respect to standard bag-of-word based models. Many approaches aim at ...
Implicit feature identification in Chinese reviews using explicit topic mining model
The essential work of feature-specific opinion mining is centered on the product features. Previous related research work has often taken into account explicit features but ignored implicit features, However, implicit feature identification, which can ...
Multi-level feature representations for video semantic concept detection
Video semantic concept detection is a fundamental problem with many practical applications such as concept-based video retrieval. The major challenge of concept detection lies in the existence of the well-known semantic gap between the low-level visual ...
Comments