Fei Huang
ABSTRACT
Fine-grained Sketch-based Image Retrieval (Fine-grained SBIR), which uses hand-drawn sketches to search the target object images, has been an emerging topic over the last few years. The difficulties of this task not only come from the ambiguous and abstract characteristics of sketches with less useful information, but also the cross-modal gap at both visual and semantic level. However, images on the web are always exhibited with multimodal contents. In this paper, we consider Fine-grained SBIR as a cross-modal retrieval problem and propose a deep multimodal embedding model that exploits all the beneficial multimodal information sources in sketches and images. In our experiment with large quantity of public data, we show that the proposed method outperforms the state-of-the-art methods for Fine-grained SBIR.
- A. Chalechale, G. Naghdy, and P. Premaratne. Sketch-based shape retrieval using length and curvature of 2d digital contours. In IWCIA, pages 474--487, 2004.Google Scholar
Digital Library
- Y. Wang, M. Yu, and Q. Jia. Query by sketch: An asymmetric sketch-vs-image retrieval system. CISP, 3:1368--1372, 2011. Google ScholarCross Ref
- Y. Li, T. M. Hospedales, Y.-Z. Song, and S. Gong. Fine-grained sketch-based image retrieval by matching deformable part models. In BMVC, 2014.Google Scholar
- P. Xu, Q. Yin, Y. Qi, Y-Z. Song, Z. Ma and L. Wang. Instance-Level Coupled Subspace Learning for Fine-Grained Sketch-Based Image Retrieval. In ECCV, pages 19--34, 2016. Google ScholarCross Ref
- Q. Yu, F. Liu, Y-Z Song, T. Xiang, T. M. Hospedales, and C-C Loy. Sketch me that shoe. In CVPR, pages 799--807, 2016 Google ScholarCross Ref
- P. Sangkloy, N. Burnell, C. Ham, and J. Hays. The sketchy database: learning to retrieve badly drawn bunnies. ACM TOG, 35(4):119, 2016. Google ScholarDigital Library
- N. Dalal and T. Bill. Histograms of oriented gradients for human detection. In CVPR, pages 886--893, 2005. Google ScholarDigital Library
- C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, pages 1--9, 2015. Google ScholarCross Ref
- R. Kiros, Y. Zhu, R. R. Salakhutdinov, R. Zemel, R. Urtasun, A. Torralba, and S. Fidler. Skip-thought vectors. In NIPS, pages 3294--3302, 2015.Google ScholarDigital Library
- Y. Zhu, R. Kiros, R. S. Zemel, R. Salakhutdinov, R. Urtasun, A. Torralba, and S. Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In ICCV, pages 19--27, 2015.Google ScholarDigital Library
- M. Eitz, R. Richter, T. Boubekeur, et al. Sketch-based shape retrieval. ACM Trans. Graph. 31(4): 31:1--31:10, 2012.Google ScholarDigital Library
Index Terms
- Deep Multimodal Embedding Model for Fine-grained Sketch-based Image Retrieval
Recommendations
Sketch-based Image Retrieval using Generative Adversarial Networks
MM '17: Proceedings of the 25th ACM international conference on MultimediaFor sketch-based image retrieval (SBIR), we propose a generative adversarial network trained on a large number of sketches and their corresponding real images. To imitate human search process, we attempt to match candidate images with theimaginary image ...
Deep cascaded cross-modal correlation learning for fine-grained sketch-based image retrieval
Highlights- A simple yet effective pipeline for FG-SBIR is created through combining all the beneficial multimodal cues involved in sketches and annotated images.
- A deep cascaded neural network architecture with deep representation, embedding, and ...
AbstractFine-grained Sketch-based Image Retrieval (FG-SBIR), which utilizes hand-drawn sketches to search the target object images, has recently drawn much attention. It is a challenging task because sketches and images belong to different modalities and ...
Multi-granularity Association Learning for On-the-fly Fine-grained Sketch-based Image Retrieval
AbstractFine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a specific photo from a given query sketch. However, its widespread applicability is limited because it is difficult for most people to draw a ...
Comments