Home Photo Indexing using Learned Visual Keywords

Lim, J.-H. and Jin, J.S.

    With rapid advances in sensor, storage, processor, and communication technologies, consumers can now afford to create, store, process, and share large digital photo collections. With more and more digital photos accumulated, consumers need effective and efficient tools to index and retrieve relevant photos. In this paper, we propose a novel image representation called Visual Keyword Histogram (VKH) for content-based indexing and retrieval. Visual keywords are domain-relevant visual prototypes (e.g. faces, foliage, buildings etc) with both perceptual appearance and textual semantics. Collectively, VKHs are computed over spatial tessellation to represent the distribution of visual keywords in various parts of an image. To construct a vocabulary of visual keywords, an incremental neural network is adopted to learn visual keywords for examples. This allows us to build domain-specific visual vocabularies rapidly and incrementally. We demonstrate our approach on 2400 home photos with 15 semantic queries.
Cite as: Lim, J.-H. and Jin, J.S. (2003). Home Photo Indexing using Learned Visual Keywords. In Proc. Pan-Sydney Area Workshop on Visual Information Processing (VIP2002), Sydney, Australia. CRPIT, 22. Jin, J. S., Eades, P., Feng, D. D. and Yan, H., Eds. ACS. 69.
pdf (from crpit.com) pdf (local if available) BibTeX EndNote GS