Modeling semantic concepts to support query by keywords in video

Publisher Website
Google Scholar
Abstract
Supporting semantic queries is a challenging problem in video retrieval. We propose the use of a lexicon of semantic concepts for handling the queries. We also propose auto- matic modeling of lexicon items using probabilistic tech- niques. We use Gaussian mixture models to build compu- tational representations for a variety of semantic concepts including rocket-launch, outdoor, greenery, sky etc. Train- ing requires a large amount of annotated (labeled) data. Us- ing the TREC Video test bed we compare the performance of this system supporting query by keywords with the con- ventional approach of query by example. Results demon- strate significant gains in performance using the automati- cally learnt models of semantic concepts. queries as V-TREC queries in this paper. respect to this database. In this paper, we discuss a frame- work for modeling semantic concepts and answering the V- TREC queries on the V-TREC database. We also compare the retrieval effectiveness of this framework with that of the traditional paradigm of query by examples. To represent keywords or key-concepts in terms of au- diovisual features Naphade et al. (1) presented a framework of multijects. Chang et al. (5) use a library of examples ap- proach, which they call semantic visual templates. In both cases, the implication is that if a lexicon-based approach to retrieval is advocated, there must exist a method to derive these representations from a set of user provided examples. Often the problem as with the QBE paradigm is the lack of a sufficient number of these examples to estimate generic representations that are effective.

This publication has 4 references indexed in Scilit: