An overview of cluster-based image search result organization: background, techniques, and ongoing challenges

Abstract

Digital photographs and visual data have become increasingly available, especially on the Web considered as the largest image database to date. However, the value of multimedia content depends on how easy it is to search and manage. Thus, the need to efficiently index, store, and retrieve images is becoming evermore important, particularly on the Web where existing image search and retrieval techniques do not seem to keep pace. Most existing solutions return a large quantity of search results ranked by their relevance to the user query. This can be tedious and time-consuming for the user, since the returned results usually contain multiple topics mixed together, and the user cannot be expected to have the time to scroll through the huge result list. A possible solution is to better organize the output information (prior or after query refinement), providing a means to facilitate the assimilation of the search results. In this context, image search result organization (ISRO) has been recently investigated as an effective and efficient solution to improve image retrieval quality on the Web. Most methods in this context exploit image clustering as a methodology capable of topic extraction and rendering semantically more meaningful results to the user. This survey paper provides a concise and comprehensive review of the methods related to cluster-based ISRO on the Web. It is made of four logical parts: First, we provide a glimpse on image information retrieval. Second, we briefly cover the background on ISRO. Third, we describe and categorize various steps involved in cluster-based ISRO, ranging over image representation, similarity computation, image clustering or grouping, and cluster-based search result visualization. Fourth, we briefly summarize and discuss ongoing research challenges and future directions, including high-dimensional feature indexing, joint word image modeling and implicit semantics, describing images based on aesthetics, automatic similarity metric learning, combining ensemble clustering methods, performing adaptive clustering, allowing dynamic trade-off between clustering quality and efficiency, diversifying image search results, integrating user feedback, and adapting results to mobile devices.