Unsupervised knowledge representation of panoramic dental X-ray images using SVG image-and-object clustering

Abstract

Given that the meaning of an image is rarely self-evident using traditional keyword and/or content-based descriptions, the general goal of this study is to convert, with minimal human intervention, a stream of web vector graphics into a searchable knowledge graph structure that encodes semantically relevant image contents. To do so, we introduce an original framework titled SSG which automatically converts a stream of SVG images and objects into a semantic graph. We introduce an incremental clustering approach to semantically annotate SVG images and their constituent objects in a fast and efficient manner, using an aggregation of shape, area, color, and location similarity measures. We then produce an RDF graph representation of the input image and integrate it in a reference knowledge graph, incrementally extending its semantic expressiveness to improve future annotation tasks. This achieves semantization of vector image contents with minimum human effort and training data, while complying with native Web standards (i.e., SVG and RDF) to preserve transparency in representing and searching images using Semantic Web stack technologies. Our solution is of linear complexity in the number of images and clusters used. We have conducted a large battery of experiments to test and evaluate our approach. We have created a labelled SVG dataset consisting of 22,553 objects from 750 images based on panoramic dental X-ray images. To our knowledge, it is the first significant dataset of labelled SVG objects and images, which we make available online as a benchmark for future research in this area. Results underline our approach’s effectiveness, and its applicability in a practical application domain.