Large-scale Content-based Image Retrieval System (CBIR) for Interactive Search through Virtual Solar Observatory

PI: Rafal A. Angryk, Montana State University
Awarded for 2011-2015

The goal of our research is to facilitate processes of solar phenomena discovery and analysis by the creation of a large-scale, automated, Content-based Image Retrieval (CBIR) system for the Virtual Solar Observatory (VSO). We want to build a fast, interactive, and user-friendly search mechanism for solar images. The system would allow a user to simply provide example images as a query, and then improve its own searching abilities by actively learning from the user's feedback. This is a more natural way for humans to search images and it will be far more convenient then text-based querying, which is currently available through the VSO. Our preliminary work on the VSO will focus on the TRACE, SoHO/EIT and Hinode/XRT images, and then we will include more archival and the new Solar Dynamics Observatory (SDO) images.

Highlights of the proposed system include: (1) The system will allow for querying data from NASA solar missions, integrated under the Virtual Solar Observatory (VSO) umbrella, by example image(s). (2) Scaled down versions of the retrieved images will be quickly presented to the user, allowing for a convenient and interactive exploration of similar images, and further improvements to the conducted data searches. (3) As user feedback (in the form of rating returned images) is obtained, the system will learn what the researcher is really looking for and automatically adjust the results returned in the next iteration. (4) Each researcher will be allowed to store multiple personalized search profiles. This will allow researchers to go back to prior searches and start where they left off. (5) All components of our system will be built as free and reliable software tools (in the form of Open Source code, under the GPL license), and in a modular matter, to allow further extensions and modifications if needed.

We want to disseminate results of our research not only by publishing research papers, but also by developing free software modules that act in a "plug-and-play" fashion. The separation of modules for image querying, ranking, indexing, etc. has potential to greatly increase the productivity of many NASA Science Directorate’s researchers. Essentially, all researches whose daily work involves analysis of NASA images could benefit from this project. We hope that this effort will help to set up a foundation for collaborative work between people struggling with the processing of NASA’s massive data repositories. This will also provide a common foundation for the NASA community to evaluate their own algorithms against free and common benchmarks. Moreover, the software could be used in academic classrooms when teaching advanced topics related to data mining, databases, and information retrieval. Graduate and undergraduate students will be involved both in the theoretical research and in the programming necessary to create the system.

The proposal falls under the NASA Science Missions Directorate (SMD), Mission Operations and Data Analysis (MO&DA) programs. HelioPhysics is a science division within SMD and our proposal is clearly directed to it. Within Heliophysics MO&DA, our proposal concerns the Heliophysics Data Environment (HPDE) program.

Contact Info