Information Retrieval (IR) systems are the prominent means for searching and accessing huge amounts of unstructured information on the Web and elsewhere. They are complex systems, constituted by many different components interacting together, and evaluation is crucial to both tune and improve them. Nevertheless, in the current evaluation methodology, there is still no way to determine how much each component contributes to the overall performances and how the components interact together. This hampers the possibility of a deep understanding of IR system behaviour and, in turn, prevents us from designing ahead which components are best suited to work together for a specific search task.
This paper presents a visual tool – AVIATOR – that integrates the
progressive visual analytics paradigm in the IR evaluation process.
This tool serves to speed-up and facilitate the performance assessment of retrieval models enabling a result analysis through visual
facilities. AVIATOR goes one step beyond the common “compute–
wait–visualize” analytics paradigm, introducing a continuous evaluation mechanism that minimizes human and computational resource consumption.
We present a PhD project regarding the application of Visual Analytics (VA) methods for the automatic generation of wiki documents - i.e. wikification - and event storylines from streaming data. In contrast to static automatically generated wiki-like documents, this project investigates the employment of VA techniques for the automatic generation of wiki documents made up of dynamic contents, based on user preferences. The purpose of the project is to make the user an active component for the wikification process, able to provide useful feedback regarding which contents are more relevant for the topic of interest, thus improving the wikification algorithms. For this purpose, the project focuses on exploiting VA methods and data provenance to enhance data comprehension, by means of continuous interaction with the user according to the human-in-the-loop model.