Research Topics
The five research topics (yellow boxes) categorized into three conceptual stages (creation, exploration, and experience). Arrows show how the research topics are related.
Topic 1: Content/time prediction
Large collections of historical film material cannot be labeled fully automatically as state-of-the-art methods for the automatic analysis of film material cannot be transferred directly to historical material. The central research question of this topic is therefore how to automatically analyze scenes of complex real-world situations and abstract scene relations in historical material stemming from different sources. The expected outcomes are spatio-temporal scene analysis and similarity estimation methods to generate labels/tags for historical films on shot and frame level for further analysis. Rich historical film material manually annotated with content and time information for training and validation will be provided by the Ludwig Boltzmann Institute for Digital History.
Topic 2: Intent analysis
Historical image forensics aims to identify truthfulness by revealing the traces of an image. What simplifies this task for historical images is that the number of cameras and lenses was limited in the past so that expert content analysts are often able to identify the camera properties–and therefore the photographer–of an image. The research questions of this topic are twofold: The first question is whether it is possible to automatically classify the historical camera and lens type, either with or without contextual information of the scene such as location, time, and historical event. The second, more sophisticated question is whether it is possible to computationally support users’ reasoning about the intention of the photographer while taking the photo based on the predicted scene type and camera position. The project will primarily work with the historical photo archive of the workers’ newspaper (Austrian Labour History Society VGA) consisting of hundreds of thousands of photos by known photographers. Intent predictions will be evaluated by domain experts from the Photo Institut Bonartes.
Topic 3: Context analysis
The sparsity and inconsistency of labels/tags assigned to historical visual media collections severely limit the possibilities for interactive exploration not only for experts but also for the interested lay users. For example, users may want to explore the social role of women throughout the centuries by analyzing the contexts in which women are shown in historical photographs. For such an analysis, rich metadata would be required, which is hardly available. The research question is how to best facilitate interactive machine learning and visual analytics to support interested lay users in such complex analysis tasks on changing context over time. For evaluations, the project will build upon expert-annotated photos from the Photo Institute Bonartes and crowd-annotated photos from the Topothek Austria.
Topic 4: Relationship analysis
Cultural objects capture relationships over time and space, which expert content analysts explore to gain new insights into historical events. Relationships can be defined through metadata, such as space, time, or common labels of the artifacts, or through content similarity of the raw visual media content itself. The main research question is how to support historians in gaining insights into changing relationships between cultural objects, institutions, or persons over time and space. The project will initially work with carefully scrutinized historical Austrian prosopographic information captured by the APIS project from the Institute of Balkan and Habsburg Studies at the Austrian Academy of Sciences, the handcrafted dataset from the IMMV project as well as crowd-annotated photos and documents from the Topothek Austria.
Topic 5: Situated storytelling
Cultural heritage artifacts are often related to real-world locations. Examples are geo-tagged historical videos or images of archaeological objects that were discovered at a specific location. This information is usually shown in museums or online exhibits, but there is no intuitive and engaging way to obtain this kind of information on-site. The research question is how to best design interactive visual storytelling techniques to relate digital information to physical spaces in the form of situated mobile visualizations that are informative, engaging, and tailored to the users’ interests and current activities. We will initially work with expert-annotated photos from the Photo Institute Bonartes as well as crowd-annotated photos from the Topothek Austria.