We aim at bringing all our lines of research to a level of maturity that allows hands on access to some of our most recent results and technologies. Watch this space as it grows and take the time to navigate through our demonstrations.
Follow this link to navigate through examples of image-language associations in TV travel series. The examples follow the COSMOROE (CMR) multimedia semantic interplay framework which introduces one into the wealth of semantic associations between verbal and nonverbal modalities that go well beyond literal equivalence. COSMOROE is also being used in the analysis of newspaper caricatures, and Hollywood movies, while its experimental validation in complex naturalistic stimuli is also ongoing.
- COSMOROE Data Downloads: CMR Annotated Data Corpus, CMR Annotation Specification File, CMR Language-Visual Element Parallel Corpus, CMR Utterance-Visuals Parallel Corpus, CMR EN-EL Parallel Corpus, CMR Labeled Object Dataset, CMR Labeled Body Movement & Gesture Dataset
- COSMOROE Software Downloads: CMR Annotation Checking Script
- COSMOROE Videos: Analysis of Newspaper Caricatures
- COSMOROE Technical Reports: Annotation Guide & Examples
- COSMOROE Papers: Theory, Search Engine, Newspaper Caricature Analysis
- COSMOROE Related Projects: iV&L, CogniMuse, Latsis Study
Watch a sample video for each of the POETICON Lithic Tool (PLT) Corpus Recordings: Experiment 1, Experiment 2, Experiment 3. The recordings took place in the framework of a large scale cognitive experiment, in which participants were presented with a number of unfamiliar to the modern man objects and they were asked to answer what these objects were and how they could be used. The audiovisual recordings comprise more than 95 hours of free speech, coming from more than 120 participants recorded en face and profile (to capture gestures and facial expressions) under a number of conditions, including presentation of the image of the object on a computer screen and live presentation of the object on the table in front of the participant.
- POETICON Lithic Tool Corpus Data Downloads: PLT Audiovisual Recordings & Transcripts Set, PLT Gesture & Exploratory Action Set, PLT Verbal Object Attribute and Affordance Matrix, PLT Argumentation Corpus, PLT Semantic Annotation Set, PLT Semantic & Gesture Annotation Specification File
- POETICON Lithic Tool Corpus Software Downloads: PLT Semantic Annotation Exploration Tool, PLT Semantic Inference Extraction Tool
- PLT Videos: PLT sample recordings (Experiment 1, Experiment 2, Experiment 3)
- PLT Technical Reports: Semantic and Gesture Annotation Guide & Examples
- PLT Papers: Multimodal Object Affordances Recordings and Transcripts Dataset , Object Knowledge & Exploratory Acts
- PLT Related Projects: POETICON, POETICON++
A link to navigate through the first release of the PRAXICON Semantic Network will soon be available. From within a text-based graphical user interface, one will be able to search for specific concepts in order to explore their multisensory characteristics, their distinct attributes and their rich set of associations to other related concepts. No domain-specific information restrictions apply; this is common sense and general knowledge of everyday facts structured in a dynamic, recursive network.
- PRAXICON Software Downloads: PRAXICON GitHub Project
- PRAXICON Technical Reports: The PRAXICON Database, The PRAXICON Visualization
- PRAXICON Videos: PRAXICON for Intention Prediction
- PRAXICON Papers: Theory, Computational Implementation, Human-Robot Interaction Application
- PRAXICON Related Projects: POETICON, POETICON++