Ontology-based Understanding of Everyday Activity Instructions

Going from natural language instructions to fully specified executable plans for household robots involves a challenging variety of reasoning steps. In this paper, a processing pipeline to tackle these steps is proposed and implemented. It uses the ontological Socio-physical Model of Activities (SOMA) as a common interface between its components. The major advantage of employing an overarching ontological framework is that its asserted facts can be stored alongside the semantics of instructions, contextual knowledge, and annotated activity models in one central knowledge base. This allows for a unified and efficient knowledge retrieval across all pipeline components, providing flexibility and reasoning capabilities as symbolic knowledge is combined with annotated sub-symbolic models.