CHI 2023 Workshop – Uncovering Craft: Identifying and Capturing the Rationale Behind Actions in Craft.

Adopting digital tools and platforms has become increasingly common in many creative industries. Still, creative practitioners often require self-directed and informal learning when adopting new technologies. Our research project, AutoCraft, investigates the potential of digital tools, such as Augmented Reality (AR) and Virtual Reality (VR) technologies, to assist novices in acquiring new craft skills. Our approach focuses on creating a common craftsmanship language and capturing crafts’ embodied (how) and conceptual (why) knowledge to aid novices in acquiring new skills and ongoing learning. The project involves developing novel algorithms for modeling the craftsmanship skills (Skill Model), which will be transferred through novel digital tools, such as AR and VR training simulators utilizing multi-modal feedback methods. The resulting Skill Model allows for personalized feedback in the AR and VR applications, creating a novel virtual learning experience that provides an optimized procedure to follow, personalized levels of feedback, and an explanation of the rationale behind the actions that led to a particular outcome. This paper describes the initial project’s findings. It aims to provide new insights into our methodology of capturing and modeling crafting skills using a knowledge representation framework to capture the rationale behind crafting actions. The ultimate goal is to create a multi-modal feedback system in AR and VR training systems for skill acquisition, assessment, and improvement.