SYNERGIA aims to address the limitations of current virtual training and support systems, which remain rigid, scripted and unable to respond to the cognitive, emotional and accessibility needs of diverse users.
This lack of adaptivity undermines safety, inclusiveness and trust in domains where human-machine interaction is critical, from industrial assembly to healthcare rehabilitation and human-robot collaboration.
The project envisions the next generation of Generative AI for Virtual Worlds, combining multimodal sensing, immersive content creation and explainable governance into a coherent framework for hyper-personalised, human-centric interaction. It builds upon 4 key pillars:
lightweight multimodal inference models capturing facial, vocal, gestural and physiological cues in real time, deployable at the edge with privacy protection;
generative 3D assistants capable of emotionally responsive, multilingual and accessible interaction, including gesture and sign language;
immersive pipelines that transform text, images and video into adaptive 3D environment swith dynamic storytelling and collaborative scenario generation;
a GDPR-compliant learning dataspace ensuring interoperability, user control, explainability and trust worthy governance.
Starting at TRL4-5 and progressing to TRL 6-7, the approach will be validated through pilots in:
machine assembly and maintenance,
personalised speech therapy for Parkinson’s patients,
road-safety training for cognitively diverse learners,
human-robot collaboration in complex industrial contexts.
These pilots will provide quantifiable evidence of impact, including at least 25% performance improvement, 30% reduction in training time and enhanced accessibility for vulnerable groups.
By integrating social sciences and humanities expertise, ethics-by-design principles and pathways to standardisation, SYNERGIA aims to deliver exploitable toolkits and contribute to European and international standards.
