Generative AI driven task planning for human-robot collaborative cells
Loading...
Date
2026-02-20
Authors
Καΐπης, Μιχαήλ
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Task modeling and assignment generation in Human-Robot Collaborative (HRC) environments remain complex and labor-intensive, even with the support of modern CAx and planning tools. This thesis introduces a generative AI-driven framework that leverages Large Multi-Modal Models (LMMs) and a Digital Twin to automate the creation of task models, sequences, and assignment plans. The proposed system processes video streams containing both visual and audio cues to identify resources, tools, and tasks, enabling the automatic extraction and modeling of production activities. Furthermore, the same LMMs facilitate task-to-resource allocation while considering factors such as human ergonomics and resource workload. The framework is validated through a case study in an industrial assembly scenario, demonstrating significant reductions in manual planning effort, improved resource utilization, and enhanced human-robot collaboration. The results highlight the potential of AI-driven approaches to streamline task planning and deployment in dynamic manufacturing environments. As a product of this thesis, was synthesized and presented the paper (Nikos Dimitropoulos, Michalis Kaipis, Stavros Giartzas, George Michalos,Generative AI for automated task modelling and task allocation in human robot collaborative applications,CIRP Annals,2025)
Description
Keywords
Human-robot collaboration, Generative AI, Large multimodal models, Task planning, Digital twins, Smart manufacturing