Towards gestured-based technologies for human-centred Smart Factories

Despite the increasing degree of automation in industry, manual or semi-automated are commonly and inevitable for complex assembly tasks. The transformation to smart processes in manufacturing leads to a higher deployment of data-driven approaches to support the worker. Upcoming technologies in this context are oftentimes based on the gesture-recognition, - monitoring or - control. This contribution systematically reviews gesture or motion capturing technologies and the utilization of gesture data in the ergonomic assessment, gesture-based robot control strategies as well as the identification of COVID-19 symptoms. Subsequently, two applications are presented in detail. First, a holistic human-centric optimization method for line-balancing using a novel indicator - ErgoTakt - derived by motion capturing. ErgoTakt improves the legacy takt-time and helps to find an optimum between the ergonomic evaluation of an assembly station and the takt-time balancing. An optimization algorithm is developed to find the best-fitting solution by minimizing a function of the ergonomic RULA-score and the cycle time of each assembly workstation with respect to the workers' ability. The second application is gesture-based robot-control. A cloud-based approach utilizing a generally accessible hand-tracking model embedded in a low-code IoT programming environment is shown.