NEW: READ THE UPDATED ON THE DEMONSTRATION OF THE PERCEPTION SYSTEM (MAY 2022)
The WP5 is focused on the development of a multi-level perception system. It is structured in two separate developing blocks, perception for environment understanding and perception for process control. During the first half of the developing phase, the work was centred on environment perception with the implementation of a system for person tracking and movement decomposition for the deployment of robotic solutions in shared environment with humans, covering the safety and also the human robot interaction. As preliminary results, we have obtained a system able to decompose the movement of a worker in shared space with robots, using fifteen descriptors, allowing to decompose all the limbs movements. The main aspect of our technical approach is the use of exteroceptive sensing precluding the use of wearables sensors that make final solutions more difficult to transfer to real industry applications. Regarding the human-robot interaction, we have obtained a robust system for the identification of static gestures using the Information obtained from the tracking system, and we are improving the robustness of the solution increasing the repeatability in the identification of static gestures.
The second part of the perception system is focused on flexible material detection, identification, and manipulation. As we are proposing a general solution for textile manipulation, one of the main restrictions comes from not considering the textures, as the fabrics used in one of our use-cases are highly variable and the production change in very short periods of time. This precludes the use of solutions based on Machine Learning that could take features from the fabric texture and transform into shape deformations. With this restriction in mind, we are using point cloud sensing and this information will be fused with other sensing technologies to allow tracking the part deformation at real-time while the materials are being manipulated. We are identifying the materials by means of real-time point cloud processing and deep learning techniques using synthetic data generated though simulation.