One of the selected use cases for MERGING, a European research project on soft robotics manipulation, coordinated by CEA and involving 11 partners from industry and academia, is the textile industry. It is a good example of a sector that could benefit from automation, but faces huge challenges in handling textile materials. For the European clothing sector, innovation through automation could be a way to boost competitiveness and reduce outsourcing, but most manufacturing processes are still manual and often outsourced to countries with low labor costs. This leads to a trade deficit for Europe, which exports € 61bn worth of textile products, but imports € 109 bn. Automation could help reverse this trend.
Textile and clothing are vital for the European economy, with around 160,000 companies employing over 1.5 million workers and generating a turnover of 162 billion euros.
A robotic system that can assist human operators in the most tedious and exhausting tasks before the final product is stitched has been developed by researchers and engineers from various institutions within MERGING, a European research project on soft robotics manipulation. They have been working with SELMARK, a leading lingerie manufacturer based in Vigo (Spain).
The robots are responsible of taking textiles from the storage, then grabbing, placing and measuring them while preventing them from wrinkling. These operations are not feasible for traditional robotic manipulators designed for rigid objects, as they cannot easily adjust in real time to the unpredictable behavior of fabric. By automating these tasks completely or partially, personnel can be assigned to tasks that have more value added.
The MERGING solution comprises a pair of collaborative robot arms designed to operate safely alongside human workers. These arms are equipped with grippers developed by OMNIGRASP that utilize electroadhesion technology, leveraging electric fields to make objects adhere to their end-effectors. This feature enhances their ability to handle objects effectively. Additionally, AIMEN provides stereo vision technology, enabling precise perception and real-time tracking of textiles, allowing the robot’s movements to be adjusted as needed. CEA’s multi-modal programming by demonstration allows even inexperienced users to instruct the robot on task execution.
For high-level coordination, LMS’s Workcell Controller is employed to ensure runtime synchronization and monitoring of the execution process. Within SELMARK’s implemented robotic system, the robot arms carefully pick fabrics from a stack of textile materials, gently separating each layer from those below it. Subsequently, each ply is placed onto the thermoforming press, with the perception system promptly notifying if any wrinkles are detected. This integrated solution aims to enhance working conditions by assigning operators to less repetitive, more value-added tasks.
MERGING started in late 2019 and, within its four-year duration, it has worked on the implementation of innovative modules in the areas of handling, perception, cognition, and programming. Among others, the integrated demonstrator at SELMARK facilities consists of a benchmark not only for the robotics community but also for the textiles industry.
For more information: merging-project.eu
Media contact: [email protected]
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 869963
This skin will induce electrostatic attraction between the gripping surface and the object, thanks to an electric field produced by skin-enclosed electrodes. The skin will also have the ability to conform to the objects to handle to raise the contact surface. Thanks to this skin, the new gripper will show enhanced gripping performances, while reducing the gripping forces, thus avoiding damaging the soft objects. The control of the complete robotic system will include, firstly, perception and supervision functions to adapt the system’s response to the execution conditions and to high variability of the flexible object’s behaviour; secondly, control abilities to make the human-robot or multi-robot co-manipulation of the flexible object safer, using Artificial Intelligence and Machine Learning. Thus, the robot will be able to learn how to handle soft objects without damaging them, as well as how to safely work side by side with humans.
In the context of this project, AIMEN has developed the perception systems for the robotic cell, considering both perception for human robot interaction and perception for grasping and manipulation. The perception system for human robot interaction is based on stereo vision and deep learning techniques. It is in charge of the detection and tracking of operators in the shared workspace as well as of their movement decomposition. This movement decomposition allows the system to be aware of the position of the head and the arms of the operator and this information is used as a base by the gesture recognition system.
On the other hand, regarding the perception for enabling the cognitive grasping and manipulation of fabrics, AIMEN is advancing on the fabric recognition algorithms and on the motion planning, especially for removing wrinkles, while EPFL have been working on the feedback from soft force sensor, to be mounted later at the fingertip of the enhanced gripper, in order to improve the object handling monitoring.
The perception system uses 2D and 3D information to detect the fabric and its wrinkles and then, deep learning techniques are used to get the best strategy of manipulation to remove wrinkles, that is, to know the best grasping points and the best movements to do. This manipulation task will be done taking into account the information of this perception module and will be modified online with the feedback of the fingertip soft force sensors.
Moreover, AIMEN is working on the monitoring of a ply while an operator is manipulating it. For this, a 3D sensor is used and the results achieved for a white ply were successful.
In the video, it can be seen how the point clouds received from the 3D sensor in free run mode are processed in real time.
]]>
A new scientific article by the LMS team in Merging, was published on CIRP Annals 2022.
Sotiris Makris, Emmanouil Kampourakis, Dionisis Andronas,
On deformable object handling: Model-based motion planning for human-robot co-manipulation, CIRP Annals, 2022,
https://doi.org/10.1016/j.cirp.2022.04.048.
Abstract
Despite extensive automation in multiple industrial sectors, manufacturing operations involving deformable objects are mostly performed manually. Challenges originating from flexible objects’ dynamic distortion underline handicaps in robot cognition and dexterity. This paper presents a model-based motion planner for deformable object co-manipulation. The developed closed-loop controlling framework interprets manipulation inputs into appropriate handling actions by simulating fabric’s distortion through a mass-spring model. The planner incorporates tools for rapid system commissioning and reconfiguration, grasping point planning, and monitoring of human actions. Inspired by automotive composite industry, two experimental setups are used for validating the system’s performance during translational and rotational co-manipulation.
Krishna Manaswi Digumarti, Michael Smith and Herbert R. Shea. A variable stiffness soft gripper with integrated ion-drag pump.
Abstract:
We present a compact, prehensile and soft gripper capable of varying its stiffness on demand, allowing not only grasping but also manipulation of objects. The gripper consists of fluidic chambers within a silicone structure and two electrostatic clutches bonded to opposite external surfaces. Actuation is achieved by pressurizing the chambers using an integrated electrohydrodynamic ‘ion-drag’ pump while simultaneously blocking one of the clutches, causing the structure to bend around and grasp an object. Once the object is grasped, the second clutch is blocked, significantly increasing the bending stiffness of the structure and allowing the object to be manipulated.
https://doi.org/10.1117/12.2615068
]]>- On Monday, 27 September, the project was presented during the RoMaDO-RA (Robotic Manipulation of Deformable Objects: Challenges in Perception, Planning and Control for Real-World Applications) workshop. Diego Perez, from AIMEN and Dionisis Andronas from LMS contributed for MERGING project with an invited presentation on “Simulation for enhanced perception and planning”. They also participated in the roundtable discussion about Future of Robotic Deformable Object Manipulation.
- On Wednesday, 29 September, the EPFL Merging team, comprising Krishna Digumarti, Vito Cacucciolo and Herbert Shea, presented a paper on “Dexterous textile manipulation using electroadhesive fingers” during the session on “Grippers and other end-effectors”. The paper has been subsequently published on the IEEE website.
The #Merging project was presented at the RoMaDO-RA (Robotic Manipulation of Deformable Objects in Real-World Applications) workshop at IROS2021, on Monday 27 September.
Diego Perez from AIMEN Centro Tecnológico and Dionisis Andronas from University of Patras contributed with an invited presentation on “Simulation for enhanced perception and planning” and participated in the roundtable..
A European Robotics Forum 2021 workshop, organised by the H2020 projects APRIL – MERGING – REMODEL – SOFTMANBOT
13 April, 11:20 am
The handling of soft materials with robots will play a significant role in the factories of the future, in particular in sectors such as the textile, food, electric and electronics industries. Four projects funded by the European Commission are currently working to advance the state of the art on this challenging scientific and technological problem: APRIL , MERGING, REMODEL, SOFTMANBOT.
The four projects are co-organizing a workshop during the European Robotics Forum, on the 13th of April, to compare their approaches to soft objects manipulation and to identify synergies and specificities. It aims to increase awareness of the importance of developing solutions for manipulating soft and deformable objects, to inform about the main scientific and technical challenges involved, and to highlight the economic and societal impact of such solutions for the European industry. In order to give the audience an overview of the specific needs of various industries, each project will focus its presentation on one chosen use-case – although they all cover several industrial sectors in their work plan. Thus, use cases from the textile, electric, toy and food industries will be presented, describing current industrial situation, needs and specificity of a robotic cell, technological challenges and proposed approaches, and elements that can be applied to other cases. Finally, a 25 minutes roundtable and Q&A will discuss synergies between the four projects, the changes in relative function and place of humans and robots in future European factories, and possibilities for standardization.
Registration:
https://www.eu-robotics.net/robotics_forum/registration/index.html
Agenda:
11:20
The Factories of the Future
Welcome and introduction from José Carlos Caldeira, Honorary Board Member, EFFRA – European Factories of the Future Research Association
11:25
Robotic manipulation for the textile industry: the MERGING project
Dionisis Andronas, project technical manager of MERGING
Current industrial situation – needs and specificity of a robotic cell in this use-case – technological challenges and proposed approach – elements that can be applied to other cases.
11:40
Robotic manipulation for the food industry: the APRIL project
Xenia Beltran, project coordinator of APRIL
Current industrial situation – needs and specificity of a robotic cell in this use-case – technological challenges and proposed approach – elements that can be applied to other cases.
11:55
Robotic manipulation for the electric industry: the REMODEL project
Gianluca Palli, project coordinator of REMODEL
Current industrial situation – needs and specificity of a robotic cell in this use-case – technological challenges and proposed approach – elements that can be applied to other cases.
12:10
Robotic manipulation for the toy industry: the SOFTMANBOT project Juan Antonio Corrales Ramón, project technical manager of SOFTMANBOT
Current industrial situation – needs and specificity of a robotic cell in this use-case – technological challenges and proposed approach – elements that can be applied to other cases.
12:25
Final roundtable, Q&A
Moderators: Leonard Engels (Dissemination manager, APRIL project) and Nicola Nosengo (Dissemination manager, MERGING project)
Synergies between the four projects; opportunities for standardisation; how the function and place of humans and robots will change in future European factories;
]]>Shadow’s Tactile Telerobot won under the “Excellence in R&D”, “Intelligent Systems” and “Tech For Good” categories.
The (virtual) award ceremony can be viewed here
]]>