SOMA Project breakthroughs

Our collaboration as part of the Horizon 2020 Soft Manipulation (SOMA) consortium led to breakthrough innovations in robotic manipulation capabilities that are already disrupting the way robots are used in industry. This pioneering work was carried out by leading researchers and roboticists and will have a major impact in both our Customer Fulfilment Centres, and across industry.

The SOMA project

In May, Ocado and its partners in the SOMA consortium announced the end of our highly acclaimed project. Launched in 2015 as part of the EU’s Horizon 2020 research and innovation programme, with a mission to advance the state-of-the-art robotic manipulation capabilities for industry, the SOMA project has created a legacy of major scientific breakthroughs.

It was an honour to collaborate with experts from world-renowned institutions across Europe, including: Technische Universität Berlin, the University of Pisa, IIT - Istituto Italiano di Tecnologia, Deutsches Zentrum für Luft- und Raumfahrt (DLR, the German Aerospace Center), and the Institute of Science and Technology Austria. From Ocado, we’d like to take a moment to credit:

• Graham Deacon, PhD Artificial Intelligence/Robotics, University of Edinburgh
• Panagiotis Sotiropoulos, PhD Robotics, University of Patras
• Pavlos Triantafyllou, MRes Medical Robotics and Image-Guided Intervention, Imperial College London
• Hussein Mnyusiwalla, PhD Robotics, University of Poitiers
• Duncan Russell, PhD Computing, University of Leeds

Past team members:

• Murilo Fernandes Martins, PhD Intelligent Robotics, Imperial College London
• Michall Kowalik, MEng Control Engineering and Robotics, Poznan University of Technology

The challenge

The SOMA consortium set out to research new methods of robotic manipulation - how robots grasp and move objects. The Ocado team was particularly interested in the handling of fruit and vegetables, which present a range of difficult problems for robotic manipulation. The natural characteristics of these objects are an interesting challenge because they can be easily damaged, are never the same shape twice, are deformable, and their locations are poorly constrained (they don’t sit neatly and uniformly in the box).

Traditional solutions for robotic manipulation don’t work well in grocery conditions. Usually, they are designed for objects that are rigid, with simple and known geometric shapes, placed in highly constrained locations. In other words, the opposite of the objects we want to manipulate.

SOMA hands – grasping the problem

The consortium developed compliant, underactuated hands – meaning they use soft fingers that are not rigidly connected to one another – a mechanism that is forgiving of inaccurate knowledge of both the object to be grasped and the environment it’s in. The mechanism can also be designed to apply limited pressure to the object.

This allowed us to develop grasping and manipulation strategies which can cope with objects of unknown size and shape, and which can handle them delicately. Crucially, it also allowed us to use environmental constraints around the robot to increase the robustness of grasping operations. For example, allowing resistance from the environment to constrain the object, and resistance from the object to guide the shape of the grasp.

As part of our research and testing, the Ocado team worked with pickers to compare the pressure applied by robot and human, in order to verify that SOMA hands would cause no more damage to fruit and vegetables than a person might. We did this by creating instrumented gloves that allowed us to measure pressure, and then similarly instrumented SOMA hands for comparison.

Developing a benchmarking framework allowed us to score the SOMA system against our business requirements.

SOMA planning system – seeing the problem

In collaboration with other partners, we created the planning system, which generates the manipulation strategy for a given object and environment.

As part of this, we developed a vision system that captures a scene for the robot and segments it into elements, allowing it to reason about the objects and environment.

We also contributed to the development of software which ‘reasons’ around the best way to successfully grasp different objects. It looks at which object to select next (for example, which apple from the pile) and the approach and orientation of the hand as it grasps.

The big breakthrough

The common practice in robotic manipulation is to treat everything around the object as an obstacle to be avoided. Studying how humans manipulate objects led to a radically different approach for SOMA.

We observed that people exploit the environment rather than avoid it, working with environmental constraints to make manipulation strategies more robust. For example, pushing against a surface for leverage, or brushing against it for guidance.

Once we’d observed this crucial difference, the consortium made a study of the different strategies people adopt to exploit environmental constraints. We looked at everyday tasks, as well as grasping using the SOMA hand attached to a stick, and tele-operating a robot with the SOMA hands attached.

By sampling all possible approach directions and orientations for a range of items, we were able to study the likelihood of successful grasps of SOMA hands when adopting those strategies.

The legacy of SOMA

Following the publication of the SOMA project research, we expect to see industry adopt robotic systems designed to physically interact with the environment.

But there’s a lot more research and development for the team applying our findings to picking robots for our Customer Fulfilment Centres. The principles that SOMA has exposed with respect to exploiting environmental constraints will be incorporated wherever we see that they will provide us with an operational or societal advantage. Sectors which could see a marked benefit from this type of research could include, for example, the assisted living sector. This is one area which could benefit markedly from advanced robotics to assist with varied tasks from mobility to monitoring and even service - as long as they are adapted to handle any permutation of a task (eg be able to pour water from any kettle into any cup), and are capable of interacting WITH the surroundings, not avoiding them.

Change your world with us

Across Ocado Technology, we have a diverse, rich mix of teams and expertise working to solve complex problems. Learn more about our full range of opportunities here.