For the past 4 years I have been working with ROMI, Robotics for Microfarms, a European funded project that designs and builds open source robots to help farmers work and manage their (small) farms. I travelled a bit around France, Germany and Spain to visit the different partners of the projects and ended up capturing about 18 interviews varying in length from 20 mins to an hour.

What came out from all these interviews are 13 videos, split into 4 series:

  • The basics of ROMI: the problems it solves and how, and why using open source is key to the project
  • ROMI and research in biology: how ROMI’s tools helps accelerate research in plant developmental biology
  • Computer vision: the central role computer vision has in making these robots useful
  • Tell me more: deeper dives into the 4 tools developed by ROMI, and a look at the open source softwares and libraries that were used to develop these tools

Part 1 – The basics of ROMI: the problems it solves and how

This series introduces what ROMI does, why it exists and which problems it solves. In short, local organic farms are important for our future, but they have inherent problems that makes running one complex and exhausting. ROMI aims to relieve some of that physical and mental strain by helping the farmer weed its crops, and plan his crop beds.

Give it a watch, it should explain everything you’d want to know.

Below is a YouTube playlist of 3 episodes, let it play to watch this part of the series.

Episode 1 – Microfarms and computational agriculture: a future of farming?

An introduction to what are microfarms, why we like them, but also what are the problems that come with them.

Episode 2 – The ROMI platform: a helping hand to farming communities

Introducing ROMI’s tools to help with the some of the problems mentioned in episode 1, with helping with weeding being one of the main tasks that is already helping farmers today.

Episode 3 – Open source and open hardware: developing tools that are accessible and adaptable

One of the main features of ROMI is that it follows the philosophy of open source: showing and explaining how everything is done, from building to coding, so that the users can tweak and adapt the machines to their specific tasks.

An introduction to what are microfarms, why we like them, but also what are the problems that come with them.

Part 2 – ROMI and research in biology: a powerful combination

One of the major roles of the ROMI project was to develop tools for research in biology, and it achieved that by developing a low-cost phenotyping station, renamed as the plant scanner.

This tool allows to scan in details real plants in 3D and then run algorithms on them to calculate metrics such as total leaf area, angle between the different organs, internode length and much more. These machines are already commercially available, but are not affordable.

This opens a whole new world for plant developmental biologist (and many more professionals) to advance research in biology, and maybe even answer one of the ultimate question: how is life created?

Episode 4 – Plant developmental biology: how ROMI’s tools help advance research

ROMI works in association with CNRS (Centre National de la Recherche Scientifique) to develop tools that are useful for farmers, but also can help speed up basic research with the PLANT SCANNER, also known as a phenotyping station in the biological world.

Episode 5 – Data segmentation and analysis: developing algorithms to virtually dissect plants

The PLANT SCANNER creates detailed 3D reconstruction of real plants… but then what do you do with it? You analyse it to extract useful data, such as how many grains of rice are on the plant, or even if the leaves are wilting for lack of watering.

Episode 6 – Synthetic plant modelling: creating plants in 3D to train neural networks

When the PLANT SCANNER takes pictures of the plant, it needs to recognise that this shape is a leaf, this other one is the stem, and these are flowers, but how does it do that? By using machine learning. Researchers create training data for these neural networks by painstakingly labelling images by highlighting different parts of the plant and noting what organ it is. This takes a lot of time to do as there are thousands of images, each with dozens of organs (leaves, stems, flowers).

Christophe Godin developed a software library that can generate 3D models of plants. Having created this 3D model, you can then capture a screenshot at different angles, and use this as the training data for neural networks. Having generated the plant, you do not need to label the images, as you already know what is what.

Part 3 – Tell me more about these cool tools

In this part we talk to some of the engineers responsible for the development of the ROMI tools. We talk about distributed design and manufacturing (the process of designing hardware to be built with basic tools all over the world), open source softwares and libraries as well as common components used.

Below is a playlist of 5 episodes, let it play to watch this part of the series.

Annex: Computer vision and Embodied AI

Computer vision plays a central role in the the ROMI project. It is used by the ROVER to navigate around the farm and follow crops beds, but also to recognise plants and weed around them. It is used in conjunction with the CABLEBOT to analyse the growth of crop beds, and can help detect early on potential problems with your crops. It is also used extensively in the PLANT SCANNER to reconstruct plants in 3D and segment their organs for later analysis.

Annex 1 – Computer vision: algorithms to make sense of the world

Annex 2 – Embodied AI: making robots curious

People without whom this would not be possible

I want to thank all the ROMI partners who worked on this amazing project. What has been achieved is amazing and has surpassed what was expected.

I want to also thank Manuela for designing these beautiful graphics, it truly makes it look professional. James Frei for colour correcting and grading the project and tying all this footage from different cameras nicely together. The camera operators, Marcel, Lina and Agu.

But, the biggest and warmest thanks goes to my good friend Jonathan Minchin, which you’ve seen appear extensively throughout these videos.

Jon is the project manager of ROMI, and, with Peter Hannape, was the man with the vision. A few years ago he asked me if I could come with him to Lyon and Paris to shoot some interviews for this European project that had just gotten approved. I didn’t know anything about it, but as we travelled around and asked questions to experts, I slowly got a grasp of what this project was about, and how exciting it was.

After months of editing everyday, knowing every interview by heart, I am happy, and a bit sad, that this project has finished. I am not somebody that is usually proud of what I do, but I must say that today, I am proud.

Looking forward to work on more sustainable, good for the planet kind of projects.

Director
Benjamin Scott

Editor
Benjamin Scott

Producer
Jonathan Minchin

Cinematographers
Benjamin Scott
Lina Sofia Cordoba Flechas
Agu Lucardi
Marcel Rodríguez