Smart Agriculture with Intel RealSense and Directed Machines

The United Nations projects that the world’s population will reach 9.7 billion by 2050, causing global agricultural production to rise 69% between 2010 and 2050. To meet this demand, farmers and agricultural companies are turning to automation. The Smart Agriculture industry was valued at $11.5 billion in 2021 and is expected to reach $24.3 billion by 2028, a compound annual growth rate of more than 11 percent.

As farmers and renewable energy providers attempt to make up for labor shortfalls with automation and increased mechanization, a new line of autonomous Land Care Robots (LCRs) from Directed Machines offers a cost-effective way to complement the efforts of human workers.

The LCR is a heavy-duty, solar-electric machine that can perform many types of land care tasks such as mowing, hauling, security monitoring, grading, and plowing, either under operator control or in autonomous mode. Its machine vision system is anchored by two Intel® RealSense™ D455 cameras, which include an inertial measurement unit (IMU) for refined depth awareness. In addition to navigation, the cameras equip the robots with dynamic obstacle avoidance capabilities, and include an IR illuminator for night time operation.

Enabled with Intel RealSense cameras, Land Care Robots can autonomously manage vegetation, provide security intelligence, and assess the health of plants and site assets, all while towing heavy loads of equipment and supplies. In addition to agriculture, these versatile mechanized assistants are navigating the renewable energy market, where companies need to manage vegetation around solar panels and other equipment assets.

The LCR has found a home in a wide range of vertical markets including golf courses, agricultural farms, solar farms, ranches, and plant nurseries.

THE OPTIMAL SOLUTION FOR AUTONOMOUS NAVIGATION

Most autonomous vehicles designed for smart agriculture or renewable energy navigate using global positioning system (GPS) data. Depth cameras are typically used to detect and avoid obstacles, not for navigation. However, GPS navigation is not effective in structured environments, where the GPS signals are often occluded. The LCR autonomously navigates using the Intel RealSense depth cameras in conjunction with programmable Raspberry Pi* 4 RP2040 microcontrollers. There is no need for April tags or QR codes or any other type of localization symbols. The robot perceives its surroundings in real time and navigates accordingly. By using a probabilistic graph-like (topological) approach commonly called simultaneous localization and mapping, the LCR eliminates the additional overhead of first mapping the environment and then storing/updating localization data..

“Trees and solar panels can interfere with the GPS signal and introduce a lot of positional error,” explains Dan Abramson, co-founder and chief operating officer at Directed Machines. “For this reason, we use Intel RealSense depth camera D455 as our primary navigation sensors, and GPS devices as our secondary sensors. We can achieve centimeter precision using machine vision, a massive improvement over GPS in challenging environments and vitally important when operating equipment around critical infrastructure.”

LCR traveling autonomously on a GPS-occluded forest path map, showing transformed depth image and RGB image.

A SOPHISTICATED 3D VISION SYSTEM

Directed Machine’s sensor suite is anchored by two Intel RealSense cameras D455, which include an inertial measurement unit (IMU) for refined depth awareness. There are two data streams for each camera: RGB data and depth perception data. In addition to navigation, the Intel RealSense cameras equip the Land Care Robots with dynamic obstacle avoidance. To improve the stability of the RGB image and the correspondence between the depth and RGB images, the RGB sensor includes a global shutter and is matched to the depth field of view. A compact form factor makes it easy to unobtrusively mount the cameras on the LCR’s solar panel.

“Having a wide field of view is extremely important because each robot only needs two cameras to operate in all types of topographies and climate zones, including open areas and challenging structured areas,” Abramson explains.

A near infrared projector, also known as an illuminator, enables the camera to capture depth data even in total darkness. Being able to work at night effectively doubles the useful work time of the robot. It allows work to be performed during off hours at parks, golf courses, and other areas that are busy during the day. When placed in sentry mode, the cameras can identify intruders and alert stakeholders. “The same LCR can mow grass or haul equipment during the day, then do security work at night, accelerating ROI and improving the value proposition for our customers,” Abramson notes.

WHY INTEL REALSENSE

According to Abramson, LCR customers benefit from Intel RealSense technology due to its high quality and low price. “The cost/benefit ratio is better than any other depth cameras out there,” he says. “Our bill of material cost is much lower than if we had used LiDAR sensors, for instance. We can pass on those savings to customers in terms of lower prices.”

In addition to its comparatively low cost, Directed Machines selected Intel RealSense technology because it has a solid Linux kernel, which integrates well with many OS options available for the Raspberry Pi.

Finally, Intel RealSense technology processes the depth data within the camera, without burdening the robot’s CPU. This is one of the reasons that Directed Machines can use a low cost, highly efficient compute processor such as the Raspberry Pi to interpret the visual data.

“A lot of our competitors rely on software running on an expensive GPU to perform those calculations,” Abramson says. “Intel RealSense technology has already manipulated the data and put it in a usable form by the time it gets to the CPU, so we can use a much more cost-effective Raspberry Pi board.”

Intel RealSense depth cameras allow the LCR to navigate using machine vision in structured environments such as orchards and solar farms, as well as unstructured environments like ranches, parks, and golf courses, some of which comprise thousands of acres.

Watch the video: an LCR managing vegetation in the rows of a utility-scale solar farm.

Learn more about depth camera D455