One of the primary indicators for the development of modern agriculture is the skillful use of automated technology. To advance the applications of robots in an operational environment, horticultural images have to be processed accurately. Research conducted at the College of Optical, Mechanical and Electrical Engineering, Zhejiang A&F University, by Professor Yu Liu’s group has shown an image feature-matching algorithm for horticultural images that is superior to earlier methods. Their developments, which have been published in the latest issue of Remote Sensing, use advanced Oriented FAST and Rotated BRIEF (ORB) and Lucas–Kanade (LK) optical flow algorithms.
Study: Horticultural Image Feature Matching Algorithm Based on Improved ORB and LK Optical Flow. Image Credit: MrDDK/Shutterstock.com
A crucial foundational sector for safeguarding a country’s economy is agriculture. In the quest to maximize production, development and resource distribution for horticulture, automated technology has become an industry driver. One crucial area for current agricultural growth is the creation of highly intelligent agricultural equipment.
Horticultural robots, a novel idea in agricultural machinery, have enormous economic advantages in agricultural output and a wide range of future applications. Its independence and intelligence are continuously being improved.
Horticultural Images
Typically a horticultural robotic system first uses an image acquisition tool, such as a camera, to acquire a two-dimensional image of a three-dimensional object. Feature-matching algorithms are then used to process the resulting two-dimensional images to perform image extraction and other image comprehension operations.
Feature-Matching Algorithms
Feature extraction, feature point description, and feature vector matching are the three stages of conventional feature-matching algorithms. However, real-time image capture has been challenging with this technique for various technical limitations in filtering out mismatches.
The optical flow technique-based feature-matching algorithm has increased calculation speed and high-frequency efficiency. But the supposition of grayscale invariability without proper characterization has limited its application in horticultural image feature-matching thus far.
Professor Yu Liu’s team applied an inventive approach of horticultural image feature-matching algorithm based on improved ORB and LK optical flow methods in their recently published study. The objective was to increase the accuracy, consistency and robustness of horticultural image feature-matching.
Algorithm Development
The innovative application of the ORB method consisted of two methods. First, the feature point extraction of the ORB was improved systematically. Secondly, a combined feature-matching protocol was implemented.
The first steps to improve the ORB feature point extraction involved building a layered construction of a Gaussian pyramid from the two-dimensional image. This procedure is described in detail in the Remote Sensing report. The layers of the image pyramid are aligned with rising resolution from high to low. This is accomplished by performing fixed magnification reduction and Gaussian filtering at each layer. Feature-matching achieves scale invariance by matching the images of the pyramid layer by layer at adjacent moments.
By analyzing the distance separating the descriptors, brute-force matching is frequently used in ORB feature-matching. However, when there are many points to match, this method takes longer, which reduces the effectiveness of the matching. An advanced LK optical flow matching algorithm was combined with the feature descriptor to increase the effectiveness and accuracy of feature matching.
Optical flow, which in computer vision describes the time-dependent movement of a pixel’s distance and direction, illustrates the connection between the object’s motion and the changing information in the image. The conventional LK optical flow approach tracks the instantaneous velocity of feature points using the brightness difference between two images under the premise of grayscale invariance. The optical flow predicted by the new algorithm increases the robustness and effectiveness by modifying the gradient calculation method of the original algorithm.
Validating the Algorithm
Two experiments for feature point extraction and feature-matching were designed to test the new algorithm. In each case, the appropriate existing algorithms were also applied for comparison with the new approach.
Four types of typical robotic images of horticultural samples were gathered for the feature extraction experiment. They were categorized as being under normal light, weak light, high texture, and low texture conditions.
The horticultural image feature-matching algorithm developed by the Liu group improved the accuracy and uniformity of the feature points extracted by 50.47% and 0.22, respectively, over the conventional methods. The matching accuracy of the new algorithm increased by 14.59%, while the matching time and overall time fell by 39.18% and 44.79%, respectively. The improved ORB and LK techniques have a lot of potential for use in horticulture robots’ visual simultaneous localization and mapping, allowing for better real-time positioning and map construction accuracy.
References
Chen, Qinhan, Lijian Yao, Lijun Xu, Yankun Yang, Taotao Xu, Yuncong Yang, and Yu Liu. 2022. Horticultural Image Feature Matching Algorithm Based on Improved ORB and LK Optical Flow. Remote Sensing 14, no. 18: 4465. https://doi.org/10.3390/rs14184465
