CN102800083A - Crop spraying positioning method based on binocular vision gridding partition matching algorithm - Google Patents

Abstract

The invention discloses a crop spray positioning method based on a binocular vision grid division matching algorithm in the technical field of crop spraying. Including: use dual cameras to calibrate the target crop and obtain the image of the target crop; respectively obtain the binary image of the target crop of the left camera and the binary image of the target crop of the right camera; respectively obtain the binary image of the target crop obtained by the left camera and the right camera The acquired binary image of the target crop is divided into grids; for each point in the grid area on the left, a matching search is performed in the grid area on the right, and the matching points are obtained to form matching point pairs; Calculate the parallax of the left and right images and obtain the three-dimensional coordinates of the points of the corresponding target crops of the matching point pairs; delete the wrong points and perform fitting processing on the points of the target crops to obtain a fitting curve or surface; according to the fitting curve or Surface planning sprinkler path. The invention realizes the three-dimensional information contour extraction and positioning of target crops.

Description

Crop spray positioning method based on binocular vision grid division matching algorithm

technical field

The invention belongs to the technical field of crop spraying, and in particular relates to a crop spray positioning method based on a binocular vision grid division matching algorithm.

Background technique

In the process of agricultural production, in order to prevent pests and diseases, it is often necessary to spray pesticides many times. The process of applying pesticides will cause certain harm to the environment and the health of operators, especially in greenhouse conditions, where the space is relatively closed and the frequency of pesticide application is high, so the hazards are more obvious. Through the development of an automated precision pesticide application system, the liquid medicine can be sprayed directly on the surface of the crops, avoiding waste of the liquid medicine, improving the use efficiency of the medicine liquid, reducing environmental pollution, ensuring the health of workers, and reducing labor intensity. The development of automatic pesticide application system has important practical significance and social value.

In the automated pesticide application system, the main problems currently are:

1. The spray positioning is not accurate enough, and the liquid medicine is wasted seriously. According to the 2010 International Conference on Plant Protection Machinery and Pesticide Application Technology, the average utilization rate of pesticides in my country is extremely low, only about 20%. Most of the pesticides have not been fully and effectively used. The root cause is that the methods and methods of pesticide application are not scientific and reasonable. In addition, extensive spraying is often used in the use of pesticides, and the technology and conditions for precise pesticide application are lacking.

2. The spraying of pesticides is not uniform enough, and the residues of pesticides on the surface of crops exceed the standard, especially for crops produced in greenhouses. The atomization effect and spray operation mode of the chemical liquid spray have a great influence on the uniformity of the spray. According to the data, the use of electrostatic spraying can form tiny droplet particles with good adhesion, which is beneficial to reduce re-spraying and missed spraying, and improve the uniformity of spraying. Anti-drift and other technologies can also improve the spray effect to a certain extent, but fundamentally speaking, the accuracy of spray positioning will directly affect the spray quality.

3. The use adaptability of spray agricultural machinery is limited. For example, sprayers used in foreign orchards use ultrasonic spray positioning. This method requires fruit trees to be cultivated at a specific distance and arrangement. As long as there is an object within the ultrasonic detection range, it will be sprayed. It works the same way regardless of the crop form when spraying. Therefore, it is difficult to work effectively when the environment and crops change.

4. The robot positioning detection effect for automatic and precise spraying is not ideal, and the real-time performance is poor. For example, for a robot that uses visual detection technology to spray a specific area of pests and diseases, its positioning detection algorithm is often complicated and requires a certain amount of computing time. At the same time, there is also a certain error rate in the detection of target crops that need to be sprayed.

5. The two-dimensional positioning system is basically used for the spraying and application of crops. During the working process, it is generally first to detect and obtain the two-dimensional information of the spray object through a specific sensor or camera, to move the nozzle to the specified position or to control the opening and closing of multiple nozzles, and the distance between the nozzle and the target crop It is often set in advance and is not adjusted during the work process. Therefore, when there is a certain difference in the shape and size of the crops, different spray effects will be caused.

6. The cost performance of the pesticide spraying automation system is also a problem that restricts its wide application. However, with the continuous development of the number and technology of facility agriculture, and the rising labor costs associated with the arrival of an aging society, automated spraying operations will have broad application prospects.

To sum up, the most important problem at present is to solve the problem of spray target identification and positioning, and to develop a spray positioning system with good adaptability, accurate positioning, good real-time performance and high cost performance.

At present, the methods for obtaining three-dimensional information in object space mainly include laser, ultrasonic, radar, infrared and binocular vision. When the first four work, the distance information is usually calculated through the reflected wave time or phase difference. The binocular vision mainly uses the triangular ranging principle to achieve positioning information acquisition through left and right image matching. The advantage of the binocular vision positioning system is that it has a wide range of applications, and it can directly realize the recognition and positioning of the target through a certain algorithm; its disadvantage is that the recognition and positioning algorithms are often complicated, and the real-time performance and robustness are poor. It is more difficult to detect objects with irregular shapes, complex environments, and poor lighting conditions. In addition, using multi-sensor fusion technology, the positioning method that combines vision with laser, infrared, ultrasonic and other information can improve positioning accuracy and positioning reliability to a certain extent.

The spray positioning method proposed by the present invention belongs to the category of visual positioning, and a binocular visual positioning system is used to detect the position of target crops. In the current technical method of using binocular vision positioning, how to quickly, stably, accurately and reliably identify and judge the target and determine the target position contour information is the main problem that needs to be solved urgently.

Contents of the invention

The object of the present invention is to provide a crop spray positioning method based on binocular vision grid division matching algorithm, which is used to quickly and accurately calculate the position information of the target crop, and plan the running path of the sprinkler according to the position information of the target crop so that The nozzle sprays according to a reasonable spray distance and angle.

In order to achieve the above object, the technical solution provided by the present invention is a crop spray positioning method based on a binocular vision grid division matching algorithm, characterized in that the method includes:

Step 1: Use dual cameras to calibrate the target crop and obtain images of the target crop; the dual cameras are respectively recorded as the left camera and the right camera;

Step 2: Separate the image of the target crop acquired by the left camera and the image of the target crop acquired by the right camera from the background respectively, and obtain the binary image of the target crop of the left camera and the binary image of the target crop of the right camera;

Step 3: Carry out grid division on the binary image of the target crop acquired by the left camera and the binary image of the target crop acquired by the right camera respectively, to obtain the left grid area and the right grid area;

Step 4: For each point in the grid area on the left, perform a matching search in the grid area on the right to obtain matching points and form a pair of matching points;

Step 5: Calculate the parallax of the left and right images by each matching point pair and obtain the three-dimensional coordinates of the point of the target crop corresponding to the matching point pair;

Step 6: Analyze the three-dimensional coordinates of the points of the target crop, and delete the wrong points;

Step 7: Fitting the points of the target crop to obtain a fitting curve or surface;

Step 8: Plan the nozzle path according to the fitting curve or surface.

The image of the target crop acquired by the left/right camera is separated from the background, and the binary image obtained by the left/right camera target crop includes:

Step 101: In the HSI color space, use a fixed threshold segmentation method to obtain a preliminary segmented image of the target crop;

Step 102: Obtain the grayscale image of the target crop by using the super green algorithm;

Step 103: Perform histogram statistics on the grayscale image of the target crop to obtain the histogram of the target crop;

Step 104: smoothing the histogram of the target crop using the nearest neighbor multi-point averaging method;

Step 105: Search for the peak of the smoothed histogram of the target crop and calculate the positions of valleys on the left and right sides of the peak, so as to obtain a binary image of the target crop.

Said step 4 includes:

Step 201: Initialize the parameters, let j=1, wj=1, Min=10000; wherein, j is the ordinate of the point in the right grid area, and wj is used to record the matching points in the right grid area The ordinate of , Min is used to record the minimum value of the sum of the corrected absolute differences of the left grid and the right grid;

Step 202: Select a point (u, i) in the left grid area, record the left grid where it is located as p, and calculate the gray mean Mi and variance Ei(u,i) of the left grid p ;

Step 203: Select the point (u, j) in the grid area on the right, make the abscissa of the point the same as the point (u, i) in the grid area on the left, and set the point on the right The side grid is denoted as q and the gray mean N _j and variance Fj(u,j) of the right grid q are calculated;

Step 204: Judging whether |Mi-Fj|<ε is true, if |Mi-Fj|<ε is true, then execute step 205; otherwise, set j=j+1 and return to step 203; where ε is a set value;

Step 205: Calculate the sum of absolute differences between the left grid p and the right grid q, denoted as SAD; calculate the variance Ei(u,i) of the left grid p and the variance Fj(u ,j) of the absolute value of the difference, recorded as FCC;

Step 206: Calculate the sum of the corrected absolute differences of the left grid p and the right grid q according to the formula SF=SAD×a+FCC×b, where a and b are scale parameters respectively;

Step 207: If the sum of the corrected absolute differences of the left grid p and the right grid q is less than Min, then execute step 208; otherwise, set j=j+1 and return to step 203;

Step 208: Let Min=SF, wj=j; judge whether the value of j passes through all epipolar points, if the value of j passes through all epipolar points, then execute step 209; otherwise, make j=j+1 and return Step 203;

Step 209: The point (u, wj) in the grid area on the right is the matching point of the point (u, i) in the grid area on the left.

The ratio of the proportional parameters a and b is 1:1.

Said step 5 includes:

Step 301: Use the formula D= _XL - _XR to calculate the parallax of the left and right images, where _XL is the abscissa of one point in the matching point pair, and _XR is the abscissa of the other point in the matching point pair;

Step 302: Use the formula $\left\{\begin{array}{c}{x}_c=\frac{{\mathrm{BX}}_L}{\mathrm{D.}}\\ {\mathrm{the\; y}}_c=\frac{\mathrm{BY}}{\mathrm{D.}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{D.}}\end{array}\right.$ Calculate the three-dimensional coordinates of the point of the target crop corresponding to the matching point pair; where x _c , y _c and z _c are the three-dimensional coordinate values of the point of the target crop respectively, B is the distance between the optical axes of the left and right cameras, and X _L is the abscissa of a point in the matching point pair, Y is the ordinate of any point in the matching point pair, and f is the focal length of the left or right camera.

The invention adopts the method of fine segmentation self-applicable threshold segmentation, which improves the robustness and accuracy of the adaptive segmentation algorithm; adopts the method of grid division to reduce the overall calculation amount; in the image matching process of binocular vision positioning , using the improved SAD algorithm, greatly reduces the error matching that is easy to occur in a single SAD algorithm, and is faster than the maximum correlation coefficient method in terms of computational efficiency.

Description of drawings

Fig. 1 is the flow chart of the crop spray positioning method based on the binocular vision grid division matching algorithm;

Fig. 2 is a schematic diagram of using a dual camera to calibrate a target crop; wherein, (a) is a schematic diagram of using a left camera to calibrate a target crop, and (b) is a schematic diagram of using a right camera to calibrate a target crop;

Figure 3 is a preliminary segmented image of the target crop obtained by using the fixed threshold segmentation method;

Fig. 4 is a schematic diagram of meshing the binary image of the target crop acquired by the left camera;

Fig. 5 is a flowchart of forming a matching point pair;

Figure 6 is a schematic diagram of dual cameras with parallel optical axes.

Detailed ways

The preferred embodiments will be described in detail below in conjunction with the accompanying drawings. It should be emphasized that the following description is only exemplary and not intended to limit the scope of the invention and its application.

Fig. 1 is a flow chart of crop spray positioning method based on binocular vision grid division matching algorithm. As shown in Figure 1, the crop spray positioning method based on the binocular vision grid division matching algorithm provided by the present invention includes:

Step 1: Use dual cameras to calibrate the target crop and obtain images of the target crop, and the dual cameras are respectively marked as the left camera and the right camera.

When arranging the left and right cameras, it should be ensured that the optical axes of the two cameras are parallel to each other and on the same horizontal plane. Figure 2 is a schematic diagram of using dual cameras to calibrate target crops.

Step 2: Separate the image of the target crop captured by the left camera and the image of the target crop captured by the right camera from the background respectively, and obtain the binary image of the target crop of the left camera and the binary image of the target crop of the right camera.

The following takes the left camera as an example to illustrate the process of obtaining the binary image of the target crop of the left camera.

Step 101: In the HSI color space, use a fixed threshold segmentation method to obtain a preliminary segmented image of the target crop.

In this step, it is first necessary to convert the target crop image (RGB image) acquired by the left camera into an HSI image and perform normalization processing.

Next, set the value ranges of H (hue), S (saturation), and I (brightness), position the pixels outside the range as black, and keep the rest of the original values.

Next, the pixels that retain the original value are converted from the HSI image to the RGB image and denormalized, and finally the preliminary segmented image of the target crop is obtained. Figure 3 is a preliminary segmented image of the target crop obtained by using the fixed threshold segmentation method.

Step 102: Obtain the grayscale image of the target crop by using the supergreen algorithm.

The super green algorithm is an algorithm that increases the contrast with the non-green background by increasing the weight of the green channel. This algorithm can better extract the information of green crops, which is often used in image processing of green crops. The algorithm processes each pixel using the formula 2×G-R-B, where G, R, and B represent the values of the pixel's green, red, and blue channels, respectively. After the processing of the super green algorithm, the grayscale image of the target crop can be obtained.

Step 103: Perform histogram statistics on the grayscale image of the target crop to obtain the histogram of the target crop.

Step 104: smoothing the histogram of the target crop by using the nearest neighbor multi-point averaging method.

The nearest neighbor multi-point averaging method is an algorithm solution to solve when the histogram has a local mutation, but the macro trend has not yet reached the trough. Perform super-green calculation on the image whose background is set to zero after preliminary segmentation, and remove the background part with a gray value of 0 in its histogram. When only the number of pixels corresponding to a single gray level is a local small value, the point is not considered It must be the trough position, so using the average value of adjacent continuous points as the trough position detection can effectively avoid the influence of local minima on the segmentation results. The specific algorithm is to divide the found minimum position into three equal intervals by the left neighborhood, right neighborhood, and central area, calculate the average value of the histogram data in each small area, and judge whether it is a reasonable trough based on the numerical results Location. For example, when finding the position of the left trough, the regional numerical characteristics appear to be low on the left and high on the right, and the average value of the three areas is the smallest on the left and the largest on the right, so you should continue to move to the left to find a new trough to avoid local extremes. small impact.

Step 105: Search for the peak of the smoothed histogram of the target crop and calculate the positions of valleys on the left and right sides of the peak, so as to obtain a binary image of the target crop.

Search the troughs on both sides of the peak, and after the troughs are obtained, the image between the troughs on both sides is the binary image of the target crop.

Step 3: Carry out grid division on the binary image of the target crop acquired by the left camera and the binary image of the target crop acquired by the right camera respectively, to obtain the left grid area and the right grid area.

Fig. 4 is a schematic diagram of meshing the binary image of the target crop acquired by the left camera. In FIG. 4 , still taking the left camera as an example, the process of meshing the acquired binary image of the target crop is described. Use the upper left corner of the image as the starting position of the search target pixel, and locate the first grid when the pixel of the target crop appears. Continue to divide the grid around it, judge whether there are pixels of the target crop around the grid, and if so, then locate another grid. For the target pixel at the edge, the grid is added from the reverse direction for the area that is not enough to draw a full grid. This ensures that all target pixels are drawn into the grid area of the same size.

Step 4: For each point in the grid area on the left, perform a matching search in the grid area on the right to obtain matching points to form a pair of matching points.

Fig. 5 is a flowchart of forming matching point pairs. As shown in Figure 5, the process of forming matching point pairs includes:

Step 201: Initialize the parameters, let j=1, wj=1, Min=10000; wherein, j is the ordinate of the point in the right grid area, and wj is used to record the matching points in the right grid area The ordinate of , Min is used to record the minimum value of the sum of the corrected absolute differences of the left grid and the right grid.

Step 202: Select a point (u, i) in the left grid area, record the left grid where it is located as p, and calculate the gray mean Mi and variance Ei(u,i) of the left grid p .

Among them, the calculation formula of the gray value Mi of the grid p on the left is

$\mathrm{<span\; class="notranslate">\; Mi</span>}<span\; class="notranslate">\; =</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; mn</span>}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

The formula for calculating the variance Ei(u,i) of the grid p on the left is

$\mathrm{<span\; class="notranslate">\; Ei</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; mn</span>}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

In the above formulas (1) and (2), u and i are the abscissa and ordinate of the point (u, i), respectively, and m and n are the number of horizontal pixels and vertical pixels of the grid p on the left, respectively , g(x,y) is the function expression of the pixel in the grid p on the left.

Step 203: Select the point (u, j) in the grid area on the right, make the abscissa of the point the same as the point (u, i) in the grid area on the left, and set the point on the right The side grid is denoted as q and the gray mean N _j and variance Fj(u,j) of the right grid q are calculated.

Among them, the calculation formula of the gray value Nj of the grid q on the right is

${\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; mn</span>}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

The formula for calculating the variance Fj(u,j) of the grid q on the right is

$\mathrm{<span\; class="notranslate">\; Fj</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; mn</span>}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

In the above formulas (3) and (4), u and j are the abscissa and ordinate of the point (u, j), respectively, and m and n are the number of horizontal pixels and vertical pixels of the grid q on the right side, respectively , f(x,y) is the function expression of the pixel in the grid q on the right.

Step 204: Judging whether |Mi-Fj|<ε is true, if |Mi-Fj|<ε is true, it means that the grid p on the left and the grid q on the right are approximately equal, and step 205 can be executed; otherwise, set j= j+1 and return to step 203; wherein, ε is a set value.

Step 205: Calculate the sum of absolute differences between the left grid p and the right grid q, denoted as SAD; calculate the variance Ei(u,i) of the left grid p and the variance Fj(u ,j) The difference of the absolute value, recorded as FCC.

Calculate the sum of the absolute difference between the left grid p and the right grid q using the formula

为左侧网格p或者右侧网格q中像素的函数表达式。In the above formula (5), c and r are respectively the abscissa and ordinate of point (u, i) or point (u, j),

is the function expression of the pixels in the grid p on the left or the grid q on the right.

The difference between the absolute value of the variance Ei(u,i) of the grid p on the left and the variance Fj(u,j) of the grid q on the right is calculated using the formula

FCC=|Ei(u,i)-Fj(u,j)| （6）

Step 206: Calculate the sum of the corrected absolute differences of the left grid p and the right grid q according to the formula SF=SAD×a+FCC×b, where a and b are scale parameters respectively. The proportion parameters a and b determine the proportion of variance factor and sum of absolute difference factor on the final result. Generally, the ratio of the proportional parameters a and b is set to 1:1.

Step 207: If the sum of the corrected absolute differences of the left grid p and the right grid q is less than Min, execute step 108; otherwise, set j=j+1 and return to step 103.

Step 208: Let Min=SF, wj=j; judge whether the value of j passes through all epipolar points, if the value of j passes through all epipolar points, then execute step 209; otherwise, make j=j+1 and return Step 203.

For a parallel-axis binocular system, if the plane determined by the optical axes of the two cameras is relatively horizontal to the world coordinate system, the epipolar constraint can be regarded as the target position defined by the same height row of the left and right images, that is, for the same target point in space Generally speaking, the image coordinates of the corresponding points in the images acquired by the left and right cameras are equal in the vertical direction, and only the pixel coordinates in the horizontal direction are different. For this reason, the search for matching points in the present invention is performed along the epipolar position. Because the matching point must be on the epipolar line, the calculation efficiency and accuracy of the matching algorithm can be improved.

Step 209: The point (u, wj) in the grid area on the right is the matching point of the point (u, i) in the grid area on the left.

Step 5: Calculate the parallax of the left and right images from each pair of matching points and obtain the three-dimensional coordinates of the points of the target crop corresponding to the pair of matching points.

Figure 6 is a schematic diagram of dual cameras with parallel optical axes. In FIG. 6 , since the optical axes of the left and right cameras are parallel to each other, the distance between them is the baseline length B. Let the optical center _OL of the left camera be the origin of the world coordinate system, that is, the world coordinate system and the left camera coordinate system completely coincide. The optical axis O _L Z _L is perpendicular to the imaging plane of the left camera, and the intersection point is O ₁ , and the optical axis O _R Z _R is perpendicular to the imaging plane of the right camera, and the intersection point is O ₂ . The two cameras view the same point P of the space object at the same time, and obtain the corresponding image points P _L and P _R on the left and right cameras respectively, and the coordinates are P _L (X _L , Y _L ) and P _R (X _R , Y _R ), then The parallax is D=X _L -X _R .

use the formula

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; BX</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}}{\mathrm{<span\; class="notranslate">\; D.</span>}}$

${\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; BY</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Bf</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}$

The three-dimensional coordinates of the point corresponding to the target crop can be calculated from the matched point pair. Wherein, x _c , y _c and z _c are respectively the three-dimensional coordinate values of the points of the target crop. B is the distance between the optical axes of the left and right cameras, X _L is the abscissa of the point acquired by the left camera in the matching point pair, and Y is the ordinate of any point in the matching point pair. Since the two cameras are optical axis parallel coordinate systems, the Y-axis coordinates of the left camera and the right camera are actually the same value, that is, Y=Y _L =Y _R . f is the focal length of the left camera or the right camera.

Step 6: Analyze the three-dimensional coordinates of the points of the target crop, and delete the wrong points.

The deletion of wrong points is mainly based on constraints, including that the order of matching points corresponding to the binary images captured by the left and right cameras should be consistent, and if they are inconsistent, it is considered a wrong matching point. In addition, if the coordinates of the calculated point exceed the given range, it is considered as a wrong matching point and should be cleared.

Step 7: Perform fitting processing on the points of the target crop to obtain a fitting curve or surface.

According to the spatial discrete points obtained in step 6, fit the discrete points with a specific spatial curve (surface) model. For example, if the basic shape of the target crop is a spherical Multiplication obtains the best fit curve (surface) equation parameters.

Step 8: Plan the nozzle path according to the fitting curve or surface.

Plan the path of the nozzle according to the fitting curve or surface obtained in step 7, so that the nozzle can spray at the ideal spray distance and angle.

The invention can realize the three-dimensional information contour extraction and positioning of target crops. The method is accurate in positioning and moderate in calculation, which can meet the real-time requirements of spraying, and has high flexibility and adaptability in the working process. It provides an effective method for realizing precise automatic spraying, reducing environmental pollution, and improving the utilization rate of pesticides. implementation method.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present invention can easily think of changes or Replacement should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (5)

1. a crop spray location method based on binocular vision grid division matching algorithm, it is characterized in that described method comprises: Step 1: Use dual cameras to calibrate the target crop and obtain images of the target crop; the dual cameras are respectively recorded as the left camera and the right camera; Step 2: Separate the image of the target crop acquired by the left camera and the image of the target crop acquired by the right camera from the background respectively, and obtain the binary image of the target crop of the left camera and the binary image of the target crop of the right camera; Step 3: Carry out grid division on the binary image of the target crop acquired by the left camera and the binary image of the target crop acquired by the right camera respectively, to obtain the left grid area and the right grid area; Step 4: For each point in the grid area on the left, perform a matching search in the grid area on the right to obtain matching points and form a pair of matching points; Step 5: Calculate the parallax of the left and right images by each matching point pair and obtain the three-dimensional coordinates of the point of the target crop corresponding to the matching point pair; Step 6: Analyze the three-dimensional coordinates of the points of the target crop, and delete the wrong points; Step 7: Fitting the points of the target crop to obtain a fitting curve or surface; Step 8: Plan the nozzle path according to the fitting curve or surface. 2. The method according to claim 1, wherein the image of the target crop obtained by the left/right camera is separated from the background, and obtaining the binary image of the left/right camera target crop comprises: Step 101: In the HSI color space, use a fixed threshold segmentation method to obtain a preliminary segmented image of the target crop; Step 102: Obtain the grayscale image of the target crop by using the super green algorithm; Step 103: Perform histogram statistics on the grayscale image of the target crop to obtain the histogram of the target crop; Step 104: smoothing the histogram of the target crop using the nearest neighbor multi-point averaging method; Step 105: Search for the peak of the smoothed histogram of the target crop and calculate the positions of valleys on the left and right sides of the peak, so as to obtain a binary image of the target crop. 3. The method according to claim 1, characterized in that said step 4 comprises: Step 201: Initialize the parameters, let j=1, wj=1, Min=10000; wherein, j is the ordinate of the point in the right grid area, and wj is used to record the matching points in the right grid area The ordinate of , Min is used to record the minimum value of the sum of the corrected absolute differences of the left grid and the right grid; Step 202: Select a point (u, i) in the left grid area, record the left grid where it is located as p, and calculate the gray mean Mi and variance Ei(u,i) of the left grid p ; Step 203: Select the point (u, j) in the grid area on the right, make the abscissa of the point the same as the point (u, i) in the grid area on the left, and set the point on the right The side grid is denoted as q and the gray mean N _j and variance Fj(u,j) of the right grid q are calculated; Step 204: Judging whether |Mi-Fj|<ε is true, if |Mi-Fj|<ε is true, then execute step 205; otherwise, set j=j+1 and return to step 203; where ε is a set value; Step 205: Calculate the sum of absolute differences between the left grid p and the right grid q, denoted as SAD; calculate the variance Ei(u,i) of the left grid p and the variance Fj(u ,j) of the absolute value of the difference, recorded as FCC; Step 206: Calculate the sum of the corrected absolute differences of the left grid p and the right grid q according to the formula SF=SAD×a+FCC×b, where a and b are scale parameters respectively; Step 207: If the sum of the corrected absolute differences of the left grid p and the right grid q is less than Min, then execute step 208; otherwise, set j=j+1 and return to step 203; Step 208: Let Min=SF, wj=j; judge whether the value of j passes through all epipolar points, if the value of j passes through all epipolar points, then execute step 209; otherwise, make j=j+1 and return Step 203; Step 209: The point (u, wj) in the grid area on the right is the matching point of the point (u, i) in the grid area on the left. 4. The method according to claim 3, characterized in that the ratio of the proportional parameters a and b is 1:1. 5. The method according to claim 1, characterized in that said step 5 comprises: Step 301: Use the formula D= _XL - _XR to calculate the parallax of the left and right images, where _XL is the abscissa of one point in the matching point pair, and _XR is the abscissa of the other point in the matching point pair; Step 302: Use the formula $\begin{array}{c}{x}_c=\frac{B{x}_L}{\mathrm{D.}}\\ {\mathrm{the\; y}}_c=\frac{\mathrm{BY}}{\mathrm{D.}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{D.}}\end{array}$ Calculate the three-dimensional coordinates of the point of the target crop corresponding to the matching point pair; where x _c , y _c and z _c are the three-dimensional coordinate values of the point of the target crop respectively, B is the distance between the optical axes of the left and right cameras, and X _L is the abscissa of a point in the matching point pair, Y is the ordinate of any point in the matching point pair, and f is the focal length of the left or right camera.

Images