CN104615150B - A kind of adaptive accurate spraying apparatus and method based on machine vision - Google Patents

Abstract

The invention relates to a self-adaptive precise positioning spraying device and method based on machine vision. Inside the body, the image acquisition module includes a main camera, an auxiliary camera and a supplementary light installed on the housing and connected to the core computing module. The spray module includes a medicine storage box, a nozzle and a nozzle installed on the mechanical arm. There are solenoid valves connected to the core computing module. The invention utilizes machine vision technology to acquire spray task target distribution information in real time during the operation of agricultural machinery, estimates the real-time movement trajectory of the machinery through the monocular camera trajectory reconstruction technology combined with the Kalman filter algorithm, and calculates the nozzle according to the movement trajectory information of the operating machinery The correct timing of spraying realizes precise variable spraying, improves the use efficiency of chemicals, and reduces environmental pollution.

Description

A self-adaptive precision spraying device and method based on machine vision

technical field

The invention belongs to the field of agricultural machinery automation, and relates to an automatic spraying device and method based on machine vision, in particular to an adaptive and precise positioning spraying device and method based on machine vision.

Background technique

Traditional agricultural spraying machinery is suitable for large-scale planting. It has many advantages such as economical, efficient, time-saving, and significantly reduces labor intensity and cost. However, it is difficult to accurately spray according to the growth and distribution of crops during the operation, resulting in pesticides Or a large amount of waste of fertilizers, resulting in high production costs and various problems caused by pollution residues to agricultural products and the environment. With the development of machine vision technology and agricultural machinery technology, information acquisition and real-time analysis of field crop growth position and growth density through machine vision technology, and spray control through targeted variables are a trend in the development of agricultural machinery automation technology. Accurate positioning of spray targets and real-time variable spraying is a technical difficulty of automatic spraying equipment and methods based on machine vision, which is prominently reflected in the real-time and accurate speed estimation of spraying machinery. The existing methods are roughly divided into three categories: the first category is to use the average speed of the agricultural machinery at a slow speed as the reference speed in the entire spraying operation, so as to control the choice of spraying timing; the second category is to use GPS for positioning and obtain The relative real-time moving position of the spray device in the task; the third type is to use a special speed sensor for speed measurement.

The first type of scheme is easy to implement, but the target positioning error is relatively large in actual operation, and it often causes a lot of waste of spraying agents in the actual promotion of field operations. The second type of method, such as the offline analysis method provided by patent ZL02246301.1, first collects the location and density information of weeds in the field, and then makes a prescription map after offline analysis; when the specific agricultural machinery is spraying, the offline analysis information and location are sent to the sprayer device controller. Although this method solves the requirement of variable spraying offline, due to the limitation of the accuracy of GPS operation, it is only suitable for variable spraying operations with large particle sizes in large areas, and it is difficult to achieve more accurate target spraying of crops in the area in real time and accurately. . The third type of method, such as the patent ZL02829239.1, designed a rail vehicle weeding method, which uses a speed sensor for real-time speed measurement, and uses rail vehicles to reduce mechanical vibration and attitude changes during operation, so as to increase the measurement accuracy of the speed sensor , so that the spray timing control is more accurate. However, this method needs to set up a fixed track first, and the equipment cost is relatively high, and the cumulative error effect of the speed sensor in the long-term working process is difficult to eliminate, which will still have a great impact on the accuracy of the spraying operation.

Contents of the invention

Aiming at the deficiencies in the prior art, the present invention provides a self-adaptive precise positioning spraying device and method based on machine vision.

The present invention is achieved through the following technical solutions:

A self-adaptive and precise positioning spraying device based on machine vision, including a mechanical arm, a housing, a core computing module, an image acquisition module, and a spraying module; the housing is fixed on the mechanical arm; the core computing module is fixed in the housing ; The image acquisition module includes a main camera, an auxiliary camera and a supplementary light installed on the housing and connected to the core computing module; A solenoid valve connected to the core computing module is installed on the tube.

The core calculation module includes a distance parameter automatic calibration module, a target detection module, a speed detection module, a delay estimation module and a spray task execution module.

The frame rate of the main camera is 60 frames per second.

The frame rate of the auxiliary camera is 30 frames per second.

A machine vision-based self-adaptive precise positioning spraying method, comprising the following steps:

Step S1: Extract video frames through the main camera and the auxiliary camera respectively, perform feature extraction and feature matching on the target area of the picture after normalization, obtain the camera position difference in pixels from the parallax, and estimate the actual distance of the spray;

Step S2: Grid the target detection area in the video frame captured by the main camera, extract and quickly count the effective points in turn, perform target fusion on the grid unit according to the number of effective points, and analyze and judge the specific position and location of the spray target. Area, encapsulation to form a spray task;

Step S3: Obtain two adjacent video frames from the main camera, extract feature points from the target detection area in the image respectively, use the feature points to perform feature matching and purification, calculate the spatial transformation distance of the camera, and further calculate the The average moving speed within a video frame interval time, update and correct the Kalman filter with this speed information, and use the Kalman filter to estimate the moving speed of the camera every 20 frames on average;

Step S4: According to the spray task queue obtained by the target detection module and the camera movement speed estimated by the speed detection module, the method continuously traverses the task queue at a fixed cycle speed of 20ms, and continuously uses the current camera movement speed to accumulate and integrate the movement trajectory of a single task . Accurately calculate the exact execution time of the spraying task according to the interval distance set by the detection equipment and the spraying equipment and the accumulated task movement distance. Once the task reaches the spray interval set by the system, the spray task command is sent to the spray task execution module immediately, and the spray task is cleared from the task queue.

Step S5: By analyzing the task command sent, sending the set high and low level signals to control the opening or closing of the solenoid valve, and completing the opening or closing of the spraying operation.

Compared with the prior art, the present invention uses machine vision technology to obtain real-time distribution information of spraying task targets during the operation of agricultural machinery, and estimates the real-time moving trajectory of the machinery through the monocular camera trajectory reconstruction technology combined with the Kalman filter algorithm. The information of the moving track of the machine calculates the correct spraying timing of the nozzle to achieve precise variable spraying, thus solving the problem of spraying timing control in the existing field spraying technology of agricultural machinery, improving the use efficiency of chemicals, and reducing environmental pollution.

Description of drawings

Fig. 1 is a schematic diagram of the self-adaptive precision spray workflow of the present invention;

Fig. 2 is a perspective view of the self-adaptive precise spraying device of the present invention;

Fig. 3 is a structural schematic diagram of the self-adaptive precise spraying device of the present invention.

Among them, 1. Shell, 2. Core computing module, 3. Main camera, 4. Auxiliary camera, 5. Fill light, 6. Nozzle, 7. Nozzle, 8. Solenoid valve.

detailed description

The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

Fig. 2 and Fig. 3 are the three-dimensional view and structural diagram of the self-adaptive precise positioning spraying equipment based on machine vision of the present invention, including a mechanical arm, a housing 1 fixed on the mechanical arm, and a core computing module 2 fixed in the housing 1 , an image acquisition module and a spray module installed on the housing 1; the image acquisition module includes a main camera 3, an auxiliary camera 4 and a fill light 5 respectively connected to the core computing module 2, wherein the frame number of the main camera is 60 frames per second , The frame rate of the auxiliary camera is 30 frames per second; the spray module includes a medicine storage box installed on the mechanical arm, a nozzle 7 and a nozzle 6, and a solenoid valve 8 connected to the core computing module 2 is installed on the nozzle 7.

Fig. 1 is the working flow chart of the self-adaptive precise positioning spraying method based on machine vision of the present invention, the core calculation module 2 of the self-adaptive precise positioning spraying equipment includes a distance parameter automatic calibration module, a target detection module, a speed detection module, and a time delay estimation module and the spray task execution module. The method first automatically obtains the pixel distance between the detection unit and the spray unit at the specific installation height of the spray device. Then, by regularly gridding the video image, the spray target is quickly detected and a task queue is formed. The main camera is used to extract the feature points in the adjacent video images, and the pixel distance in the adjacent video frame interval of the camera is obtained through feature point matching, and the estimated moving speed of the camera at the specified time granularity is obtained by combining the Kalman filter technology. By accumulating points, the trajectory of the camera is reconstructed, so as to obtain the actual optimal spraying time of the agricultural machinery and realize the final precise spraying operation. The specific steps are as follows:

Step S1: The distance parameter automatic calibration module extracts video frames through the main camera 3 and the auxiliary camera 4 respectively, performs feature extraction and feature matching on the target area of the picture after normalization, obtains the camera position difference from the parallax, and estimates the actual distance of the spray. These include:

Step S11: first extract video frames through the main camera 3 and the auxiliary camera 4 respectively, extract feature points after normalization, and calculate their descriptors for the feature points;

Step S12: Use the ratio method to perform fast feature point matching, and then use the RANSAC algorithm to purify the matching result to obtain matching feature point pairs with higher reliability;

Step S13: Use the matching feature point pairs to obtain the average value of displacement D _cam caused by parallax, the formula is as follows:

Since the relative distance L between the main camera 3 and the auxiliary camera 4 is known, and the distance between the main camera and the sprinkler unit is 2L, the pixel distance from the target detection unit to the spraying device at the same focal length and height can be directly estimated

Step S2: The target detection module grids the target detection area in the video frame captured by the main camera 3, sequentially extracts and quickly counts effective points, performs target fusion on the grid units according to the number of effective points, and analyzes and judges the spray target The specific location and area are packaged to form a spray task. These include:

Step S21: extracting real-time detected video data through the main camera 3 . In order to speed up the speed of target extraction and the stability of positioning, the algorithm first grids the detection area, and uses the regular grid divided in the detection area as the basic unit of processing and judgment;

Step S22: In the grid unit, according to the formula

Quickly extract the target pixels, and count the number of target pixels in a single grid;

Step S23: Based on a single grid area, filter out units with too low effective point density, and count the effective point distribution centroid positions of the cells in the foreground grid whose effective point density meets the requirements. Let N be the number of effective pixels, with is the centroid of effective points in the foreground grid, the formula is as follows:

The adjacent foreground grid units are quickly fused through the seed filling algorithm, and the fused grid unit coverage area is used as a single spray target body coverage area. By accumulating the fused grid unit centroids, they are quickly summed and averaged. As the center of mass of the target area, Tx and Ty are the position of the center of mass of the fused target area, and M is the number of single-block fused foreground grid connected bodies. The formula for calculating the center of mass is as follows:

According to the current target detection state transition information, the spray task is packaged into a separate task node and pushed to the delay estimation module.

Step S3: The speed detection module obtains two adjacent video frames from the main camera 3, extracts feature points from the target detection area in the image respectively, uses the feature points to perform feature matching and purification, and calculates the spatial transformation distance of the camera, and further Calculate the average moving speed of the camera in a video frame interval, update and correct the Kalman filter with this speed information, and use the Kalman filter to estimate the moving speed of the camera every 20 frames on average. These include:

Step S31: Establish a linear motion model for the trajectory of the spraying equipment performing the spraying task. The algorithm first creates a Kalman filter with a time interval of 330ms;

Step S32: Use the main camera to acquire adjacent video image data, extract feature points in the specified target area, and describe them using descriptors;

Step S33: Purify by ratio method and RANSAC, set the minimum number of matching feature points, and randomly extract three non-collinear feature points from the purified feature point set;

Step S34: Since the image acquisition module is fixed relative to the mechanical arm, the extracted video images only have rotation and translation-based motion modes. The system can use the method of calculating motion parameters to describe the two-dimensional motion mode of the camera, so as to obtain the entire spray equipment Trajectories of motion during spraying operations. The Cartesian coordinate system is used here, so the rotation angles of the two coordinate axes of XY in plane motion are the same θ _X = θ _Y = θ, and the global motion of the camera can be estimated by using the six-parameter affine transformation model, where the starting position (X _T0 , Y _T0 ), target position (X _TP , Y _TP ), plane displacement (X _SP , Y _SP ), the specific formula is as follows:

According to the motion parameter model, the parameters are solved: scaling scale, rotation rotation, x-axis offset xtranslation, y-axis offset ytranslation. The extraction time of the known adjacent frame is t _f , and then the distance of the average moving pixel in the adjacent video frame interval is obtained, and the moving speed v=(xtranslate/t _f , ytranslate/t _f );

Step S35: Repeat the process from S32 to S34 to update the parameters of the Kalman filter with the calculated speed information in pixel units, and at the same time use the predictive model to estimate the camera movement speed value within a specified time interval.

Step S4: The time delay estimation module continuously traverses the task queue at a fixed cycle speed of 20 ms according to the spray task queue obtained by the target detection module and the moving speed of the main camera 3 estimated by the speed detection module, and continuously uses the current moving speed of the main camera 3 The trajectory of a single task is accumulated and integrated. Accurately calculate the exact execution time of the spraying task according to the interval distance set by the detection equipment and the spraying equipment and the accumulated task movement distance. Once the task reaches the spray interval set by the system, the spray task command is sent to the spray task execution module immediately, and the spray task is cleared from the task queue. These include:

Step S41: traverse the task queue with a fixed period T, take out a single task queue in turn, perform displacement integration and update parameters. The latest update time in the task node is t, the current pixel displacement accumulation is s, the latest speed estimate v in the speed detection module and the update time t ^cur of the current node, the specific update operation is as follows:

Step S42: Judging whether the current pixel distance s is close to the spray target range, once |s-2L|<ε (ε is the pixel distance vT/4) is met, send an instruction to turn on or off the spray to the spray task execution module, and at the same time set the task from Deleted from the task queue.

Step S5: The spraying task execution module analyzes the sent task command and sends the set high and low level signals to control the opening or closing of the solenoid valve 8 to complete the opening or closing of the spraying operation.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, anyone familiar with the technology can ideally obtain the transformation and modification within the technical scope disclosed in the present invention , should also be deemed to be covered within the protection scope of the present invention.

Claims (1)

1. A method based on machine vision-based self-adaptive precision spray equipment, the self-adaptive precision spray equipment based on machine vision comprises a mechanical arm, a housing, a core computing module, an image acquisition module, and a spray module; the housing is fixed on a machine on the arm; the core computing module is fixed in the housing; the image acquisition module includes a main camera, an auxiliary camera and a supplementary light installed on the housing and connected to the core computing module; the spray module includes a The medicine storage box, the spray pipe and the nozzle on the spray pipe are installed with a solenoid valve connected to the core calculation module; the core calculation module includes a distance parameter automatic calibration module, a target detection module, a speed detection module, a time delay estimation module and Spray task execution module; the number of frames of the main camera is 60 frames per second, and the number of frames of the auxiliary camera is 30 frames per second; it is characterized in that: the spraying method comprises the following steps: Step S1: Extract video frames through the main camera and the auxiliary camera respectively, perform feature extraction and feature matching on the target area of the picture after normalization, obtain the camera position difference from the parallax, and estimate the actual distance of the spray; Step S2: Grid the target detection area in the video frame captured by the main camera, extract and quickly count the effective points in turn, perform target fusion on the grid unit according to the number of effective points, and analyze and judge the specific position and location of the spray target. Area, encapsulation to form a spray task; Step S3: Obtain two adjacent video frames from the main camera, extract feature points from the target detection area in the image respectively, use the feature points to perform feature matching and purification, calculate the spatial transformation distance of the camera, and further calculate the The average moving speed within a video frame interval time, update and correct the Kalman filter with this speed information, and use the Kalman filter to estimate the moving speed of the camera every 20 frames on average; Step S4: According to the spray task queue obtained by the target detection module and the camera movement speed estimated by the speed detection module, the method continuously traverses the task queue at a fixed cycle speed of 20ms, and continuously uses the current camera movement speed to accumulate and integrate the movement trajectory of a single task According to the interval distance set by the detection equipment and spray equipment and the task movement distance accumulated and integrated, the exact execution time of the spray task can be accurately calculated. Once the task reaches the spray interval distance set by the system, the spray task command will be sent to the spray task immediately. Execute the module and clear the spray task from the task queue; Step S5: By analyzing the task command sent, sending the set high and low level signals to control the opening or closing of the solenoid valve, and completing the opening or closing of the spraying operation.