CN104615150A - Machine vision based adaptive precise mist spray device and method - Google Patents

Abstract

The invention relates to machine vision based adaptive precise mist spray device and method. The device comprises a mechanical arm, a housing, a core calculating module, an image acquiring module and a mist spray module; the housing is fixed on the mechanical arm; the core calculating module is fixed into the housing; the image acquiring module comprises a main camera, a secondary camera and a light compensating lamp which are mounted on the housing and connected with the core calculating module; the mist spray module comprises a medicine storing box, a spraying pipe and a nozzle which are mounted on the mechanical arm; a solenoid valve which is connected with the core calculating module is mounted on the spraying pipe. According to the device, the machine vision technology is utilized, the mist spray task target distribution information is acquired on real time when operating an agricultural mechanism, the single camera path reconstruction technology and kalman filtering algorithm are combined and carried out to estimate the real-time moving track of the mechanism, then the accurate mist spray time of a spray head is calculated according to the moving track of the working mechanism, and therefore, the accurate variable mist spray can be achieved, the use efficiency of drugs can be increased, and the environmental pollution can be reduced.

Description

The accurate spraying apparatus of a kind of self-adaptation based on machine vision and method

Technical field

The invention belongs to agricultural machinery automatic field, relate to a kind of auto spraying Apparatus and method for based on machine vision, particularly relate to a kind of self-adaptation precise positioning spraying apparatus based on machine vision and method.

Background technology

Traditional agrosprayer tool is applicable to fairly large plantation, there is economy, efficiently, save time, the plurality of advantages such as remarkable reduction labour intensity and cost, but be difficult to precisely spray according to the growth of crop and distribution situation in operation process, thus cause a large amount of wastes of agricultural chemicals or fertilizer, cause production cost high and pollute remain the variety of problems brought to agricultural product and environment.Along with the development of machine vision technique and agricultural mechanical technology, by machine vision technique, acquisition of information and real-time analysis are carried out to field crops growth position and stand density, and realize by target variable the trend that spraying control is the technical development of agricultural machinery automatic field.Realize the spraying precise positioning of target and real-time variable spraying is technological difficulties of the auto spraying Apparatus and method for based on machine vision, in real time accurately in velocity estimation of the present spraying mechanical of excrescence.Existing method is divided three classes substantially: the first kind be use farm machinery at a slow speed in situation average velocity as the reference velocity in whole spraying operation, control the selection on spraying opportunity with this; Equations of The Second Kind uses GPS to position, the shift position relatively in real time of the spraying plant in acquisition task; 3rd class uses special speed pickup to test the speed.

First kind scheme realizes simple, but target location error is comparatively large in actual motion, often causes a large amount of wastes of spray medicine medicament in the actual popularization of farm work.Equations of The Second Kind method, as the off-line analysis method that patent ZL02246301.1 provides, first gathers weeds in field position and density information, makes prescription map after off-line analysis; During concrete agricultural machinery spray operations, the analytical information of off-line and position are sent to spraying apparatus controller.Although the method solves the requirement of variable rate spray by off-line, owing to limitting by GPS homework precision, be only suitable for the variable rate spray operation compared with coarsegrain of large section, what be difficult to real-time and precise realizes target spraying more accurately to crop in section.3rd class methods, as patent ZL02829239.1 devises a kind of herbicidal methods of rail vehicle, Negotiation speed sensor carries out real-time velocity survey, operating mechanical shaking and posture changing is reduced with the use of rail vehicle, with the measuring accuracy of the sensor that gathers way, make spraying control on opportunity more accurate.But this method needs first to set up trapped orbit, and equipment cost is relatively high, and the cumulative errors impact that speed pickup produces in the long course of work is difficult to eliminate, and still can produce considerable influence to the order of accuarcy of spraying operation.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, provide a kind of self-adaptation precise positioning spraying apparatus based on machine vision and method.

The present invention is achieved by the following technical solutions:

Based on a self-adaptation precise positioning spraying apparatus for machine vision, comprise mechanical arm, housing, core calculations module, image capture module, spray module; Described housing is fixing on the robotic arm; Described core calculations module is fixed in housing; Described image capture module comprises and being arranged on housing and the main camera, auxiliary camera and the light compensating lamp that are connected with core calculations module; Described spray module comprises installation explosive box on the robotic arm, jet pipe and shower nozzle, jet pipe is provided with the solenoid valve be connected with core calculations module.

Described core calculations module comprises distance parameter automatic calibration module, module of target detection, velocity measuring module, time delay estimation module and spraying task execution module.

Described main camera frame number is 60 frames/second.

Described auxiliary camera frame number is 30 frames/second.

Based on a self-adaptation precise positioning spray method for machine vision, comprise the following steps:

Step S1: extract frame of video respectively by main camera and auxiliary camera, carry out feature extraction, characteristic matching to picture target area after normalization, the camera position obtained from parallax in units of pixel is poor, estimates the actual range of spraying;

Step S2: gridding is carried out in the object detection area in the frame of video capture main camera, extract successively and express statistic available point, carry out subject fusion according to available point number to grid cell, analyze the particular location and area that judge spraying target, encapsulation forms spraying task;

Step S3: obtain two adjacent frame of video from main camera, respectively from the object detection area extract minutiae image, unique point is utilized to carry out characteristic matching and purification, extrapolate the spatial alternation distance of camera, calculate the average translational speed of camera within frame of video interval time further, upgrade with this velocity information and revise Kalman filter, utilizing Kalman filter on average to estimate the translational speed of a camera every 20 frames;

Step S4: the camera translational speed that the spraying task queue obtained according to module of target detection and velocity measuring module estimate, method constantly travels through task queue with the speed of fixing 20ms one-period, constantly utilizes current camera translational speed cumulative integral to go out the movement locus of individual task.The task displacement that the spacing distance set according to checkout equipment and spraying apparatus and cumulative integral go out calculates the accurate execution time of spraying task accurately.Once task reaches default spray intervals distance, then send spraying task order immediately to spraying task execution module, and from task queue, remove this spraying task.

Step S5: by analyzing the task order sended over, the low and high level signal Controlling solenoid valve sending setting is opened or is closed, and completes unlatching or the closedown of spraying operation.

Compared with prior art, the present invention utilizes machine vision technique, Real-time Obtaining spraying task object distributed intelligence in farm machinery operational process, machinery motion track is in real time estimated by monocular cam trajectory reconstruction combine with technique Kalman filtering algorithm, the correct spraying opportunity of shower nozzle is extrapolated according to Work machine motion track information, realize variable rate spray accurately, thus solve the spraying control problem on opportunity existed in existing agricultural machinery field spray technology, improve the service efficiency of medicament, alleviate the pollution of environment.

Accompanying drawing explanation

Fig. 1 is the accurate spray operations schematic flow sheet of self-adaptation of the present invention;

Fig. 2 is the accurate spraying apparatus stereographic map of self-adaptation of the present invention;

Fig. 3 is the accurate spraying apparatus structural representation of self-adaptation of the present invention.

Wherein, 1, housing, 2, core calculations module, 3, main camera, 4, auxiliary camera, 5, light compensating lamp, 6, shower nozzle, 7, jet pipe, 8, solenoid valve.

Embodiment

Below in conjunction with drawings and Examples, technical scheme of the present invention is elaborated.

Fig. 2 and Fig. 3 is stereographic map and the structural representation of the self-adaptation precise positioning spraying apparatus based on machine vision of the present invention, comprises mechanical arm, fixing housing 1 on the robotic arm, the core calculations module 2 be fixed in housing 1, is arranged on image capture module on housing 1 and spray module; Image capture module comprises the main camera 3, auxiliary camera 4 and the light compensating lamp 5 that are connected with core calculations module 2 respectively, and wherein main camera frame number is 60 frames/second, auxiliary camera frame number is 30 frames/second; Spray module comprises installation explosive box on the robotic arm, jet pipe 7 and shower nozzle 6, jet pipe 7 is provided with the solenoid valve 8 be connected with core calculations module 2.

Fig. 1 is the self-adaptation precise positioning spray method workflow diagram based on machine vision of the present invention, and the core calculations module 2 of its self-adaptation precise positioning spraying apparatus comprises distance parameter automatic calibration module, module of target detection, velocity measuring module, time delay estimation module and spraying task execution module.The method is the pixel distance of detecting unit and sprayer unit under the concrete setting height(from bottom) of automatic acquisition spraying plant first.Then by rule gridding video image, detect spraying target fast, form task queue.Utilize the unique point in main camera extraction adjacent video image, obtain the pixel distance in camera adjacent video frame period by Feature Points Matching, and by obtaining the camera moving speed estimation value under fixed time granularity in conjunction with Kalman Filter Technology.By cumulative integral, reconstruct the movement locus of camera, thus obtain the actual best spraying opportunity of farm machinery, realize final accurate spraying operation.Its concrete steps are as follows:

Step S1: distance parameter automatic calibration module extracts frame of video respectively by main camera 3 and auxiliary camera 4, carries out feature extraction, characteristic matching after normalization to picture target area, obtains camera position difference from parallax, estimates the actual range of spraying.Comprising:

Step S11: first extract frame of video respectively by main camera 3 and auxiliary camera 4, after normalization, difference extract minutiae, calculates its descriptor to unique point;

Step S12: utilize ratioing technigue to carry out Feature Points Matching fast, then uses RANSAC algorithm purification matching result, obtains the matching characteristic point pair that confidence level is higher;

Step S13: utilize matching characteristic point to the moving average D asked for because parallax causes _cam, formula is as follows:

$D_{\mathrm{cam}}=\sqrt{{\left(\frac{\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{x}_i^a-x_i^b}{N}\right)}^2+{\left(\frac{\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{y}_i^a-y_i^b}{N}\right)}^2}.$

Due to known main camera 3, auxiliary camera 4 relative distance L, and main camera distance ejection head unit distance is 2L, directly can estimate the pixel distance under unified focal length and height that object detection unit arrives jetting system $T_d=D_{\mathrm{cam}}\frac{2L}{L}.$

Step S2: gridding is carried out in the object detection area in the frame of video that main camera 3 captures by module of target detection, extract successively and express statistic available point, according to available point number, subject fusion is carried out to grid cell, analyze the particular location and area that judge spraying target, encapsulation forms spraying task.Comprising:

Step S21: extract the video data detected in real time by main camera 3.In order to the stability of the speed and location of accelerating Objective extraction, algorithm first carries out gridding to surveyed area, and the regular grid divided in surveyed area is as process and the elementary cell that judges;

Step S22: in grid cell, according to formula

$\mathrm{Pix}=\left\{\begin{array}{c}\mathrm{255,2}G-R-B>\mathrm{Thr}\\ \mathrm{0,2}G-R-B<=\mathrm{Thr}\end{array}\right.$

Extract target pixel points fast, and add up single grid internal object pixel number;

Step S23: with single net region for benchmark, filters out the unit that available point density is too low, meets the requirements of the available point distribution centroid position in prospect grid in statistic unit in available point density.N is made to be valid pixel number, with for available point barycenter in prospect grid, formula is as follows:

$\left\{\begin{array}{c}\stackrel{\&OverBar;}{\mathrm{tx}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}_i\\ \stackrel{\&OverBar;}{\mathrm{ty}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{y}_i\end{array}\right.$

Adjacent prospect grid cell is merged fast by seed fill algorithm, using the grid cell overlay area after fusion as single spraying objective body overlay area, by the grid cell barycenter of cumulative fusion, it is sued for peace fast and is averaged the rear barycenter as target area, Tx and Ty is the centroid position merging target area, M is the number that monolithic merges prospect grid UNICOM body, and centroid calculation formula is as follows:

$\left\{\begin{array}{c}\mathrm{Tx}=\frac{1}{M}\underset{j=1}{\overset{M}{\mathrm{\&Sigma;}}}{\stackrel{\&OverBar;}{\mathrm{tx}}}_j\\ \mathrm{Ty}=\frac{1}{M}\underset{j=1}{\overset{M}{\mathrm{\&Sigma;}}}{\stackrel{\&OverBar;}{\mathrm{ty}}}_j\end{array}\right.$

According to current target detection state transition information, spraying task is packaged into independent task node, is pushed to time delay estimation module.

Step S3: velocity measuring module obtains two adjacent frame of video from main camera 3, respectively from the object detection area extract minutiae image, unique point is utilized to carry out characteristic matching and purification, extrapolate the spatial alternation distance of camera, calculate the average translational speed of camera within frame of video interval time further, upgrade with this velocity information and revise Kalman filter, utilizing Kalman filter on average to estimate the translational speed of a camera every 20 frames.Comprising:

Step S31: the driving trace performing spraying task for spraying apparatus sets up linear movement model, and first algorithm creates a time interval is the Kalman filter of 330ms;

Step S32: utilize main camera to obtain adjacent video view data, extract minutiae in desired target area, and use descriptor to be described;

Step S33: purified by ratioing technigue and RANSAC, setting smallest match feature point number, extracts the unique point of three not conllinear at random from the feature point set after purifying;

Step S34: because image capture module relative mechanical arm is fixed, only there is rotation in the video image extracted, translation is main mode of motion, system can adopt the two dimensional motion mode of method to video camera calculating kinematic parameter to be described, thus obtains the movement locus of whole spraying apparatus in spraying operation.Adopt rectangular coordinate system herein, so XY two identical θ of X-axis rotate angle in plane motion _x=θ _y=θ, can utilize six parameter affine transform models to calculate the global motion of camera, wherein reference position (X _t0, Y _t0), target location (X _tP, Y _tP), in-plane displancement (X _sP, Y _sP), concrete formula is as follows:

$\left\{\begin{array}{c}{X}_{\mathrm{TP}}=X_{T0}+Y_{\mathrm{SP}}*\sin \mathrm{\&theta;}+X_{\mathrm{SP}}*\cos \mathrm{\&theta;}\\ Y_{\mathrm{TP}}=Y_{T0}+Y_{\mathrm{SP}}*\cos \mathrm{\&theta;}-X_{\mathrm{SP}}*\sin \mathrm{\&theta;}\end{array}\right.$

According to kinematic parameter model, solve parameter: convergent-divergent scale, rotate rotation, x-axis side-play amount xtranslation, y-axis side-play amount ytranslation.The extraction time of known consecutive frame is t _f, and then obtain the distance of average mobile pixel in adjacent video frame period, movement speed v=(xtranslate/t _f, ytranslate/t _f);

Step S35: repeat the process of S32 to S34, with the velocity information renewal Kalman filter parameter under the pixel unit calculated, utilizes forecast model to estimate cam movement velocity amplitude in fixed time interval simultaneously.

Step S4: the spraying task queue that time delay estimation module obtains according to module of target detection and main camera 3 translational speed that velocity measuring module estimates, method constantly travels through task queue with the speed of fixing 20ms one-period, constantly utilizes current main camera 3 translational speed cumulative integral to go out the movement locus of individual task.The task displacement that the spacing distance set according to checkout equipment and spraying apparatus and cumulative integral go out calculates the accurate execution time of spraying task accurately.Once task reaches default spray intervals distance, then send spraying task order immediately to spraying task execution module, and from task queue, remove this spraying task.Comprising:

Step S41: travel through task queue with fixed cycle T, takes out individual task queue successively, carries out displacement integration and undated parameter.Wherein in task node, the last update time is t, and current pixel Displacement Cumulative is s, velocity estimation value v nearest in velocity measuring module and t update time of current node ^cur, concrete renewal rewards theory is as follows:

$\left\{\begin{array}{c}s=s+(t^{\mathrm{cur}}-t)v\\ t=t^{\mathrm{cur}}\end{array}\right.$

Step S42: judge that whether current pixel distance s is close to spraying target zone, once meet | s-2L|< ε (ε is pixel distance vT/4), sends to spraying task execution module and opens or close spraying instruction, this task deleted from task queue simultaneously.

Step S5: spraying task execution module is by analyzing the task order sended over, and the low and high level signal Controlling solenoid valve 8 sending setting is opened or closed, and completes unlatching or the closedown of spraying operation.

The above; be only the embodiment in the present invention, but protection scope of the present invention is not limited thereto, any people being familiar with this technology is in the technical scope disclosed by the present invention; conceivable conversion and remodeling can be managed, also should be considered as covereding within protection scope of the present invention.

Claims (5)

1. based on a self-adaptation precise positioning spraying apparatus for machine vision, it is characterized in that: comprise mechanical arm, housing, core calculations module, image capture module, spray module; Described housing is fixing on the robotic arm; Described core calculations module is fixed in housing; Described image capture module comprises and being arranged on housing and the main camera, auxiliary camera and the light compensating lamp that are connected with core calculations module; Described spray module comprises installation explosive box on the robotic arm, jet pipe and shower nozzle, jet pipe is provided with the solenoid valve be connected with core calculations module.

2. according to claim 1 based on the self-adaptation precise positioning spraying apparatus of machine vision, it is characterized in that: described core calculations module comprises distance parameter automatic calibration module, module of target detection, velocity measuring module, time delay estimation module and spraying task execution module.

3. according to claim 1 based on the self-adaptation precise positioning spraying apparatus of machine vision, it is characterized in that: described main camera frame number is 60 frames/second.

4. according to claim 1 based on the self-adaptation precise positioning spraying apparatus of machine vision, it is characterized in that: described auxiliary camera frame number is 30 frames/second.

5. according to any one of Claims 1 to 4 based on the spray method of the self-adaptation precise positioning spraying apparatus of machine vision, it is characterized in that: comprise the following steps:

Step S1: extract frame of video respectively by main camera and auxiliary camera, after normalization, feature extraction, characteristic matching are carried out to picture target area, obtain camera position difference from parallax, estimate the actual range of spraying;

Step S2: gridding is carried out in the object detection area in the frame of video capture main camera, extract successively and express statistic available point, carry out subject fusion according to available point number to grid cell, analyze the particular location and area that judge spraying target, encapsulation forms spraying task;

Step S3: obtain two adjacent frame of video from main camera, respectively from the object detection area extract minutiae image, unique point is utilized to carry out characteristic matching and purification, extrapolate the spatial alternation distance of camera, calculate the average translational speed of camera within frame of video interval time further, upgrade with this velocity information and revise Kalman filter, utilizing Kalman filter on average to estimate the translational speed of a camera every 20 frames;

Step S4: the camera translational speed that the spraying task queue obtained according to module of target detection and velocity measuring module estimate, method constantly travels through task queue with the speed of fixing 20ms one-period, constantly utilizes current camera translational speed cumulative integral to go out the movement locus of individual task.The task displacement that the spacing distance set according to checkout equipment and spraying apparatus and cumulative integral go out calculates the accurate execution time of spraying task accurately.Once task reaches default spray intervals distance, then send spraying task order immediately to spraying task execution module, and from task queue, remove this spraying task.

Step S5: by analyzing the task order sended over, the low and high level signal Controlling solenoid valve sending setting is opened or is closed, and completes unlatching or the closedown of spraying operation.