CN109059869B - Method for detecting spraying effect of plant protection unmanned aerial vehicle on fruit trees - Google Patents
Method for detecting spraying effect of plant protection unmanned aerial vehicle on fruit trees Download PDFInfo
- Publication number
- CN109059869B CN109059869B CN201810847304.XA CN201810847304A CN109059869B CN 109059869 B CN109059869 B CN 109059869B CN 201810847304 A CN201810847304 A CN 201810847304A CN 109059869 B CN109059869 B CN 109059869B
- Authority
- CN
- China
- Prior art keywords
- target
- spraying
- sprayed
- aerial vehicle
- unmanned aerial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005507 spraying Methods 0.000 title claims abstract description 156
- 238000000034 method Methods 0.000 title claims abstract description 69
- 230000000694 effects Effects 0.000 title claims abstract description 21
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 14
- 239000003814 drug Substances 0.000 claims abstract description 66
- 238000012545 processing Methods 0.000 claims abstract description 30
- 238000002156 mixing Methods 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims abstract description 4
- 238000012937 correction Methods 0.000 claims description 23
- 239000000575 pesticide Substances 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 15
- 239000000049 pigment Substances 0.000 claims description 14
- 230000007613 environmental effect Effects 0.000 claims description 13
- 238000005562 fading Methods 0.000 claims description 11
- 239000003795 chemical substances by application Substances 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000002310 reflectometry Methods 0.000 claims description 5
- 238000013519 translation Methods 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 16
- 238000011897 real-time detection Methods 0.000 abstract description 2
- 239000007788 liquid Substances 0.000 description 13
- 230000009466 transformation Effects 0.000 description 12
- 239000003086 colorant Substances 0.000 description 6
- 239000007921 spray Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 239000000126 substance Substances 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 239000002245 particle Substances 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 239000003595 mist Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 229910052704 radon Inorganic materials 0.000 description 2
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 description 2
- -1 raindrops Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000003756 stirring Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 238000000889 atomisation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- SUKJFIGYRHOWBL-UHFFFAOYSA-N sodium hypochlorite Chemical compound [Na+].Cl[O-] SUKJFIGYRHOWBL-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/28—Measuring arrangements characterised by the use of optical techniques for measuring areas
- G01B11/285—Measuring arrangements characterised by the use of optical techniques for measuring areas using photoelectric detection means
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Catching Or Destruction (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the field of plant protection spraying detection, in particular to a method for detecting the spraying effect of a plant protection unmanned aerial vehicle on fruit trees, wherein the unmanned aerial vehicle is provided with a machine vision camera device and a three-dimensional laser scanner, the method comprises the steps of carrying out color mixing treatment on a medicament to enable a spraying area of a spraying target to form a color difference with an area which is not sprayed with the medicament, then using the three-dimensional laser scanner to scan and establish a three-dimensional model of the whole tree crown of the target to be sprayed with the medicament, using the machine vision camera device to collect a visible light image of the spraying target, obtaining a spraying area according to the color difference generated when the medicament is sprayed on the target to be sprayed, then carrying out registration and fusion on the visible light image and the three-dimensional model of the tree crown of the target to be sprayed with the medicament, processing the fused image, and respectively calculating the areas of the target to. The method can realize real-time detection, and has the characteristics of high detection speed, small dependence on environment and the like.
Description
Technical Field
The invention relates to the field of plant protection spraying detection, in particular to a method for detecting the spraying effect of a plant protection unmanned aerial vehicle on fruit trees.
Background
At present, the detection method of the liquid medicine spraying effect of the plant protection unmanned aerial vehicle mainly comprises two methods: one is the airborne machine vision detection method, and this method adopts the scheme that unmanned aerial vehicle carries on the industrial camera, shoots the liquid medicine and atomizes the overall process that the liquid drop that freely scatters in the air after spouting from plant protection unmanned aerial vehicle hits target or ground, draws the aerial flight orbit of atomizing back droplet crowd from it, and the arrival region of spraying liquid medicine is predicted and judged. The detection method cannot overcome the problems that the particle size of atomized droplets of the spraying liquid medicine is too small, the liquid medicine is colorless and transparent liquid, the spray disappears immediately after hitting a spraying target and the like, so that the scattering condition of the atomized liquid medicine cannot be accurately detected, the detection error is extremely large, and the detection of a liquid medicine spraying area cannot be mentioned.
The other is a method for laying moisture-sensitive paper, namely a method for laying moisture-sensitive paper on a spraying target, which can be used for analyzing the dispersion condition of fog drops hitting the target after spraying liquid medicine atomization, but the method is an indirect detection method, so that an online detection result cannot be returned, and the detection is long in time consumption.
And because of lacking this technical link of real-time detection to its spraying effect among the current plant protection unmanned aerial vehicle liquid medicine spraying control method, plant protection unmanned aerial vehicle liquid medicine spraying control method is forced to use open loop control as the main, can only adopt the single spraying scheme of continuous incessant spraying, because open loop control to the poor reason of the accurate spraying effect of target, can cause very big liquid medicine extravagant, environmental pollution and inefficiency spraying scheduling problem.
Disclosure of Invention
The invention aims to overcome at least one defect in the prior art, and provides a method for detecting the spraying effect of a plant protection unmanned aerial vehicle on fruit trees, which can detect the spraying effect in real time, has small dependence on the environment and high detection precision.
In order to achieve the purpose, the invention adopts the technical scheme that:
the method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit trees is provided, wherein the unmanned aerial vehicle is provided with a machine vision camera device and a three-dimensional laser scanner, and the method comprises the following steps:
s1: carrying out color matching treatment on the medicament to enable the color-matched medicament and a target to be sprayed to form color difference, and placing the color-matched medicament on an unmanned aerial vehicle;
s2: hovering an unmanned aerial vehicle above a target to be sprayed with pesticide, performing autorotation for one circle, acquiring point information of the surface of the whole tree crown of the target to be sprayed with pesticide by using a three-dimensional laser scanner on the unmanned aerial vehicle, and establishing a three-dimensional model of the whole tree crown of the target to be sprayed with pesticide according to the acquired point information, so as to obtain the contour of the canopy of the target to be sprayed with pesticide;
s3: spraying the medicament subjected to color matching treatment on a target to be sprayed with the medicament by using a spraying device on the unmanned aerial vehicle to obtain a spraying target sprayed with the medicament, wherein a spraying area on the spraying target forms a color difference with an area without spraying the medicament;
s4: after spraying, flying the unmanned aerial vehicle 360 degrees around a spraying target, acquiring a visible light image of the spraying target through a machine vision camera device on the unmanned aerial vehicle, acquiring a spraying area according to a color difference generated when a medicament is sprayed on the target (5) to be sprayed, registering and fusing the visible light image and a three-dimensional model of the crown of the target to be sprayed, processing the fused image, and calculating the area of the target to be sprayed; distinguishing color differences on the fused image, and calculating the area of a spraying area so as to obtain a spraying occupation ratio; the position of the spraying area on the visible light image can be distinguished according to the color because the color of the medicament is different from that of the spraying target.
There are generally two types of methods for visually expressing spatial information: one is an image or photo based approach; the other is a geometry-based method, namely, a three-dimensional model is established by acquiring point cloud data through a three-dimensional laser scanner. The working principle of the geometry-based method is as follows: the method comprises the steps that a laser range finder on the three-dimensional laser scanner actively emits laser pulse signals, meanwhile, signals reflected by the surface of a measured target are received for ranging, three-dimensional point cloud data of the surface of the measured target are obtained, and the point cloud data are subjected to noise removal, multi-view alignment, data simplification, curved surface reconstruction and the like, so that a three-dimensional model of the surface of the measured target is obtained. The noise removal means removing data except the surface of a detected target in the point cloud data, namely removing background point cloud data; the multi-view alignment refers to scanning the surface of the measured object for multiple times from multiple viewing angles, so as to avoid that all data cannot be measured at one time due to the fact that the shape of the surface of the measured object is too complex.
In the scheme, firstly, the medicament is subjected to color mixing, so that the medicament after color mixing and the target to be sprayed form obvious color difference, the distribution of the sprayed medicament is convenient to distinguish through color, after color mixing, the medicament is placed on the unmanned aerial vehicle, then the unmanned aerial vehicle flies for 360 degrees around the target to be sprayed, point information of the surface of the whole tree crown to be sprayed is collected through a three-dimensional laser scanner on the unmanned aerial vehicle, the obtained point information is subjected to noise removal, multi-view alignment, data simplification, curved surface reconstruction and other processing, a three-dimensional model of the whole tree crown to be sprayed is obtained, the three-dimensional model is divided into a plurality of regions, extreme value edges of each region can be obtained through simple comparison, then all adjacent extreme value edges are found out according to the connectivity of the extreme value edges, the contour line of the three-dimensional model can be obtained, and the contour line of the tree crown layer of the target to be.
The color-mixing medicament and the target to be sprayed have obvious color difference, so that the spraying area and the non-spraying area on the spraying target also have obvious color difference, and the visible light image is processed according to the color difference between the spraying area and the non-spraying area to obtain a spraying area; and adjusting the spraying device to spray the non-spraying area according to the acquired spraying proportion and the distribution of the medicament on the spraying target, thereby reducing the spraying error. The three-dimensional laser scanner can quickly establish a three-dimensional visual model with a complex structure for the acquired point information, so that the measurement time is saved, the detection speed is improved, in addition, as the spraying area and the non-spraying area on the spraying target have obvious color difference, and the machine vision camera device has high resolution, the distribution of the spraying agent can be distinguished from the collected visible light image even in foggy weather, and the method has small dependence on the environment.
Preferably, in step S1, the color of the medicine is adjusted by adding a pigment capable of fading automatically to the medicine. The automatic color fading pigment is added into the pesticide, the stirring is uniform, the pesticide is enabled to change color, the color-mixed pesticide is sprayed on a target to be sprayed with the pesticide, the spraying target sprayed with the pesticide is obtained, the spraying area of the spraying target is dyed with the color, and after the color of the spraying area lasts for a period of time, the color completely fades, so that the pigment cannot influence the spraying target.
Further preferably, in the step S1, the color of the automatically faded pigment is red or blue. The color of the target to be sprayed is green, so that the color of the automatically fading pigment is preferably red or blue which is largely different from green, in order to distinguish the distribution of the toned agent on the target to be sprayed.
Preferably, before the step S2, a step of performing an initialization operation process on the three-dimensional laser scanner is further included. Checking the electric quantity of the three-dimensional laser scanner, establishing a scanned project file, carrying out initialization work processing on the three-dimensional laser scanner, adjusting the working parameters of the three-dimensional laser scanner to a required state, firstly selecting indoor operation or outdoor operation according to the scanning operation condition, then selecting resolution, quality and scanning area, then selecting color scanning, adjusting the three-dimensional laser scanner to a horizontal position, and preparing before scanning to ensure that the accuracy of the scanned point information is higher.
Preferably, the point information collected by the three-dimensional laser scanner in step S2 includes point three-dimensional coordinates, reflectivity, and crown texture information. The three-dimensional laser scanner carries out noise removal, multi-view alignment, data simplification, curved surface reconstruction and other processing on the acquired three-dimensional coordinates, reflectivity and tree crown texture information of the target to be sprayed, a three-dimensional model of the whole target tree crown to be sprayed is acquired, the point information acquired by scanning of the three-dimensional laser scanner is comprehensive, and the acquired three-dimensional model of the target tree crown to be sprayed is high in accuracy.
Preferably, before the step S4, the method further includes the steps of acquiring internal reference and external reference of the machine vision camera by a calibration method, and performing distortion correction on the visible light image acquired by the machine vision camera according to the internal reference and the external reference. Obtaining internal parameters such as focal length, image center, distortion coefficient and the like and external parameters such as a rotation matrix, a translation matrix and the like of the machine vision camera device by a calibration method, calibrating the machine vision camera device, and then carrying out distortion correction on the acquired visible light image according to a calibration result. Because the visible light image is easy to have different degrees of distortion such as color cast, blur, geometric distortion, geometric inclination and the like in the generation and transmission processes, the clearer and truer visible light image can be obtained by carrying out distortion correction on the visible light image. The calibration method can be checkerboard calibration.
Preferably, in step S4, the visible light image is acquired and then subjected to an environmental correction process, and then the visible light image after the environmental correction process is processed according to a color difference generated when a chemical is sprayed on the target (5) to be sprayed, so as to obtain a spraying area. Unmanned aerial vehicle is at the flight in-process, because of the interference of environmental factor, leads to the visible light image of gathering not clear, so need carry out the preliminary treatment to the visible light image, eliminates the influence of environmental factor.
Further preferably, in step S4, the environment correction processing includes obtaining a high frequency part and a low frequency part by applying multi-scale decomposition of wavelet transform to the visible light image, performing filtering processing on the high frequency part, and performing correction processing based on chromatic aberration on the low frequency part to obtain a clear visible light image. Visible light transmitted in the atmosphere is easily absorbed, scattered and reflected by cloud mist, raindrops, solid particles and the like, so that a visible light signal transmitted from the surface of a spraying target to a machine vision camera device is attenuated, in addition, background noise is added to the visible light signal by radiation energy scattered and diffused by the atmosphere, the contrast ratio of the spraying target and an environment background is reduced, and an obtained visible light image is deformed, blurred and distorted, so that necessary noise smoothing and background clutter suppression processing are required to be performed on the acquired visible light image. The method comprises the steps of carrying out median filtering processing of local overlapping reference transformation on a high-frequency part obtained by wavelet transformation of a visible light image to eliminate general noise and motion noise, carrying out low-frequency correction on a low-frequency part obtained by the wavelet transformation by adopting a color difference-based fuzzy correction method, effectively removing the influence of environmental colors on visible light imaging, carrying out shake compensation processing on the visible light image, namely processing a blurred image formed by shaking in a shooting process by adopting two-dimensional Fourier transformation, carrying out normalization and binary transformation on an obtained result, combining the actual situation of the blurred image, estimating the shake blur direction by adopting a maximum value (MRT) algorithm based on Radon transformation, and finally obtaining a clear visible light image.
Preferably, in step S4, the specific steps of registering and fusing the visible light image and the three-dimensional model of the target crown to be sprayed with the medicine are that SIFT feature extraction is performed on the visible light image of the spraying target and the three-dimensional model of the target crown to be sprayed with the medicine respectively, key points are searched in different feature spaces, matching points of the visible light image and the three-dimensional model of the target crown to be sprayed with the medicine are obtained by matching the key points, so that rotation and translation matrix parameters of each medicine spraying area on the visible light image and the whole three-dimensional model of the target crown to be sprayed with the medicine are obtained, then image fusion is performed on the visible light image and the three-dimensional model of the target crown to be sprayed with the medicine, and binarization processing is performed on the fused image according to the contour of the canopy of the target to be sprayed with the medicine to calculate the area of the target to be sprayed with the medicine; and carrying out color difference distinguishing on the fused image to calculate the area of a spraying area so as to obtain the spraying occupation ratio. Through SIFT feature extraction, matching and fusion processes, binarization processing, color difference distinguishing and other processes, the area of a target to be sprayed and the area of a spraying area are calculated, so that spraying effect information such as spraying proportion, spraying distribution, spraying error and the like is obtained, and reference data is provided for accurate spraying.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that: the detection method comprises the steps of carrying out color mixing treatment on a medicament in advance to enable a sprayed area and an un-sprayed area to form color difference, then scanning and establishing a three-dimensional model of the whole tree crown of a target to be sprayed by using a three-dimensional laser scanner, collecting a visible light image of a spraying target by using a machine vision camera device, and obtaining the spraying proportion, the spraying distribution and the spraying error of the medicament by analyzing and processing to realize the real-time monitoring of the medicament spraying effect; the three-dimensional laser scanner can quickly establish a three-dimensional visual model with a complex structure for the acquired point information, so that the measurement time is saved, and the detection speed is increased; in addition, as the sprayed area and the non-sprayed area on the spraying target have obvious color difference and the machine vision camera device has high resolution, the distribution of the sprayed pesticide can be distinguished from the collected visible light image even in foggy weather, and the method has small dependence on the environment.
Drawings
Fig. 1 is a schematic structural diagram of the unmanned aerial vehicle according to the present invention;
FIG. 2(a) is a schematic diagram of the three-dimensional laser scanner collecting information of a crown point of a target to be sprayed with a chemical according to the present invention;
FIG. 2(b) is a schematic diagram of the toning agent of the present invention being applied to a target to be sprayed;
fig. 2(c) is a schematic diagram of a machine vision camera device according to the present invention collecting visible light images of a spraying target;
FIG. 2(d) is a schematic representation of the complete fading of the hueing agent color on a spray target according to the present invention.
The reference numbers illustrate: 1, unmanned plane; 2, a spraying device; 3 machine vision camera device; 4, a three-dimensional laser scanner; 5, spraying a target; 6 spraying the target.
Detailed Description
In order to better understand the present invention, the following examples are further provided to illustrate the content of the present invention, but the present invention is not limited to the following examples.
Example 1
A method for detecting the spraying effect of a plant protection unmanned aerial vehicle on fruit trees is disclosed, as shown in figures 1 and 2, a machine vision camera device 3 and a three-dimensional laser scanner 4 are arranged on the unmanned aerial vehicle 1, and the method comprises the following steps:
s1: carrying out color matching treatment on the medicament to enable the color-matched medicament and the target 5 to be sprayed to form color difference, and placing the color-matched medicament on the unmanned aerial vehicle 1 to obtain the canopy contour of the target 5 to be sprayed;
s2: hovering the unmanned aerial vehicle 1 above a target to be sprayed with pesticide, autorotating the unmanned aerial vehicle for a circle, acquiring point information of the surface of the whole crown of the target 5 to be sprayed with pesticide by using a three-dimensional laser scanner 4 on the unmanned aerial vehicle 1, and establishing a three-dimensional model of the whole crown of the target 5 to be sprayed with pesticide according to the acquired point information;
s3: spraying the medicament subjected to color matching treatment on a target 5 to be sprayed by using a spraying device 2 on the unmanned aerial vehicle 1 to obtain a spraying target 6 sprayed with the medicament, wherein a spraying area on the spraying target 6 is different from an un-spraying area in color;
s4: after spraying, flying the unmanned aerial vehicle 1 around a spraying target by 360 degrees, acquiring a visible light image of the spraying target 6 through a machine vision camera device 3 on the unmanned aerial vehicle 1, acquiring a spraying area according to a color difference generated by spraying a medicament on the target 5 to be sprayed, registering and fusing the visible light image and a three-dimensional model of the crown of the target 5 to be sprayed, processing the fused image, and calculating the area of the target 5 to be sprayed; distinguishing color differences on the fused image, and calculating the area of a spraying area so as to obtain a spraying occupation ratio; since the color of the chemical is different from that of the spray target 6, the position of the spray area on the visible light image can be discriminated according to the color.
There are generally two types of methods for visually expressing spatial information: one is an image or photo based approach; the second is a geometry-based method, namely, a three-dimensional model is established by acquiring point cloud data through a three-dimensional laser scanner 4. The working principle of the geometry-based method is as follows: the laser range finder on the three-dimensional laser scanner 4 actively emits laser pulse signals, and simultaneously receives signals reflected by the surface of the measured target to perform range finding, so that three-dimensional point cloud data of the surface of the measured target is obtained, and the point cloud data is subjected to noise removal, multi-view alignment, data simplification, curved surface reconstruction and the like to obtain a three-dimensional model of the surface of the measured target. The noise removal means removing data except the surface of a detected target in the point cloud data, namely removing background point cloud data; the multi-view alignment refers to scanning the surface of the measured object for multiple times from multiple viewing angles, so as to avoid that all data cannot be measured at one time due to the fact that the shape of the surface of the measured object is too complex.
Firstly, mixing colors of the medicament, enabling the medicament after mixing colors and the target 5 to be sprayed to form obvious color difference, conveniently distinguishing the distribution of the sprayed medicament through colors, after mixing colors, placing the medicament on the unmanned aerial vehicle 1, then flying the unmanned aerial vehicle 1 for 360 degrees around the target 5 to be sprayed, acquiring point information of the surface of the whole tree crown of the target 5 to be sprayed through a three-dimensional laser scanner 4 on the unmanned aerial vehicle 1, carrying out noise removal, multi-view alignment, data simplification, curved surface reconstruction and other processing on the acquired point information, obtaining a three-dimensional model of the whole tree crown of the target 5 to be sprayed, dividing the three-dimensional model into a plurality of regions, obtaining extreme value edges of each region through simple comparison, finding out all adjacent extreme value edges according to the connectivity of the extreme value edges, obtaining the contour line of the three-dimensional model, and obtaining the contour line of the tree crown of the target 5 to be sprayed.
The color-mixed medicament and the target 5 to be sprayed have obvious color difference, so that the spraying area and the non-spraying area on the spraying target 6 also form obvious color difference, and the visible light image is processed according to the color difference of the spraying area and the non-spraying area to obtain a spraying area; and adjusting the spraying device 2 to spray the non-spraying area according to the acquired spraying proportion and the distribution of the medicament on the spraying target 6, thereby reducing the spraying error. The three-dimensional laser scanner 4 can quickly establish a three-dimensional visual model with a complex structure for the acquired point information, so that the measurement time is saved, the detection speed is improved, in addition, as the spraying area and the non-spraying area on the spraying target 6 have obvious color difference, and the machine vision camera 3 has high resolution, the distribution of the spraying agent can be distinguished from the acquired visible light image even in foggy weather, and the method has small dependence on the environment.
In step S1, a color fading process is performed on the medicine by adding a pigment capable of fading automatically to the medicine. The automatic color fading pigment is added into the pesticide, the stirring is uniform, the pesticide is enabled to change color, the color-mixed pesticide is sprayed on the pesticide target 5 to be sprayed, the spraying target 6 sprayed with the pesticide is obtained, the spraying area of the spraying target 6 is dyed with the color, and after the color of the spraying area lasts for a period of time, the color is completely faded, so that the pigment cannot influence the spraying target 6.
Specifically, in step S1, the color of the automatically faded pigment is red or blue. The pigment capable of automatically fading can be a pigment formed by mixing NaClO solution and common ink according to a ratio of 3: 7; the color of the target to be sprayed 5 is green, so that the color of the automatically fading pigment is preferably red or blue which is largely different from green, in order to distinguish the distribution of the toned agent on the target to be sprayed 6.
Before step S2, the method further includes a step of performing an initialization operation process on the three-dimensional laser scanner 4. Checking the electric quantity of the three-dimensional laser scanner 4, establishing a scanned project file, initializing the three-dimensional laser scanner 4, adjusting the working parameters of the three-dimensional laser scanner 4 to a required state, firstly selecting indoor operation or outdoor operation according to the scanning operation condition, then selecting resolution, quality and scanning area, then selecting color scanning, adjusting the three-dimensional laser scanner 4 to a horizontal position, and preparing before scanning to ensure that the accuracy of the scanned point information is higher.
The point information collected by the three-dimensional laser scanner 4 in step S2 includes three-dimensional coordinates of points, reflectivity, and crown texture information. The three-dimensional laser scanner 4 carries out noise removal, multi-view alignment, data simplification, curved surface reconstruction and other processing on the acquired three-dimensional coordinates, reflectivity and tree crown texture information of the point of the target 5 to be sprayed, a three-dimensional model of the whole tree crown of the target 5 to be sprayed is acquired, the point information acquired by scanning of the three-dimensional laser scanner 4 is comprehensive, and the acquired three-dimensional model of the tree crown of the target 5 to be sprayed is high in accuracy.
Before the step S4, the method further includes a step of acquiring internal reference and external reference of the machine vision camera 3 by a calibration method, and performing distortion correction on the visible light image acquired by the machine vision camera 3 according to the internal reference and the external reference. Obtaining internal parameters such as a focal length, an image center, a distortion coefficient and the like of the machine vision camera device 3 and external parameters such as a rotation matrix, a translation matrix and the like by a calibration method, calibrating the machine vision camera device 3, and then carrying out distortion correction on the acquired visible light image according to a calibration result. Because the visible light image is easy to have different degrees of distortion such as color cast, blur, geometric distortion, geometric inclination and the like in the generation and transmission processes, the clearer and truer visible light image can be obtained by carrying out distortion correction on the visible light image. The calibration method can be checkerboard calibration.
In step S4, the visible light image is acquired and then subjected to environmental correction, and the visible light image after environmental correction is processed according to the color difference generated when the chemical is sprayed on the target 5 to be sprayed, so as to obtain a spraying area. In the flying process of the unmanned aerial vehicle 1, the acquired visible light image is not clear due to the interference of environmental factors, so that the visible light image needs to be preprocessed, and the influence of the environmental factors is eliminated.
Specifically, in step S4, the environment correction processing includes obtaining a high frequency part and a low frequency part by applying multi-scale decomposition of wavelet transform to the visible light image, performing filtering processing on the high frequency part, and performing correction processing based on chromatic aberration on the low frequency part to obtain a clear visible light image. Visible light transmitted in the atmosphere is easily absorbed, scattered and reflected by cloud mist, raindrops, solid particles and the like, so that a visible light signal transmitted from the surface of the spraying target 6 to the machine vision image pickup device 3 is attenuated, in addition, background noise is added to the visible light signal by radiation energy scattered and diffused by the atmosphere, the contrast ratio of the spraying target 6 and an environment background is reduced, and an acquired visible light image is deformed, blurred and distorted, so that necessary noise smoothing and background clutter suppression processing are required to be performed on the acquired visible light image. The method comprises the steps of carrying out median filtering processing of local overlapping reference transformation on a high-frequency part obtained by wavelet transformation of a visible light image to eliminate general noise and motion noise, carrying out low-frequency correction on a low-frequency part obtained by the wavelet transformation by adopting a color difference-based fuzzy correction method, effectively removing the influence of environmental colors on visible light imaging, carrying out shake compensation processing on the visible light image, namely processing a blurred image formed by shaking in a shooting process by adopting two-dimensional Fourier transformation, carrying out normalization and binary transformation on an obtained result, combining the actual situation of the blurred image, estimating the shake blur direction by adopting a maximum value (MRT) algorithm based on Radon transformation, and finally obtaining a clear visible light image.
In step S4, the specific steps of registering and fusing the visible light image and the three-dimensional model of the crown of the target 5 to be sprayed are that SIFT feature extraction is performed on the visible light image of the spraying target 6 and the three-dimensional model of the crown of the target 5 to be sprayed respectively, key points are searched in different feature spaces, matching points of the visible light image and the three-dimensional model of the crown of the target 5 to be sprayed are aligned by using key point matching, so that rotation and translation matrix parameters of the three-dimensional model of each spraying area on the visible light image and the whole crown of the target 5 to be sprayed are obtained, then image fusion is performed on the visible light image and the three-dimensional model of the crown of the target 5 to be sprayed, and binarization processing is performed on the fused image according to the contour of the canopy of the target 5 to be sprayed so as to calculate the area of the target 5 to be sprayed; and carrying out color difference distinguishing on the fused image to calculate the area of a spraying area so as to obtain the spraying occupation ratio. Through SIFT feature extraction, matching and fusion processes, binarization processing, color difference distinguishing and other processes, the area of a target 5 to be sprayed and the area of a spraying area are calculated, so that spraying effect information such as spraying proportion, spraying distribution, spraying error and the like is obtained, and reference data is provided for accurate spraying.
The above examples of the present invention are merely examples for clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (8)
1. The method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit trees is characterized in that a machine vision camera device (3) and a three-dimensional laser scanner (4) are arranged on the unmanned aerial vehicle (1), and the method comprises the following steps:
s1: carrying out color matching treatment on the medicament to enable the color-matched medicament and a target (5) to be sprayed to form color difference, and placing the color-matched medicament on the unmanned aerial vehicle (1);
s2: hovering an unmanned aerial vehicle (1) above a target (5) to be sprayed with a medicine, autorotating the unmanned aerial vehicle for one circle, acquiring point information of the surface of the whole crown of the target (5) to be sprayed with the medicine through a three-dimensional laser scanner (4) on the unmanned aerial vehicle (1), and establishing a three-dimensional model of the whole crown of the target (5) to be sprayed with the medicine according to the acquired point information so as to obtain a canopy contour of the target (5) to be sprayed with the medicine;
s3: spraying the agent subjected to color mixing treatment on a target (5) to be sprayed by a spraying device (2) on an unmanned aerial vehicle (1) to obtain a spraying target (6) sprayed with the agent, wherein a spraying area and an un-spraying area on the spraying target (6) form a color difference;
s4: after spraying, flying an unmanned aerial vehicle (1) around a spraying target (6) for 360 degrees, acquiring a visible light image of the spraying target (6) through a machine vision camera device (3) on the unmanned aerial vehicle (1), acquiring a spraying area according to a color difference generated when a medicament is sprayed on the target (5) to be sprayed, registering and fusing the visible light image and a three-dimensional model of a crown of the target (5) to be sprayed, processing the fused image, and calculating the area of the target (5) to be sprayed; distinguishing color differences on the fused image, and calculating the area of a spraying area so as to obtain a spraying occupation ratio;
in the step S4, the specific steps of registering and fusing the visible light image and the three-dimensional model of the crown of the target (5) to be sprayed are that SIFT feature extraction is respectively carried out on the visible light image of the spraying target (6) and the three-dimensional model of the crown of the target (5) to be sprayed, key points are searched in different feature spaces, matching points of the visible light image and the three-dimensional model of the crown of the target (5) to be sprayed are obtained by matching the key points, so that rotation and translation matrix parameters of each spraying area on the visible light image and the whole three-dimensional model of the crown of the target (5) to be sprayed are obtained, then image fusion is carried out on the visible light image and the three-dimensional model of the crown of the target (5) to be sprayed, binarization processing is carried out on the fused image according to the contour of the canopy of the target (5) to be sprayed, so as to calculate the area of the target (5) to be sprayed; and carrying out color difference distinguishing on the fused image to calculate the area of a spraying area so as to obtain the spraying occupation ratio.
2. The method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit tree as claimed in claim 1, wherein in the step S1, the color of the pesticide is adjusted by adding a pigment capable of fading automatically into the pesticide.
3. The method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit tree as claimed in claim 2, wherein in the step S1, the color of the automatically fading pigment is red or blue.
4. The method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit tree according to claim 1, wherein before the step S2, the method further comprises a step of performing initialization work processing on the three-dimensional laser scanner (4).
5. The method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit tree as claimed in claim 1, wherein the point information collected by the three-dimensional laser scanner (4) in the step S2 includes three-dimensional coordinates, reflectivity and crown texture information of the point.
6. The method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit tree according to claim 1, wherein before the step S4, the method further comprises the steps of obtaining the internal parameter and the external parameter of the machine vision camera (3) through a calibration method, and performing distortion correction on the visible light image collected by the machine vision camera (3) according to the internal parameter and the external parameter.
7. The method for detecting the spraying effect of the plant protection unmanned aerial vehicle on the fruit tree as claimed in claim 1, wherein in step S4, the visible light image is obtained and then is subjected to the environmental correction process, and then the visible light image after the environmental correction process is processed according to the color difference generated by spraying the pesticide on the target (5) to be sprayed, so as to obtain the spraying area.
8. The method of claim 7, wherein in step S4, the environmental correction process includes obtaining a high frequency part and a low frequency part by multi-scale decomposition of wavelet transform on the visible light image, filtering the high frequency part, and performing a correction process based on color difference on the low frequency part to obtain a clear visible light image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810847304.XA CN109059869B (en) | 2018-07-27 | 2018-07-27 | Method for detecting spraying effect of plant protection unmanned aerial vehicle on fruit trees |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810847304.XA CN109059869B (en) | 2018-07-27 | 2018-07-27 | Method for detecting spraying effect of plant protection unmanned aerial vehicle on fruit trees |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109059869A CN109059869A (en) | 2018-12-21 |
| CN109059869B true CN109059869B (en) | 2020-07-21 |
Family
ID=64835818
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810847304.XA Active CN109059869B (en) | 2018-07-27 | 2018-07-27 | Method for detecting spraying effect of plant protection unmanned aerial vehicle on fruit trees |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109059869B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111008983A (en) * | 2019-12-06 | 2020-04-14 | 国网湖南省电力有限公司 | Crown multi-threshold segmentation method and system applicable to power transmission line tree height monitoring device |
| US20210176912A1 (en) * | 2019-12-16 | 2021-06-17 | Cnh Industrial America Llc | System and method for assessing agricultural operation performance based on image data of processed and unprocessed portions of the field |
| CN113331167B (en) * | 2021-08-02 | 2021-11-19 | 南京卡飞软件技术有限公司 | Control system for spraying pesticide in agricultural greenhouse and spraying method thereof |
| CN113822987B (en) * | 2021-09-22 | 2024-07-02 | 杭州趣村游文旅集团有限公司 | Automatic color matching method between adjacent three-dimensional live-action models |
| CN115184372B (en) * | 2022-07-13 | 2023-04-18 | 水利部交通运输部国家能源局南京水利科学研究院 | Intelligent detection device and method for micro-crack fluorescence permeation of inaccessible part of concrete structure |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000280979A (en) * | 1999-03-30 | 2000-10-10 | Kawasaki Heavy Ind Ltd | High speed planing boat |
| WO2011015989A1 (en) * | 2009-08-06 | 2011-02-10 | S.I.A. Società Idee Avioniche S.R.L. | A washing unit for turboprops of aircrafts |
| CN102800083A (en) * | 2012-06-19 | 2012-11-28 | 中国农业大学 | Crop spraying positioning method based on binocular vision gridding partition matching algorithm |
| DE202015000218U1 (en) * | 2015-01-12 | 2016-04-13 | Volker Kruse | Device for cleaning wall and bundle heating surfaces of an incinerator online |
| CN105753509A (en) * | 2016-02-01 | 2016-07-13 | 浙江大学 | Humidity sensing ceramic, and preparation method and application thereof |
| CN106774465A (en) * | 2016-12-14 | 2017-05-31 | 江苏蒲公英无人机有限公司 | A kind of unmanned aerial vehicle (UAV) control method for fruit tree spray |
| CN106814745A (en) * | 2017-03-22 | 2017-06-09 | 常熟理工学院 | UAV Intelligent medicine supplies spray method and system |
| CN106956778A (en) * | 2017-05-23 | 2017-07-18 | 广东容祺智能科技有限公司 | A kind of unmanned plane pesticide spraying method and system |
| JPWO2017081757A1 (en) * | 2015-11-10 | 2018-03-08 | 富士通フロンテック株式会社 | Paper sheet handling apparatus and paper sheet determination method |
| CN108090893A (en) * | 2017-11-23 | 2018-05-29 | 华南农业大学 | Plant protection unmanned plane medicament sprinkling optimization method and device based on thermal infrared identification |
| CN108255194A (en) * | 2018-01-08 | 2018-07-06 | 江苏工程职业技术学院 | A kind of agricultural plant protection unmanned plane precise positioning spraying control method based on UWB |
-
2018
- 2018-07-27 CN CN201810847304.XA patent/CN109059869B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000280979A (en) * | 1999-03-30 | 2000-10-10 | Kawasaki Heavy Ind Ltd | High speed planing boat |
| WO2011015989A1 (en) * | 2009-08-06 | 2011-02-10 | S.I.A. Società Idee Avioniche S.R.L. | A washing unit for turboprops of aircrafts |
| CN102800083A (en) * | 2012-06-19 | 2012-11-28 | 中国农业大学 | Crop spraying positioning method based on binocular vision gridding partition matching algorithm |
| DE202015000218U1 (en) * | 2015-01-12 | 2016-04-13 | Volker Kruse | Device for cleaning wall and bundle heating surfaces of an incinerator online |
| JPWO2017081757A1 (en) * | 2015-11-10 | 2018-03-08 | 富士通フロンテック株式会社 | Paper sheet handling apparatus and paper sheet determination method |
| CN105753509A (en) * | 2016-02-01 | 2016-07-13 | 浙江大学 | Humidity sensing ceramic, and preparation method and application thereof |
| CN106774465A (en) * | 2016-12-14 | 2017-05-31 | 江苏蒲公英无人机有限公司 | A kind of unmanned aerial vehicle (UAV) control method for fruit tree spray |
| CN106814745A (en) * | 2017-03-22 | 2017-06-09 | 常熟理工学院 | UAV Intelligent medicine supplies spray method and system |
| CN106956778A (en) * | 2017-05-23 | 2017-07-18 | 广东容祺智能科技有限公司 | A kind of unmanned plane pesticide spraying method and system |
| CN108090893A (en) * | 2017-11-23 | 2018-05-29 | 华南农业大学 | Plant protection unmanned plane medicament sprinkling optimization method and device based on thermal infrared identification |
| CN108255194A (en) * | 2018-01-08 | 2018-07-06 | 江苏工程职业技术学院 | A kind of agricultural plant protection unmanned plane precise positioning spraying control method based on UWB |
Non-Patent Citations (2)
| Title |
|---|
| Optimizing pesticide spray application in tree crops;Santiago Planas de Martí, Agr. Eng. PhD;《American society of agricultural and biological engineers》;20060712;全文 * |
| 航空施药雾滴沉积特性光谱分析检测系统研发与应用;张瑞瑞;《农业工程学报》;20171231;第33卷(第24期);第80-84页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109059869A (en) | 2018-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109059869B (en) | Method for detecting spraying effect of plant protection unmanned aerial vehicle on fruit trees | |
| Hyyppä et al. | Under-canopy UAV laser scanning for accurate forest field measurements | |
| CN108919838B (en) | Binocular vision-based automatic tracking method for power transmission line of unmanned aerial vehicle | |
| EP3446065B1 (en) | Flight parameter measuring apparatus and flight parameter measuring method | |
| CN107063949B (en) | Method, device and system for measuring droplet deposition distribution | |
| CN106969730B (en) | A kind of top fruit sprayer volume measuring method based on unmanned plane Detection Techniques | |
| Vandapel et al. | Natural terrain classification using 3-d ladar data | |
| CA2764135C (en) | Device and method for detecting a plant | |
| CN108828621A (en) | Obstacle detection and road surface partitioning algorithm based on three-dimensional laser radar | |
| Liu et al. | Evaluation of a laser scanning sensor in detection of complex-shaped targets for variable-rate sprayer development | |
| CA2721891C (en) | Optronic system and method dedicated to identification for formulating three-dimensional images | |
| CN110132226B (en) | System and method for measuring distance and azimuth angle of unmanned aerial vehicle line patrol | |
| CN107239748A (en) | Robot target identification and localization method based on gridiron pattern calibration technique | |
| CN108550141A (en) | A kind of movement wagon box automatic identification and localization method based on deep vision information | |
| CN112541921A (en) | Digitized accurate measuring method for urban green land vegetation information | |
| CN107329116B (en) | Airborne radar three-dimensional motion scene display method | |
| CN110031830B (en) | Distance measurement method based on laser line scanning imaging | |
| CN109816697B (en) | System and method for establishing map by unmanned model vehicle | |
| CN108168706A (en) | A kind of multispectral infrared imaging detecting and tracking system for monitoring low-altitude unmanned vehicle | |
| CN110413143A (en) | Man-machine interaction method based on laser radar | |
| CN114485477A (en) | Icing three-dimensional shape online measuring method and device | |
| CN109285141B (en) | Real-time detection method for spraying effect of plant protection unmanned aerial vehicle | |
| CN112308900B (en) | Four-rotor unmanned aerial vehicle relative pose estimation method based on LED (light emitting diode) ring detection | |
| Trosin et al. | Measuring soil surface roughness with the realsense d435i | |
| CN112298564A (en) | Variable pesticide application control method and device based on image recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |