CN107933921B

CN107933921B - Aircraft, spraying route generation and execution method and device thereof, and control terminal

Info

Publication number: CN107933921B
Application number: CN201711033665.2A
Authority: CN
Inventors: 刘波; 代双亮
Original assignee: Guangzhou Xaircraft Technology Co Ltd
Current assignee: Guangzhou Xaircraft Technology Co Ltd
Priority date: 2017-10-30
Filing date: 2017-10-30
Publication date: 2020-11-17
Anticipated expiration: 2037-10-30
Also published as: CN107933921A

Abstract

The invention provides an aircraft spraying route generation method, which comprises the following steps: acquiring image information of a target area, and constructing a three-dimensional map containing point cloud data according to the image information; determining the position information of all the target objects in the target area and the characteristic information of each target object according to the three-dimensional map; and generating an aircraft spraying route according to the position information and the characteristic information of all the target objects so as to instruct the aircraft to carry out spraying operation. The method is used for solving the problem that the mapping cost of the spraying operation route is high in the prior art, reducing the mapping time and the mapping cost of the spraying route and improving the efficiency of the aircraft in plant protection operation. The invention also provides a corresponding aircraft spraying route generating device and a control terminal. In addition, the invention also provides a corresponding aircraft and a spraying route execution method and device thereof.

Description

Aircraft, spraying route generation and execution method and device thereof, and control terminal

Technical Field

The invention relates to the technical field of aircrafts, in particular to an aircraft, a spraying route generating method and device of the aircraft, a corresponding spraying route executing method and device of the aircraft, and a control terminal.

Background

With the development of science and technology, aircrafts such as unmanned planes and the like are widely applied to various fields such as agriculture, military, industry and the like. With the maturity of the application of unmanned aerial vehicles in the field of plant protection, the demand of plant protection operation of cash crops is increasing, and to realize that the unmanned aerial vehicles are used to automatically spray pesticides on cash crops such as fruit trees, data information such as geographic positions, heights, central point longitudes and latitudes of all crops in an operation area needs to be acquired so as to generate operation routes. Among the prior art, the acquisition of above-mentioned data needs the operator to carry mapping device to walk round survey and drawing around the farmland usually and obtains, the mode of artifical survey and drawing promptly, and artifical survey and drawing link consumes time and manpower very much, artifical survey and drawing inefficiency and cost are higher, consequently spray the efficiency of operation and can't obtain guaranteeing, economic benefits is not high.

Disclosure of Invention

Based on the problems, the invention provides an aircraft, a spraying route generating method and device thereof, a corresponding aircraft spraying route executing method and device, and a control terminal, which are used for solving the problems of low spraying operation route mapping efficiency and high cost in the prior art, reducing the mapping time and the spraying route mapping cost and improving the efficiency of the aircraft in plant protection operation.

In order to achieve the purpose, the invention provides the following technical scheme:

the invention provides an aircraft spraying route generation method, which comprises the following steps: acquiring image information of a target area, and constructing a three-dimensional map containing point cloud data according to the image information; determining the position information of all the target objects in the target area and the characteristic information of each target object according to the three-dimensional map; and generating an aircraft spraying route according to the position information and the characteristic information of all the target objects so as to instruct the aircraft to carry out spraying operation.

Preferably, before the step of determining the position information of all the objects in the target area and the feature information of each object according to the three-dimensional map, the method further includes: a recognition engine is generated to determine the target location information and the characteristic information.

Specifically, the step of generating a recognition engine for determining the position information and the feature information of the target object specifically includes: acquiring multiple groups of data related to position information and characteristic information of the target object, and generating a target area model containing multiple target objects; a plurality of characteristic points which are associated with the characteristic information of each target object according to the target area model; and determining and generating the recognition engine according to the matching relationship between the position information of all the target objects and the point cloud data and the matching relationship between the plurality of groups of data corresponding to the characteristic information of all the target objects and all the characteristic points.

Further, the matching relation between the multiple groups of data corresponding to the feature information of all the target objects and all the feature points is determined through iterative operation.

Specifically, the step of determining the position information of all the objects in the object area and the feature information of each object according to the three-dimensional map specifically includes: the identification engine determines the position information of each target object according to the point cloud data; the recognition engine determines a plurality of feature points associated with each target object, and determines feature information of each target object according to the feature points corresponding to different target objects.

Specifically, the step of generating an aircraft spraying route according to the position information and the characteristic information of all the target objects to instruct the aircraft to carry out spraying operation includes: and determining the spraying operation information of the aircraft at each target object according to the characteristic information of each target object.

Preferably, the spraying operation information includes flight time, spraying amount and spraying range of the aircraft at each target object.

Preferably, the position information of the target object includes longitude and latitude of a center point of the target object and an altitude of the target object.

Preferably, the characteristic information of the target object comprises the relative height of the target object, the floor area of the target object and the effective spraying range of the target object.

Optionally, an RCNN (Regions with conditional Neural Network Features) algorithm, a Fast-RCNN algorithm, or a Fast-RCNN algorithm is used to generate the recognition engine for determining the object position information and the feature information.

Accordingly, the present invention provides an aircraft spray route generation apparatus comprising: the information processing module is used for acquiring image information of a target area and constructing a three-dimensional map containing point cloud data according to the image information; the information identification module is used for generating an identification engine and enabling the identification engine to determine the position information of all the target objects in the target area and the characteristic information of each target object according to the three-dimensional map; and the route generating module is used for generating an aircraft spraying route according to the position information and the characteristic information of all the target objects so as to instruct the aircraft to carry out spraying operation.

Specifically, the information identification module includes: and the identification engine generating module is used for generating an identification engine used for determining the position information and the characteristic information of the target object.

Specifically, the recognition engine generation module includes: a model generation unit for acquiring a plurality of sets of data on position information and feature information of the target object and generating a target area model including a plurality of target objects; a target unit, configured to target a plurality of feature points associated with feature information of each target object according to the target area model; and the execution unit determines and generates the identification engine according to the matching relationship between the position information of all the target objects and the point cloud data and the matching relationship between the plurality of groups of data corresponding to the characteristic information of all the target objects and all the characteristic points.

Specifically, the recognition engine specifically includes: the position information determining unit is used for determining the position information of each target object according to the point cloud data; and the characteristic information determining unit is used for determining a plurality of characteristic points associated with each target object and determining the characteristic information of each target object according to the characteristic points corresponding to different target objects.

Specifically, the route generation module includes: and the task configuration module is used for determining the spraying operation information of the aircraft at each target object according to the characteristic information of each target object.

Optionally, an RCNN algorithm, a Fast-RCNN algorithm or a Fast-RCNN algorithm is used to generate the recognition engine for determining the position information and the characteristic information of the target object.

Correspondingly, the invention also provides an aircraft spraying route execution method, which comprises the following steps: receiving spray route related information, the spray route related information being used for defining a route for the aircraft to carry out a spray operation; and starting the aircraft, and carrying out spraying operation according to the spraying route related information.

Specifically, the information associated with the aircraft performing the spraying operation includes the flight time, the spraying amount and the spraying range of the aircraft at each target object.

Specifically, the spraying route related information includes position information of the target object and feature information of the target object.

Further specifically, the position information of the target object includes a longitude and latitude of a center point of the target object and an altitude of the target object.

More specifically, the characteristic information of the target object comprises the relative height of the target object, the floor area of the target object and the effective spraying range of the target object.

Correspondingly, the invention also provides an aircraft spraying route execution device, which comprises: the information receiving module is used for receiving spraying route related information, and the spraying route related information is used for limiting a route of the aircraft for spraying operation; and the starting module is used for starting the aircraft and carrying out spraying operation according to the spraying route related information.

Correspondingly, the present invention also provides a control terminal, which is characterized by comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: carrying out the steps of the aircraft spray route generation method of any one of the preceding claims.

Accordingly, the present invention also provides an aircraft comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: carrying out the steps of the aircraft spray route execution method of any one of the preceding claims.

Compared with the prior art, the scheme of the invention has the following beneficial effects:

according to the method, the device and the control system for generating the spraying route of the aircraft, a three-dimensional map is constructed through the acquired image information of a target area, and the three-dimensional map comprises point cloud data, namely position information of all target objects in the target area; therefore, the position information of all the target objects and the characteristic information of each target object can be determined according to the image information and the point cloud data; and finally, correspondingly generating an aircraft spraying route according to the position information and the characteristic information of all the target objects, so that the aircraft can finish spraying operation conveniently. The whole process does not need to artificially survey and draw the position information and the characteristic information of the target object, so that the working efficiency of surveying and drawing the position information and the characteristic information of the target object is greatly improved, the surveying and drawing cost is reduced, and the efficiency of spraying operation implemented by the aircraft is improved.

The recognition engine can recognize all the target objects in the target area according to the acquired image information and the corresponding three-dimensional map, and can determine the position information and the characteristic information of each target object, so that for different target areas, only the image information corresponding to the different target areas needs to be acquired, and the generation of the spraying route can be facilitated.

The position information of the target object is determined by point cloud data in a three-dimensional map, and the position information of the target object generally refers to information corresponding to the position of the target object, such as longitude, latitude, altitude and the like of the position of the target object, and for a fixed target object, the position information is constant. The characteristic information of the target object is identified by the identification engine to determine, and the characteristic information of the target object generally refers to information such as a relative height of the target object, a floor area of the target object, an effective spraying range of the target object, and the like. Therefore, by determining the position information of each target object, each point on the spraying route of the aircraft in the target area can be planned according to the position information of the target object, and each point corresponds to one target object; by determining the characteristic information of each target object, the time length of the aircraft staying at each point, the spraying amount corresponding to each target object, the spraying range corresponding to each target object and the like can be determined, so that the spraying operation of the aircraft can be balanced.

The aircraft spraying route execution method and the aircraft spraying route execution device can improve the efficiency of the spraying operation of the aircraft by receiving the spraying route related information for limiting the route of the spraying operation carried out by the aircraft and carrying out the spraying operation according to the related information.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic flow chart diagram of an exemplary embodiment of an aircraft spray route generation method of the present invention;

FIG. 2 is a schematic flow chart diagram illustrating one exemplary embodiment of a method for generating an identification engine in the aircraft spray route generation method of the present invention;

FIG. 3 is a schematic flow chart diagram illustrating one exemplary embodiment of determining position information and feature information for a target object in the aircraft spray route generation method of the present invention;

FIG. 4 is a functional block diagram of an exemplary embodiment of an aircraft spray route generation arrangement of the present invention;

FIG. 5 is a functional block diagram of one exemplary embodiment of a recognition engine generation module in the aircraft spray route generation apparatus of the present invention;

FIG. 6 is a functional block diagram of an exemplary embodiment of an identification engine in the aircraft spray route generation apparatus of the present invention;

FIG. 7 is a schematic flow chart diagram of an exemplary embodiment of a method of performing a spray route for an aircraft of the present invention;

fig. 8 is a schematic block diagram of an exemplary embodiment of an aircraft spray route actuation device of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative only and should not be construed as limiting the invention.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

An aircraft generally refers to a mechanical flying object which is manufactured by human beings, can fly off the ground, fly in the air and is controlled by human beings, such as an aviation airplane, a helicopter, a unmanned aerial vehicle and the like, wherein the unmanned aerial vehicle is widely applied in many fields. The unmanned aerial vehicle is a small aircraft mainly controlled by radio remote control or self program, has the advantages of small volume, relatively low manufacturing cost, convenient control and the like, is often used in the fields of aerial photography, plant protection operation, environmental monitoring, military reconnaissance, disaster patrol, terrorism prevention, life saving and the like, can overcome the defects of the conventional aircraft such as a manually piloted aircraft in aerial operation, reduces the manufacturing cost and the maintenance cost, and simultaneously improves the carrying safety.

In the plant protection operation field, unmanned aerial vehicle is used for carrying out the pesticide spraying operation to plants such as trees that have the sick worm grass evil usually, has improved the efficiency of plant protection operation. Adopt manual control unmanned aerial vehicle to spray the operation usually and unmanned aerial vehicle automatic execution sprays operation two kinds of modes, obviously, the former relatively, the latter can improve work efficiency greatly, and the precision is also higher. Necessarily, to implement pesticide spraying operation on cash crops such as fruit trees and the like by using an unmanned aerial vehicle, data information such as geographic positions, heights, central point longitudes and latitudes of all crops in an operation area needs to be acquired so as to generate a spraying operation route. In the prior art, the data are usually obtained by manual surveying and mapping, the manual surveying and mapping link consumes time and labor, the surveying and mapping cost is high, and the improvement of the plant protection operation efficiency is extremely limited.

Referring to fig. 1, the present invention provides a method for generating a spraying route of an aircraft, comprising the following steps:

step S11: acquiring image information of a target area, and constructing a three-dimensional map containing point cloud data according to the image information.

For performing a spraying operation using an aircraft, the target area is an area where the spraying operation is to be performed, such as a farmland, a hill, or the like, in which crops such as fruit trees, rice, flowers, or the like are planted. The image information comprises pictures, image data and the like used for recording a target area, the image information of the target area can be shot by the unmanned aerial vehicle which is specially used for shooting a high-definition map and geographic mapping, and a background system of the unmanned aerial vehicle can generate three-dimensional point cloud. The point cloud generally refers to a point set containing coordinate information, and accordingly, the point cloud data contained in the three-dimensional map represents coordinate information of all points in the point set, including longitude, latitude, altitude, and the like. The coordinate information of the target object at different positions is fixed and unique. For example, for trees planted on a hillside, the longitude and latitude of trees at the same altitude are different, the altitude and latitude of trees at the same longitude are different, and the altitude and longitude of trees at the same latitude are different. The three-dimensional map includes specific shapes and corresponding data information of all the objects, rather than simplifying the objects into one coordinate point, so that each object can be conveniently identified and distinguished in the subsequent steps. The selection rule of the center point of the target object may be defined according to different requirements, for example, the center point of the target object may be the center point of the target object, or the geometric center of the target object may be the center point of the target object, or a feature object shared by all the target objects may be the center point of the target object. In the same target area coordinate system, all the targets need to follow the same rule to determine the center point of each target, so as to reduce systematic errors.

Of course, besides the point cloud data, a fixed reference object may be selected for the target area, and a spatial coordinate system of the target area is constructed based on the reference object, and since the fixed reference object is selected as the reference point and the relative position of each target object and the reference object in the target area is fixed, the specific position of each target object in the spatial coordinate system can be determined accordingly.

Step S12: and determining the position information of all the target objects in the target area and the characteristic information of each target object according to the three-dimensional map.

The position information of the target corresponds to the above-described coordinate information, i.e., point cloud data. The point cloud data is contained in the three-dimensional map, so that the point cloud data can be analyzed and presented only by acquiring the three-dimensional map.

Characteristic information of the target includes, but is not limited to, the relative height of the target, the footprint of the target, and the effective spray range of the target. For example, when the target object is a fruit tree, the relative height refers to the height of the vertex of the fruit tree relative to the ground, and the maximum height of the unmanned aerial vehicle flying at the position of the fruit tree can be set through the index; the occupied area of the fruit tree is not limited to the ground area occupied by the root of the fruit tree, but also comprises the maximum cross-sectional area of the crown of the fruit tree, namely the orthographic projection area of the crown on the ground, the size of the orthographic projection area can be determined by determining the boundary of the crown, generally, the orthographic projection of the crown on the ground can also be regarded as a circle, so that the orthographic projection area of the crown on the ground can be determined by measuring the radius of the crown, and the spraying area of the unmanned aerial vehicle at the position of the fruit tree can be determined by the index; the effective spraying range of the fruit tree refers to a part on the fruit tree, which needs to be sprayed, and is usually a part capable of efficiently absorbing nutrients and moisture, such as the root and the leaves of the fruit tree, and the spraying amount of the unmanned aerial vehicle at the position of the fruit tree can be determined through the index. When the target is a field crop (such as rice, wheat, etc.), the area of the field (corresponding to the footprint of the target), the spacing distance between different fields (corresponding to the effective spraying range of the target), etc. need to be determined. Specific characteristic information is required to be different for different crops, and the characteristic information is not limited to the relative height of the target, the floor area of the target, the effective spraying range of the target and the like, and is specifically required to be determined by combining different crops.

Step S13: and generating an aircraft spraying route according to the position information and the characteristic information of all the target objects so as to instruct the aircraft to carry out spraying operation.

The method comprises the steps of determining various points on a spraying route of the aircraft in a target area according to position information of the target object, wherein each point corresponds to one target object, and constructing the spraying route by connecting the points.

By determining the characteristic information of each target object, the spraying operation information of the aircraft at each target object can be determined, wherein the spraying operation information comprises but is not limited to the time length of the aircraft staying at the position of each target object, the spraying amount corresponding to each target object, the spraying range corresponding to each target object and the like, so that the spraying operation of the aircraft can be balanced.

Before the above step S12, a recognition engine is generated, which has deep learning capability and can determine the position information and feature information of the object according to the three-dimensional map and the data information associated with the three-dimensional map, for example, the recognition engine can determine the coordinates of different objects according to the point cloud data, and can determine the feature information of the object, such as the relative height of the object, the floor area of the object, etc., according to the shape of the object and the related data information contained in the three-dimensional map.

Optionally, the recognition engine may be generated by any one of an RCNN algorithm (Regions with conditional Neural Network Features), a Fast-RCNN algorithm, or a Fast-RCNN algorithm, where the three algorithms mainly use deep learning to perform target detection, the three algorithms are in a sequentially progressive relationship, and each algorithm is improved and optimized on the basis of the previous algorithm, so that the effects of the three algorithms are also in a sequentially progressive relationship.

Referring to fig. 2, on the basis of any one of the RCNN algorithm, the Fast-RCNN algorithm, or the Fast-RCNN algorithm, the following steps are required to be completed for generating the recognition engine:

step S01: and acquiring multiple groups of data related to the position information and the characteristic information of the target object, and generating a target area model containing a plurality of target objects.

The target area contains a large number of target objects, so that a plurality of target objects can be selected from the target objects, and the position information and the characteristic information of the target objects can be obtained. The position information of the target object can be determined through point cloud data, and the characteristic information of the target object is determined through manual marking. As mentioned above, the position information of the target includes the longitude, latitude, altitude, etc. of the target, and the characteristic information of the target includes the relative height of the target, the floor area of the target, the effective spraying range of the target, etc., for example, for a tree, the distance between the top of the tree and the ground, the boundary of the tree crown, the radius of the tree crown, etc. need to be marked. The position information of the target object is determined, the characteristic information of the target object is determined through manual marking, a relevant data table is correspondingly formed, and then a target area model can be generated on the basis of the data table.

Of course, the target area model may also be generated using the related data of other areas, which have similar features to the target area, and the position information and feature information of the corresponding target object in the area may be determined. For example, the geographical environment of the area is the same as or close to the geographical environment of the target area, the crop in the area has the same or similar shape as the crop in the target area, the crop in the area has the same or similar distribution law as the crop in the target area, and so on.

Step S02: and according to the target area model, a plurality of characteristic points which are associated with the characteristic information of each target object are marked.

In the target area model and the actual target area, the target cannot be represented by only a single coordinate point, and the spraying operation cannot be smoothly performed. Since the feature information of different objects is different, it is very labor and material consuming to actually measure and determine the feature information of each object, which is not available. In view of this, on the basis of determining the shape of the target object, a plurality of feature points are selected according to the shape of the target object for targeting, and then feature information of the target object is determined through the position relationship and the distance between different feature points. For example, for a tree, a target object is selected in the target area model, and a plurality of feature points, such as a top point of the tree, a boundary point between the tree and the ground (the boundary point is usually located on the axis of the tree root), and any two opposite points of the bottom edge of the crown, are selected from the labeled target object to form a plurality of sets of corresponding data. Specifically, the relative height of the tree in the target area model can be determined by combining the top point of the tree and the boundary point of the tree and the ground, and the crown radius in the target area model can be determined by combining any two opposite points of the bottom edge of the crown, so that the occupied area of the tree in the target area model can be determined. Obviously, on this basis, and with the known size ratio of the target area model to the actual target area, the relative heights, footprints, etc. of the trees in the actual target area can be determined.

Step S03: and determining and generating the recognition engine according to the matching relationship between the position information of all the target objects and the point cloud data and the matching relationship between the plurality of groups of data corresponding to the characteristic information of all the target objects and all the characteristic points.

Since the position information of the target object corresponds to the point cloud data one to one, the position information of each target object can be determined as long as the point cloud data is determined.

For the matching relationship between the multiple sets of data corresponding to the feature information of the target object and the feature points on the target object, in order to make the matching relationship more accurate, the error needs to be reduced, and the target object in the target area model is made to be consistent with the corresponding target object in the actual target model to the greatest extent. Therefore, the target area model can be continuously optimized by adopting iterative operation, in short, after the matching relation between the target object in the model and the target object in the actual target area is determined once, further matching is carried out until the error is reduced to the allowable range.

Similarly, the matching relationship between the position information of each target object and the point cloud data can also be optimized through iterative operation.

After the series of steps, the recognition engine can be generated, on the basis of training, the recognition engine has the capability of deep learning, after the three-dimensional map is obtained, the recognition engine can target a plurality of feature points according to the specific shape of the target object, and determine the feature information of the target object according to the feature points, so that the target object in the three-dimensional map does not need to be manually marked, and the operation efficiency is improved. As previously described, the recognition engine may determine location information and feature information of each object in the target area from the three-dimensional map.

Referring to fig. 3, specifically, the step of determining the position information and the feature information of the target object by the recognition engine includes the following specific steps:

step S121: and the identification engine determines the position information of each target object according to the point cloud data.

As can be seen from the foregoing description, in the target area model, the position information of the target object is matched with the point cloud data, and the recognition engine can determine the specific position of each target object in the target area coordinate system according to the point cloud data.

Step S122: the recognition engine determines a plurality of feature points associated with each target object, and determines feature information of each target object according to the feature points corresponding to different target objects.

This step is different from step S03 in that the purpose of step S03 is to generate the recognition engine by determining the matching relationship of the sets of data corresponding to the feature information of the object and the feature points, the data, information, and the like employed being obtained from the object area, and step S122 is performed by the recognition engine under the condition that the matching relationship of the sets of data corresponding to the characteristic information of the object and the feature points has been established, the data employed being a three-dimensional map constructed from the image information of the object area. Because the recognition engine has the capability of deep learning, the recognition engine can recognize the target object in the target area similar to the target area model on the basis of the matching relationship between the plurality of groups of data corresponding to the established target object characteristic information and the characteristic points, so as to determine the characteristic information of the target object.

In summary, in the aircraft spraying route generating method of the present invention, a three-dimensional map is constructed by the acquired image information of the target area, and the three-dimensional map includes point cloud data, that is, position information of all target objects in the target area; therefore, the position information of all the target objects and the characteristic information of each target object can be determined according to the image information and the point cloud data; and finally, correspondingly generating an aircraft spraying route according to the position information and the characteristic information of all the target objects, so that the aircraft can finish spraying operation conveniently. The whole process does not need to artificially survey and draw the position information and the characteristic information of the target object, so that the working efficiency of surveying and drawing the position information and the characteristic information of the target object is greatly improved, the surveying and drawing cost is reduced, and the efficiency of spraying operation implemented by the aircraft is improved.

Correspondingly, the invention also provides a control terminal, which comprises: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: carrying out the steps of the aircraft spray route generation method of any one of the preceding claims. Therefore, the control terminal has the advantages of the aircraft spraying route generation method, and therefore, the detailed description is omitted.

Referring to fig. 4, correspondingly, the present invention further provides an aircraft spraying route generating device, which includes an information processing module 11, an information identifying module 12 and a route generating module 13, specifically:

the information processing module 11 is configured to acquire image information of a target area, and construct a three-dimensional map including point cloud data according to the image information.

The information identification module 12 is configured to generate an identification engine, and enable the identification engine to determine position information of all the objects in the object area and feature information of each object according to the three-dimensional map.

And the route generating module 13 is configured to generate an aircraft spraying route according to the position information and the feature information of all the target objects, so as to instruct the aircraft to perform spraying operation.

The route generation module 13 includes a task configuration module (not shown, the same applies below) that, after determining the characteristic information of each target object, determines the spraying operation information of the aircraft at each target object, including but not limited to the length of time the aircraft stays at the location of each target object, the spraying amount corresponding to each target object, the spraying range corresponding to each target object, and the like, thereby facilitating balancing the spraying operation of the aircraft.

The recognition engine has the capability of deep learning and can determine the position information and the characteristic information of the target object according to the three-dimensional map and the data information associated with the three-dimensional map, for example, the recognition engine can determine the coordinates of different target objects according to point cloud data, and can determine the characteristic information of the target object, such as the relative height of the target object, the floor area of the target object and the like, according to the shape of the target object and the related data information contained in the three-dimensional map.

To generate the recognition engine, the information recognition module 12 includes a recognition engine generation module (not shown). Optionally, for the recognition engine generation module, the recognition engine may be generated by any one of an RCNN algorithm, a Fast-RCNN algorithm, or a Fast-RCNN algorithm, where the three algorithms mainly use deep learning to perform target detection, the three algorithms are in a sequentially progressive relationship, and each algorithm is improved and optimized on the basis of the previous algorithm, so that the effects of the three algorithms are also in a sequentially progressive relationship.

Referring to fig. 5, on the basis of any one of the RCNN algorithm, Fast-RCNN algorithm or Fast-RCNN algorithm, the recognition engine generating module needs to include a model generating unit 101, a target unit 102 and an executing unit 103, specifically:

the model generating unit 101 is configured to acquire multiple sets of data related to position information and feature information of the target object, and generate a target area model including multiple target objects.

The target unit 102 is configured to target a plurality of feature points associated with feature information of each target object according to the target area model.

In the target area model and the actual target area, the target cannot be represented by only a single coordinate point, and the spraying operation cannot be smoothly performed. Since the feature information of different objects is different, it is very labor and material consuming to actually measure and determine the feature information of each object, which is not available. In view of this, on the basis of determining the shape of the target object, a plurality of feature points are selected according to the shape of the target object for targeting, and then feature information of the target object is determined through the position relationship and the distance between different feature points. For example, for a tree, a target object is selected in the target area model, and a plurality of feature points, such as a top point of the tree, a boundary point between the tree and the ground (the boundary point is usually located on the axis of the tree root), and any two opposite points of the bottom edge of the crown, are selected from the labeled target object to form a plurality of corresponding sets of data. Specifically, the relative height of the tree in the target area model can be determined by combining the top point of the tree and the boundary point of the tree and the ground, and the crown radius in the target area model can be determined by combining any two opposite points of the bottom edge of the crown, so that the occupied area of the tree in the target area model can be determined. Obviously, on this basis, and with the known size ratio of the target area model to the actual target area, the relative heights, footprints, etc. of the trees in the actual target area can be determined.

The execution unit 103 is configured to determine and generate the recognition engine according to the matching relationship between the position information of all the targets and the point cloud data and the matching relationship between the sets of data corresponding to the feature information of all the targets and all the feature points.

On the basis of training, the recognition engine has the capability of deep learning, after the three-dimensional map is obtained, the recognition engine can target a plurality of feature points according to the specific shape of the target object and determine the feature information of the target object according to the feature points, so that the target object in the three-dimensional map does not need to be manually marked, and the operation efficiency is improved. As previously described, the recognition engine may determine location information and feature information of each object in the target area from the three-dimensional map.

Referring to fig. 6, specifically, the recognition engine includes a location information defining unit 121 and a feature information defining unit 122, specifically:

the position information determining unit 121 is configured to determine position information of each target object according to the point cloud data.

The characteristic information defining unit 122 is configured to determine a plurality of characteristic points associated with each target object, and determine characteristic information of each target object according to the characteristic points corresponding to different target objects.

The difference from the execution unit 103 is that the execution unit 103 is intended to generate the recognition engine by determining the matching relationship between the feature points and the sets of data corresponding to the feature information of the object, and the data, information, and the like used by the recognition engine are obtained from the object region, and the feature information clarification unit 122 is performed by the recognition engine under the condition that the matching relationship between the feature points and the sets of data corresponding to the feature information of the object has been established, and the data used by the recognition engine is a three-dimensional map constructed from the image information of the object region. Because the recognition engine has the capability of deep learning, the recognition engine can recognize the target object in the target area similar to the target area model on the basis of the matching relationship between the plurality of groups of data corresponding to the established target object characteristic information and the characteristic points, so as to determine the characteristic information of the target object.

In summary, the aircraft spraying route generating device of the present invention constructs a three-dimensional map by using the acquired image information of the target area, where the three-dimensional map includes point cloud data, that is, location information of all target objects in the target area; therefore, the position information of all the target objects and the characteristic information of each target object can be determined according to the image information and the point cloud data; and finally, correspondingly generating an aircraft spraying route according to the position information and the characteristic information of all the target objects, so that the aircraft can finish spraying operation conveniently. The whole process does not need to artificially survey and draw the position information and the characteristic information of the target object, so that the surveying and drawing cost is reduced, the working efficiency of surveying and drawing the position information and the characteristic information of the target object is greatly improved, and the efficiency of spraying operation implemented by the aircraft is also improved.

Referring to fig. 7, correspondingly, the present invention further provides an aircraft spraying route executing method, which includes the following steps:

step S21: receiving spray route related information for defining a route for the aircraft to perform a spray operation.

The spraying route related information includes, but is not limited to, position information of the target object and feature information of the target object. The spray route related information is generally detected by an external device, and the aircraft only needs to receive the information from the external device and then perform the spraying operation according to the information.

Specifically, the position information of the target object includes longitude, latitude, altitude, and the like of the position of the target object, and can be determined by establishing a corresponding coordinate system.

Step S22: and starting the aircraft, and carrying out spraying operation according to the spraying route related information.

Typically, the information associated with the aircraft performing the spray operation includes, but is not limited to, the flight time, spray volume, and spray range of the aircraft at each target object, among other things, to facilitate balancing the spray operation of the aircraft.

Referring to fig. 8, the present invention further provides an aircraft spraying route executing device, which includes an information receiving module 21 and a starting module 22, specifically:

the information receiving module 21 is configured to receive spraying route related information, where the spraying route related information is used to define a route for the aircraft to perform spraying operation.

The starting module 22 is configured to start the aircraft, and perform a spraying operation according to the spraying route related information.

The aircraft spraying route execution device corresponds to the aircraft spraying route execution method, and therefore details are omitted.

Furthermore, the invention also provides an aircraft comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the steps of the aircraft spray route execution method are performed. Therefore, the aircraft has the advantages of the aircraft spraying route execution method, and therefore, the detailed description is omitted.

Usually, the aircraft spray route execution means are required in combination with the aircraft spray route generation means, in other words by means of the aircraft spray route generation means to generate an aircraft spray route, while by means of the aircraft spray route execution means a spray operation is carried out on the basis of the aircraft spray route.

The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An aircraft spray route generation method, characterized by comprising the steps of:

acquiring image information of a target area, and constructing a three-dimensional map containing point cloud data according to the image information;

acquiring multiple groups of data related to position information and characteristic information of the target object, and generating a target area model containing multiple target objects;

a plurality of characteristic points which are associated with the characteristic information of each target object according to the target area model;

determining and generating a recognition engine according to the matching relationship between the position information of all the target objects and the point cloud data and the matching relationship between the plurality of groups of data corresponding to the feature information of all the target objects and all the feature points;

determining the position information of all the target objects in the target area and the characteristic information of each target object by using the identification engine according to the three-dimensional map; the characteristic information comprises effective spraying ranges of all the target objects;

and generating an aircraft spraying route according to the position information and the characteristic information of all the target objects so as to instruct the aircraft to carry out spraying operation.

2. The aircraft spray route generation method according to claim 1, wherein the matching relationship between the plurality of sets of data corresponding to the feature information of all the target objects and all the feature points is determined through iterative operation.

3. The method for generating a spraying route for an aircraft according to claim 1, wherein the step of determining the position information of all the targets and the feature information of each target in the target area from the three-dimensional map by using the recognition engine specifically comprises:

the identification engine determines the position information of each target object according to the point cloud data;

the recognition engine determines a plurality of feature points associated with each target object, and determines feature information of each target object according to the feature points corresponding to different target objects.

4. The aircraft spray route generation method according to claim 1, wherein the step of generating an aircraft spray route to instruct the aircraft to carry out a spray operation according to the position information and the characteristic information of all the targets comprises:

and determining the spraying operation information of the aircraft at each target object according to the characteristic information of each target object.

5. The aircraft spray route generation method according to claim 4, wherein the spray operation information includes flight time, spray amount, and spray range of the aircraft at each target object.

6. The aircraft spraying route generating method according to claim 1, wherein the position information of the target object comprises longitude and latitude of a center point of the target object and an altitude at which the target object is located.

7. The aircraft spray route generation method of claim 1, wherein the characteristic information of the target object includes a relative altitude of the target object and a footprint of the target object.

8. The aircraft spray route generation method of claim 1, wherein the RCNN algorithm, the Fast-RCNN algorithm, or the Fast-RCNN algorithm is used to generate the recognition engine for determining the target position information and the characteristic information.

9. An aircraft spray route generation device, comprising:

the information processing module is used for acquiring image information of a target area and constructing a three-dimensional map containing point cloud data according to the image information;

the information identification module is used for acquiring multiple groups of data related to the position information and the characteristic information of the target object and generating a target area model containing multiple target objects; a plurality of characteristic points which are associated with the characteristic information of each target object according to the target area model; determining and generating a recognition engine according to the matching relationship between the position information of all the target objects and the point cloud data and the matching relationship between the plurality of groups of data corresponding to the feature information of all the target objects and all the feature points; enabling the recognition engine to determine the position information of all the target objects in the target area and the characteristic information of each target object according to the three-dimensional map; the characteristic information comprises effective spraying ranges of all the target objects;

and the route generating module is used for generating an aircraft spraying route according to the position information and the characteristic information of all the target objects so as to instruct the aircraft to carry out spraying operation.

10. The aircraft spray route generation device according to claim 9, wherein the matching relationship between the plurality of sets of data corresponding to the feature information of all the targets and all the feature points is determined through iterative operation.

11. The aircraft spray route generation device according to claim 9, wherein the recognition engine specifically comprises:

the position information determining unit is used for determining the position information of each target object according to the point cloud data;

and the characteristic information determining unit is used for determining a plurality of characteristic points associated with each target object and determining the characteristic information of each target object according to the characteristic points corresponding to different target objects.

12. The aircraft spray route generation apparatus of claim 9, wherein the route generation module comprises:

and the task configuration module is used for determining the spraying operation information of the aircraft at each target object according to the characteristic information of each target object.

13. The aircraft spray route generation apparatus of claim 12, wherein the spray operation information includes a flight time, a spray amount, and a spray range of the aircraft at each target.

14. The aircraft spray route generation device of claim 9, wherein the position information of the target object comprises a longitude and latitude of a center point of the target object and an altitude at which the target object is located.

15. The aircraft spray route generating device of claim 9 wherein the characteristic information of the target object includes a relative altitude of the target object and a footprint of the target object.

16. The aircraft spray route generation apparatus as claimed in claim 9, wherein the recognition engine for determining the position information and the characteristic information of the target object is generated by using an RCNN algorithm, a Fast-RCNN algorithm or a Fast-RCNN algorithm.

17. An aircraft spray route execution method, comprising the steps of:

receiving spray route related information, the spray route related information being used for defining a route for the aircraft to carry out a spray operation;

starting the aircraft, and carrying out spraying operation according to the spraying route related information;

the spraying route related information is generated by a preset identification engine according to the position information of all the target objects in the target area and the characteristic information of each target object; the characteristic information comprises effective spraying ranges of all the target objects; the generation mode of the pre-recognition engine comprises the following steps: acquiring multiple groups of data related to position information and characteristic information of the target object, and generating a target area model containing multiple target objects; a plurality of characteristic points which are associated with the characteristic information of each target object according to the target area model; determining and generating the recognition engine according to the matching relationship between the position information of all the target objects and the point cloud data and the matching relationship between the plurality of groups of data corresponding to the feature information of all the target objects and all the feature points;

the three-dimensional map is constructed according to the image information of the target area; the three-dimensional map includes the point cloud data.

18. The aircraft spray route execution method of claim 17 wherein the information associated with the aircraft performing a spray operation includes a time of flight, a spray volume, and a spray range of the aircraft at each target.

19. The aircraft spray route execution method of claim 17, wherein the target object location information comprises a target object center point longitude and latitude and an altitude at which the target object is located.

20. The aircraft spray route execution method of claim 17 wherein the target object characteristic information comprises a relative height of the target object and a footprint of the target object.

21. An aircraft spray route actuation device, comprising:

the information receiving module is used for receiving spraying route related information, and the spraying route related information is used for limiting a route of the aircraft for spraying operation; the spraying route related information is generated by a preset identification engine according to the position information of all the target objects in the target area and the characteristic information of each target object; the characteristic information comprises effective spraying ranges of all the target objects; the three-dimensional map is constructed according to the image information of the target area; the three-dimensional map comprises point cloud data; the generation mode of the pre-recognition engine comprises the following steps: acquiring multiple groups of data related to position information and characteristic information of the target object, and generating a target area model containing multiple target objects; a plurality of characteristic points which are associated with the characteristic information of each target object according to the target area model; determining and generating the recognition engine according to the matching relationship between the position information of all the target objects and the point cloud data and the matching relationship between the plurality of groups of data corresponding to the feature information of all the target objects and all the feature points;

and the starting module is used for starting the aircraft and carrying out spraying operation according to the spraying route related information.

22. A control terminal, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the steps of performing the aircraft spray route generation method of any one of claims 1-8.

23. An aircraft, characterized in that it comprises:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the steps of performing the aircraft spray route execution method of any one of claims 17-20.