WO2021087685A1 - 结合多光谱和地表语义的作业规划方法、装置及设备 - Google Patents

结合多光谱和地表语义的作业规划方法、装置及设备 Download PDF

Info

Publication number
WO2021087685A1
WO2021087685A1 PCT/CN2019/115414 CN2019115414W WO2021087685A1 WO 2021087685 A1 WO2021087685 A1 WO 2021087685A1 CN 2019115414 W CN2019115414 W CN 2019115414W WO 2021087685 A1 WO2021087685 A1 WO 2021087685A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
target area
diagnostic information
image
workload
Prior art date
Application number
PCT/CN2019/115414
Other languages
English (en)
French (fr)
Inventor
刘志鹏
李鑫超
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2019/115414 priority Critical patent/WO2021087685A1/zh
Priority to CN201980033739.3A priority patent/CN112204569A/zh
Publication of WO2021087685A1 publication Critical patent/WO2021087685A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB

Definitions

  • This application relates to the field of image technology, and in particular to a method, device and equipment for job planning that combine multi-spectrum and surface semantics.
  • the surface area includes not only objects that need to be diagnosed, such as farmland, but also objects that do not need to be diagnosed, such as roads, and all objects that need to be diagnosed may have problems, and some may not have problems, so
  • the user needs to compare the color image corresponding to the multispectral image to further determine the area corresponding to the diagnosis result and the work plan for the area.
  • the embodiments of the present application provide a method, device, and equipment for operation planning that combine multispectral and surface semantics to solve the need to manually determine the area corresponding to the diagnosis result in the prior art, and the operation planning for this area requires high labor costs.
  • an embodiment of the present application provides a job planning method combining multi-spectrum and surface semantics, including:
  • an operation plan for the target area on the surface is generated.
  • an embodiment of the present application provides a job planning device that combines multi-spectrum and surface semantics, including: a processor and a memory; the memory is used to store program code; the processor calls the program code, when the program When the code is executed, it is used to perform the following operations:
  • an operation plan for the target area on the surface is generated.
  • an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes at least one piece of code, the at least one piece of code can be executed by a computer to control all The computer executes the method described in any one of the above-mentioned first aspects.
  • an embodiment of the present application provides a computer program, when the computer program is executed by a computer, it is used to implement the method described in any one of the above-mentioned first aspects.
  • an embodiment of the present application provides an agricultural drone that performs operations in the target area based on the operation plan obtained by the method described in the first aspect.
  • an embodiment of the present application provides a drone equipped with a multispectral image acquisition device.
  • the drone includes a processor and a memory.
  • the memory contains instructions. When the drone is on the target Call the command to execute the following steps when flying in the area:
  • an operation plan for the target area on the surface is generated.
  • an embodiment of the present application provides a ground-end device for communication and connection with an unmanned aerial vehicle equipped with a multi-spectral image acquisition device, which flies in a target area and obtains a multi-spectral image of the target area on the surface ;
  • the ground terminal device has a memory and a processor, and instructions are stored in the memory, and the instructions are called to perform the following steps:
  • an operation plan for the target area on the surface is generated.
  • the embodiments of the present application provide a method, device, and equipment for job planning that combine multispectral and surface semantics.
  • the diagnostic information of the surface target area and the first feature map containing the surface semantic information are obtained.
  • the operation plan of the surface target area is generated, and the processing method of the operation planning combined with the diagnosis information of the surface target area and the surface semantics is realized, so that the user can directly obtain the operation plan corresponding to the diagnosis information.
  • labor costs are saved.
  • FIG. 1 is a schematic diagram of an application scenario of a job planning method combining multi-spectrum and surface semantics provided by an embodiment of the application;
  • FIG. 2 is a schematic flow diagram of a job planning method combining multi-spectrum and surface semantics according to an embodiment of the application
  • FIG. 3 is a schematic flowchart of a job planning method combining multi-spectrum and surface semantics according to another embodiment of the application;
  • FIG. 4 is a schematic diagram 1 of the relationship between the work position and the work load provided by an embodiment of this application;
  • FIG. 5 is a second schematic diagram of the relationship between the work position and the work load provided by an embodiment of the application.
  • FIG. 6 is a block diagram of a job planning method combining multi-spectrum and surface semantics according to another embodiment of this application;
  • FIG. 7 is a schematic structural diagram of a job planning device combining multi-spectrum and surface semantics provided by an embodiment of the application.
  • the job planning method combining multi-spectrum and surface semantics provided by the embodiments of the present application can be applied to scenarios that need to diagnose surface objects, and the method can be specifically executed by a job planning combining multi-spectrum and surface semantics.
  • the application scenario of this method can be as shown in Figure 1.
  • the job planning device 11 that combines multispectral and surface semantics can obtain multispectral images from other devices/equipment 12, and use the multispectral images provided in the embodiments of this application for multispectral images. Combine multi-spectral and surface semantics operation planning method for processing.
  • the specific method of communication connection between the job planning device 11 and other devices/equipment 12 combining multispectral and surface semantics is not limited in this application.
  • wireless communication connection may be realized based on Bluetooth interface, or wired communication connection may be realized based on RS232 interface.
  • the type of equipment including the operation planning device 11 may not be limited in the embodiment of the present application.
  • the equipment may be, for example, a desktop computer, an all-in-one computer, a notebook computer, a palm computer, a tablet computer, a smart phone, or a screen with a screen. Remote control, drone, ground terminal equipment, etc.
  • the job planning device 11 obtains multispectral images from other devices or equipment as an example.
  • the job planning device 11 may obtain multispectral images in other ways.
  • the job planning device 11 Can shoot to obtain multi-spectral images.
  • the job planning method combining multispectral and surface semantics obtained by the embodiments of the present application obtains the diagnostic information of the surface target area and the first feature map containing the surface semantic information according to the multispectral image of the surface target area, and according to the diagnosis information and The first feature map generates the operation plan of the surface target area, and realizes the processing method of combining the diagnosis information of the surface target area and the surface semantics to perform the operation planning, and saves the labor cost.
  • FIG. 2 is a schematic flow chart of a job planning method combining multi-spectrum and surface semantics provided by an embodiment of the application.
  • the execution subject of this embodiment may be a job planning device that combines multi-spectrum and surface semantics, specifically the job planning device Processor.
  • the method of this embodiment may include:
  • Step 201 Obtain a multispectral image of the target area on the surface.
  • the surface target area refers to the surface area that needs to be diagnosed, and the surface target area includes the object to be diagnosed.
  • the objects to be diagnosed can be different.
  • the object to be diagnosed may be farmland.
  • the object to be diagnosed may be a tree.
  • the multi-spectral image is an image obtained by repeatedly photographing the surface target area in multiple wavebands, and the multiple wavebands include visible light wavebands and adjacent visible light wavebands. It should be noted that the number of bands of the multi-spectral image may be several, dozens, dozens, hundreds or even thousands. When the number of bands of the multispectral image is large, that is, when the resolution of the band is high, the multispectral image can also be understood as a hyperspectral image.
  • the application does not limit the specific method of obtaining the multispectral image.
  • the multispectral image can be obtained by shooting with a multispectral camera mounted on the drone.
  • Step 202 Obtain diagnostic information of the target area on the surface and a first feature map containing semantic information on the surface according to the multispectral image.
  • the surface objects in the surface area have different reflections of part of the spectrum under different conditions, the surface objects can be diagnosed based on the information of the part of the spectrum in the multispectral image.
  • the obtaining the diagnostic information of the surface target area according to the multispectral image may specifically include: calculating the diagnostic information of the surface target area according to part of the spectral information of the multispectral image.
  • the diagnostic information of the target area on the surface may be calculated according to the partial spectrum information of the multispectral image and the preset vegetation factor calculation formula.
  • the normalized vegetation index (Normalized Vegetation Index, NDVI) can be calculated according to the information of the near-infrared band and the infrared band of the multi-spectral image and the preset vegetation index calculation formula.
  • the normalized vegetation index is one of the important parameters reflecting crop growth and nutritional information.
  • the diagnostic information of the nitrogen demand of crops can be obtained, which has an important guiding role in the rational application of nitrogen fertilizer.
  • the types of diagnostic information can be implemented flexibly according to requirements.
  • the diagnostic information may specifically include the diagnostic information of nutrient element content, for example, the diagnostic information of nitrogen element content.
  • the diagnosis information may specifically include the diagnosis information of the distribution of plant diseases and insect pests.
  • the size of the first feature map may be the same as the size of the multispectral image, for example, both are 100 times 200.
  • the specific manner in which the first feature map includes the surface semantic information may be that the pixel values in the feature map may represent the surface semantics of the corresponding pixels, where the surface semantics may include the recognizable surface object categories.
  • the types of surface objects that can be identified include the types corresponding to the objects to be diagnosed, such as farmland.
  • the types of surface objects that can be identified may also include other types other than the objects to be diagnosed, such as roads, buildings, and telephone poles.
  • the first feature map obtained from the multispectral image the pixel position with the pixel value of 1 is In order to identify the pixel position as farmland, the pixel position with the pixel value of 2 is the pixel position identified as a road, and the pixel position with the pixel value of 3 is the pixel position identified as the building.
  • the multi-spectral image may be processed based on the characteristics of the surface object to identify different types of surface objects, so as to obtain the first feature map.
  • Step 203 Generate an operation plan for the target area on the surface according to the diagnosis information and the first characteristic map.
  • the area corresponding to the object to be diagnosed can be determined according to the first feature map, and the diagnostic information of the target area on the surface can be further combined to obtain the diagnostic information of the area corresponding to the object to be diagnosed. Since the diagnosis information of the area corresponding to the object to be diagnosed can characterize the problem state of the object to be diagnosed, such as whether nitrogen is lacking and the degree of nitrogen demand, based on the diagnosis information of the area corresponding to the object to be diagnosed, a work plan for the diagnosis information can be determined.
  • the diagnostic information of the surface target area and the first feature map containing the semantic information of the surface are obtained based on the multispectral image of the target area on the surface, and the operation plan of the surface target area is generated based on the diagnostic information and the first feature map.
  • Fig. 3 is a schematic flow chart of a job planning method combining multi-spectrum and surface semantics provided by another embodiment of the application.
  • this embodiment mainly describes based on the diagnosis information and the The first characteristic map is to generate an optional implementation manner of the operation plan of the target area on the surface.
  • the method of this embodiment may include:
  • Step 301 Obtain a multispectral image of the target area on the surface.
  • step 301 is similar to step 201 and will not be repeated here.
  • Step 302 Obtain diagnostic information of the target area on the surface and a first feature map containing semantic information on the surface according to the multispectral image.
  • the obtaining the diagnostic information of the target area on the surface according to the multi-spectral image may specifically include: processing the multi-spectral image using a pre-trained first neural network model to generate a characterization image.
  • the size of the second feature map may be the same as the size of the multispectral image, for example, both are 100 times 200.
  • the specific manner in which the second feature map represents the diagnostic information of each location point of the target area on the surface may be that the pixel value in the feature map can represent the diagnostic information of the corresponding pixel.
  • the pixel value can represent the nitrogen content.
  • the pixel value can characterize the status of pests and diseases.
  • using a pre-trained first neural network model to process the multispectral image to generate a second feature map representing the diagnostic information of each location point of the surface target area may specifically include the following steps A1 and A2.
  • Step A1 Input the multi-spectral image into the first neural network model to obtain a model output result of the first neural network model.
  • the model output result of the first neural network model may include the confidence characteristic maps outputted by multiple output channels, and the multiple output channels may be associated with multiple target values determined according to the value range of the diagnostic information (taken The value sub-range) has a one-to-one correspondence, and the pixel value of a single confidence feature map is used to characterize the probability that the diagnostic information of the pixel is the target value. For example, assuming that the diagnostic information has a value range of 1 to 10, multiple channels can correspond to the values 1 to 10 respectively, and the output channel corresponding to the value i outputs the confidence characteristic map i, i is equal to 1, 2,... ...10, the pixel value in the confidence feature map i can represent the probability that the diagnostic information of the pixel is i.
  • Step B2 Obtain the second feature map according to the model output result of the first neural network model.
  • the target value corresponding to the confidence feature map with the largest pixel value at the same pixel position in the multiple confidence feature maps corresponding to the multiple output channels one-to-one may be used as the diagnostic information of the pixel location, To get the second feature map.
  • the 10 confidence feature maps are respectively the confidence feature map 1 to the confidence feature map 10, and the confidence feature map 1 corresponds to the target value i.
  • the pixel value at the pixel location (100, 100) in the confidence feature map 1 is 70
  • the pixel value at the pixel location (100, 100) in the confidence feature map 2 is 50
  • the pixel at the pixel location (100, 100) in the confidence feature map 3 The value is 20.
  • the pixel value of the pixel position (100, 100) in the confidence feature graph 4 to the confidence feature graph 10 is all 20, it can be determined that the pixel location (100, 100) corresponds to the value 1, that is, the diagnostic information of the pixel location (100, 100) The value is 1.
  • the first neural network model can determine the diagnostic information with pixel-level granularity, the first neural network model is used to process the multispectral image to generate the diagnostic information of the target area on the surface, which is beneficial to improve the accuracy of obtaining the diagnostic information.
  • the first neural network model may specifically be a convolutional neural network (Convolutional Neural Networks, CNN) model.
  • CNN convolutional Neural Networks
  • the obtaining the first feature map containing the surface semantic information according to the multispectral image may specifically include: processing the multispectral image using a pre-trained second neural network model to obtain the first feature map containing the surface semantic information A feature map.
  • using the pre-trained second neural network model to process the multispectral image to obtain the first feature map containing the surface semantic information may specifically include the following steps B1 and B2.
  • Step B1 Input the multi-spectral image into the second neural network model to obtain a model output result of the second neural network model.
  • the model output result of the second neural network model may include the confidence feature maps respectively output by multiple output channels, and the multiple output channels may correspond to multiple surface object categories one-to-one, and the confidence level of a single surface object category
  • the pixel value of the feature map is used to characterize the probability that the pixel is the category of the surface object. For example, suppose that three surface object categories can be identified, namely farmland, road, and building, and the output channel corresponding to the farmland outputs the confidence feature map 1, the output channel corresponding to the road output confidence feature map 2, the output corresponding to the building
  • the channel output confidence feature is shown in Figure 3.
  • the pixel value in the confidence feature Figure 1 can represent the probability that the pixel is farmland, and the pixel value in the confidence feature Figure 2 can represent the probability that the pixel is a road.
  • the confidence feature in Figure 3 The pixel value can characterize the probability that the pixel is a building.
  • Step B2 Obtain the first feature map according to the model output result of the second neural network model.
  • the surface object category corresponding to the confidence feature map with the largest pixel value at the same pixel location in the multiple confidence feature maps one-to-one corresponding to the multiple output channels may be used as the surface object category of the pixel location,
  • the first characteristic map is obtained.
  • the four confidence feature maps are respectively the confidence feature map 1 to the confidence feature map 4, and the confidence feature map 1 corresponds to the farmland and the confidence feature Figure 2 corresponds to the road, the confidence feature Figure 3 corresponds to the building, and the confidence feature Figure 4 corresponds to "other".
  • the pixel value at the pixel location (100, 100) in the confidence feature map 1 is 70
  • the pixel value at the pixel location (100, 100) in the confidence feature map 2 is 50
  • the pixel at the pixel location (100, 100) in the confidence feature map 3 When the value is 20, and the pixel value of the pixel location (100, 100) in the confidence feature map 4 is 20, it can be determined that the pixel location (100, 100) is farmland.
  • the pixel value at the pixel location (100, 80) in the confidence feature map 1 is 20
  • the pixel value at the pixel location (100, 80) in the confidence feature map 2 is 30, and the pixel location in the confidence feature map 3
  • the pixel value of (100,80) is 20
  • the pixel value of the pixel position (100,80) in the confidence feature figure 4 is 70
  • the second neural network model may specifically be a convolutional neural network model.
  • the obtaining the first feature map containing the surface semantic information according to the multispectral image may specifically include: obtaining the first feature map containing the surface semantic information according to part of the spectral information in the multispectral image .
  • a pre-trained second neural network model is used to process part of the spectral information in the multispectral image to obtain a first feature map containing semantic information of the ground surface.
  • the partial spectrum information used to obtain the first feature map may include red (red, R) spectrum, green (green, G) spectrum, and blue (blue, B) spectrum information.
  • the obtaining a first feature map containing semantic information of the surface according to the multispectral image may specifically include: obtaining a depth map (Depth Map) of the target area on the surface according to the multispectral image and The first feature map of surface semantic information.
  • obtaining a depth map (Depth Map) of the target area on the surface according to the multispectral image and The first feature map of surface semantic information.
  • the height factor of the surface object can be considered when identifying the surface object category to improve the accuracy of recognition. For example, trees and grass can be distinguished according to the depth map.
  • Step 303 According to the diagnosis information and the first characteristic map, determine the corresponding relationship between the working position and the workload.
  • the operating location is used to characterize the location of the object to be diagnosed in the target area on the surface that needs to be operated on agricultural machinery, for example, the location of the agricultural drone for pesticide spraying. Since the pixel points in the first feature map can correspond to the position points in the target area on the surface, the working position can correspond to the pixel points in the first feature map.
  • the diagnosis information of the surface target area may include the diagnosis information of the corresponding location points of the object to be diagnosed in the surface target area, and may also include the diagnosis information of the corresponding positions of other surface objects. Since the first feature map contains surface semantic information, The object to be diagnosed can be distinguished from other surface objects. Therefore, the diagnosis information of the object to be diagnosed can be obtained by combining the diagnosis information of the surface target area with the first feature map, so that the corresponding relationship between the working position and the workload can be determined.
  • step 303 may specifically include the following steps C1 and C2.
  • Step C1 Determine the correspondence between the work position and the diagnosis information according to the respective diagnosis information of the at least one area and the first characteristic map.
  • the at least one area may include the area where the object to be diagnosed is located, and may also include the area where other surface objects are located. Since the first feature map contains the semantic information of the surface, the diagnosis information of at least one area is combined with the first feature map to obtain the corresponding relationship between the location point (ie, the working position) of the object to be diagnosed and the diagnosis information.
  • step C1 may specifically include: screening the target pixels of the surface object category as the object to be diagnosed from the first feature map, and establishing the diagnosis of the working position corresponding to the target pixel and the area to which the target pixel belongs Correspondence of information, the corresponding relationship between the job location and the diagnostic information is obtained.
  • Step C2 According to the corresponding relationship between the working position and the diagnostic information and the corresponding relationship between different diagnostic information levels and the workload, according to the linear transition strategy of the workload, the corresponding relationship between the working position and the workload is obtained.
  • step C2 may specifically include: matching the diagnostic information corresponding to the work location with the corresponding relationship between different diagnostic information levels and workload, and establishing the corresponding relationship between the work location and the workload matched by the corresponding diagnostic information to obtain the initial
  • the corresponding relationship between the working position and the workload is adjusted according to the linear transition strategy of the workload, and the corresponding relationship between the final working position and the workload is obtained.
  • the diagnostic information level to which it belongs can be determined based on one piece of diagnostic information, and the corresponding workload can be determined further based on the diagnostic information level to which it belongs and the correspondence between different diagnostic information levels and workloads.
  • the value range of the diagnostic information can be divided into degree levels, that is, the diagnostic information levels are divided.
  • the value range of nitrogen content can be divided into 4 sub-ranges, which correspond to 4 levels respectively.
  • the nitrogen content sub-range 1 corresponds to the extreme deficiency level
  • the nitrogen content sub-range 2 corresponds to the severe deficiency level.
  • Nitrogen content sub-range 3 corresponds to a relatively lack grade
  • nitrogen content sub-range 4 corresponds to a health grade.
  • the value range of the pest status can be divided into 4 sub-ranges, corresponding to 4 levels.
  • the pest status sub-range 1 corresponds to the extreme severity level
  • the pest status sub-range 2 corresponds to the severity level
  • the pest status sub-range 3 corresponds to the more severe level
  • the disease and insect pest status sub-range 4 corresponds to the health level.
  • this application does not limit the division method of obtaining the diagnostic information level.
  • the division of the diagnostic information level may be preset, or the user may divide the diagnostic information level.
  • the corresponding workload can be set for different diagnostic information levels. Taking nitrogen content as an example, the workload corresponding to the extreme deficiency level can be 100 grams of nitrogen fertilizer per square meter, the workload corresponding to the severe deficiency level can be 70 grams per square meter, and the workload corresponding to the relatively lacking level can be 40 grams per square meter. Nitrogen fertilizer. The workload corresponding to the health level can be 0 grams of nitrogen fertilizer per square meter. It should be noted that the amount of nitrogen fertilizer operation here is only an example, not as a basis for agricultural production, and the specific amount of fertilizer application needs to be set according to the actual needs of the crops. It should be noted that this application does not limit the correspondence between different diagnostic information levels and workload. For example, the correspondence between diagnostic information levels and workload can be preset, or the correspondence between diagnostic information levels and workload can be set by the user .
  • step 303 may specifically include the following steps D1 and D2.
  • Step D1 according to the first characteristic map and the second characteristic map, determine the correspondence between the working position and the diagnosis information.
  • the second feature map can characterize the diagnostic information of each location point in the surface target area, and the first feature map contains surface semantic information, the second feature map can be combined with the first feature map to obtain the diagnosis information. Correspondence between the position point of the object (that is, the work position) and the diagnosis information.
  • step D1 may specifically include: screening the target pixels of the surface object category as the object to be diagnosed from the first feature map, and establishing the working position corresponding to the target pixel to the target pixel in the second feature map The corresponding relationship between the diagnostic information corresponding to the point, and the corresponding relationship between the job position and the diagnostic information is obtained.
  • Step D2 Obtain the correspondence between the working position and the workload according to the correspondence between the working position and the diagnostic information and the correspondence between different diagnostic information and the workload.
  • step D2 may specifically include: matching the diagnostic information corresponding to the job location with the corresponding relationship between different diagnostic information and the workload, and establishing the corresponding relationship between the job location and the workload matched by the corresponding diagnostic information, thereby obtaining Correspondence between job location and workload,
  • this application does not limit the correspondence between different diagnostic information and workload.
  • the correspondence between diagnostic information and workload may be preset, or the correspondence between diagnostic information and workload may be set by the user.
  • step D1 and step D2 adopt the method of first combining the first feature map with the diagnostic information, and then determining the workload.
  • the workload can also be determined based on the diagnostic information first, and then combined with the first feature. Figure combined.
  • Step 304 Generate a work plan for the target area on the surface according to the corresponding relationship between the work location and the work load.
  • step 304 may specifically include: marking the corresponding relationship between the work position and the work load in the target image, and generating a work planning diagram of the target area on the surface.
  • the target image includes one or more of the following: an all-black image, an all-white image, the multispectral image, or an image corresponding to part of spectral information in the multispectral image.
  • the all-black image may be an image in which the R value, G value, and B value of each pixel are all 0, and the all-white image may be an image in which the R value, G value, and B value of each pixel are all 255.
  • the image corresponding to part of the spectral information in the multi-spectral image may specifically be an RGB image corresponding to the multi-spectral image.
  • the job planning diagram of the target area on the surface is generated, and the correspondence between the job location and the workload of the target area on the surface is visualized through the job planning diagram. relationship.
  • the diagnosis information includes diagnosis information of the distribution of plant diseases and insect pests
  • the operation plan is a pesticide application plan.
  • the operation plan can help users perform accurate operations on surface objects (such as farmland) and solve the problem of how much pesticides are used. On the one hand, it can reduce the amount of pesticides used and reduce production costs. On the other hand, it can avoid the environmental impact of excessive use of pesticides. Pollution.
  • the diagnostic information includes diagnostic information of nutrient element content
  • the operation plan is a fertilization plan.
  • the operation plan can help users perform precise operations on surface objects (such as farmland) and solve the problem of how much fertilizer to use. On the one hand, it can reduce the amount of fertilizer used and reduce production costs. On the other hand, it can avoid the environmental impact of excessive use of fertilizer. Pollution.
  • step 304 may specifically include: generating a planned operation route and operation parameters corresponding to the operation route according to the corresponding relationship between the operation position and the operation amount. It realizes automatic planning of operation routes and operation parameters, which is conducive to further saving manpower.
  • the operating route can specifically be a flight route, and operating parameters can include flight height, flight speed, etc. in addition to the amount of work.
  • the method of this embodiment may further include: sending the operation plan to an agricultural drone.
  • sending the operation plan to the agricultural drone the agricultural drone can perform operations according to the generated operation plan.
  • the method of this embodiment may further include: displaying the operation plan.
  • the user can learn the operation plan of the target area on the surface through the display screen of the equipment including the operation planning device 11, which improves the convenience for the user to know the operation plan.
  • the diagnostic information of the surface target area and the first feature map containing the semantic information of the surface are obtained based on the multispectral image of the surface target area, and the work location and the first feature map are determined based on the diagnostic information and the first feature map.
  • the corresponding relationship of the workload based on the corresponding relationship between the job location and the workload, the job plan of the surface target area is generated, which realizes the processing method of combining the diagnostic information of the surface target area and the surface semantics to perform the job planning, so that the user can
  • the operation plan corresponding to the diagnosis information is directly obtained, which saves labor costs.
  • the above-mentioned operation planning diagram can also be understood as a prescription diagram for the problem of the object to be diagnosed, that is, the operation prescription diagram, taking the diagnosis information including the nutrient element content or the status of diseases and insect pests, and the object to be diagnosed is a crop as an example.
  • data can be obtained.
  • a drone mounted multi-spectral camera can be used to survey and map crops to obtain multi-spectral information of the crops, and then a complete multi-spectral image of the entire farmland can be obtained through image stitching
  • the depth map can be obtained according to UAV surveying and mapping.
  • the second step is to use multi-spectral images and depth maps and input them into the trained second convolutional neural network model to predict farmland, roads, ground, water and other categories and their credibility scores (equivalent to the aforementioned confidence features Figure), fusion of the two can get the final semantic map of farmland (equivalent to the first feature map mentioned above), thereby solving the problem of which areas to fertilize and spray.
  • the third step is to input the multi-spectral image into the trained first convolutional neural network model to obtain the diagnosis information of plant nutrient elements or the distribution of plant diseases and insect pests, or it can also be obtained through the vegetation factor calculation formula.
  • the fourth step is to combine the semantic map of farmland with plant nutrient elements or pests and diseases to obtain a map of nutrient elements or pests and diseases of the farmland area, and then classify them. Taking the distribution of pests and diseases as an example, they are divided into four categories: extremely severe, severe, lighter, and healthy. For each level, the corresponding pesticide dosage is set for each level to generate the final farmland fertilization/application prescription map.
  • the farmland fertilization/application prescription map can also be used to segment the farmland spraying area.
  • the farmland fertilization/application prescription map is obtained according to the multi-spectrum, the operation area is divided into several blocks, and the operation situation of the agricultural drone is planned according to the division.
  • the types and amounts of pesticides/fertilizers to be sprayed in each section of the farmland operation area are obtained, and then the operation of the agricultural drone is planned.
  • the operation can be that a single agricultural drone is loaded with different drugs. It can also be realized by using multiple drones to load different medicine boxes. During operation, multiple drones can be controlled to work together, such as controlling the overlap of the operating areas of two drones to increase the amount of spraying in a certain area.
  • FIG. 7 is a schematic structural diagram of a job planning device combining multi-spectrum and surface semantics provided by an embodiment of the application. As shown in FIG. 7, the device 700 may include a processor 701 and a memory 702.
  • the memory 702 is used to store program codes
  • the processor 701 calls the program code, and when the program code is executed, is configured to perform the following operations:
  • an operation plan for the target area on the surface is generated.
  • the job planning device combining multi-spectrum and surface semantics provided in this embodiment can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar to the method embodiments, and will not be repeated here.
  • an embodiment of the present application also provides an agricultural drone that performs operations in the target area based on the operation plan obtained by the operation planning method combining multispectral and surface semantics.
  • An embodiment of the application also provides an unmanned aerial vehicle equipped with a multispectral image acquisition device.
  • the unmanned aerial vehicle includes a processor and a memory.
  • the memory contains instructions. When the unmanned aerial vehicle flies in a target area, Call the instructions to perform the following steps:
  • an operation plan for the target area on the surface is generated.
  • the embodiment of the present application also provides a ground-end device for communication connection with a drone equipped with a multi-spectral image acquisition device, and the drone flies in a target area and obtains a multi-spectral image of the target area on the surface;
  • the ground terminal device has a memory and a processor, and instructions are stored in the memory, and the instructions are called to perform the following steps:
  • an operation plan for the target area on the surface is generated.
  • a person of ordinary skill in the art can understand that all or part of the steps in the foregoing method embodiments can be implemented by a program instructing relevant hardware.
  • the aforementioned program can be stored in a computer readable storage medium. When the program is executed, it executes the steps including the foregoing method embodiments; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Development Economics (AREA)
  • Educational Administration (AREA)
  • Remote Sensing (AREA)
  • Game Theory and Decision Science (AREA)
  • Astronomy & Astrophysics (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Image Analysis (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

一种结合多光谱和地表语义的作业规划方法、装置及设备。该方法包括：获得地表目标区域的多光谱图像；根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。本申请实现了结合地表目标区域的诊断信息和地表语义进行作业规划的处理方式，节约了人力成本。

Description

结合多光谱和地表语义的作业规划方法、装置及设备 技术领域
本申请涉及图像技术领域，尤其涉及一种结合多光谱和地表语义的作业规划方法、装置及设备。
背景技术
近年来，随着科技发展，无人飞行器的应用越来越广泛。
目前，可以通过无人飞行器航拍获得地表区域的多光谱图像，并通过多光谱图像对地表对象进行诊断。由于地表区域中既包括了需要诊断的对象，例如农田，也可能包括不需要诊断的对象，例如道路，并且需要诊断的所有对象中有的对象可能存在问题，有的可能不存在问题，因此在获得地表区域的诊断结果之后，用户需要对照多光谱图像对应的彩色图像进一步确定诊断结果对应的区域，以及针对该区域的作业规划。
上述处理方式，在根据多光谱图像对地表对象进行诊断之后，需要人工确定诊断结果对应的区域，以及针对该区域的作业规划，人力成本较高。
发明内容
本申请实施例提供一种结合多光谱和地表语义的作业规划方法、装置及设备，用以解决现有技术中需要人工确定诊断结果对应的区域，以及针对该区域的作业规划，人力成本较高的技术问题。
第一方面，本申请实施例提供一种结合多光谱和地表语义的作业规划方法，包括：
获得地表目标区域的多光谱图像；
根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语 义信息的第一特征图；
根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
第二方面，本申请实施例提供结合多光谱和地表语义的作业规划装置，包括：处理器和存储器；所述存储器，用于存储程序代码；所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
获得地表目标区域的多光谱图像；
根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
第三方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序包含至少一段代码，所述至少一段代码可由计算机执行，以控制所述计算机执行上述第一方面任一项所述的方法。
第四方面，本申请实施例提供一种计算机程序，当所述计算机程序被计算机执行时，用于实现上述第一方面任一项所述的方法。
第五方面，本申请实施例提供一种农业无人机，所述农业无人机基于第一方面所述的方法获得的作业规划，在所述目标区域执行作业。
第六方面，本申请实施例提供一种无人机，搭载有多光谱图像获取装置，所述无人机包括处理器和存储器，所述存储器中包含有指令，当所述无人机在目标区域飞行时调用所述指令以执行如下步骤：
获得地表目标区域的多光谱图像；
根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
第七方面，本申请实施例提供一种地面端设备，用于与搭载有多光谱图像获取装置的无人机通信连接，所述无人机在目标区域飞行并获得地表目标区域的多光谱图像；
所述地面端设备具有存储器和处理器，所述存储器中存储有指令，所述 指令被调用以执行如下步骤：
根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
本申请实施例提供一种结合多光谱和地表语义的作业规划方法、装置及设备，通过根据地表目标区域的多光谱图像，获得地表目标区域的诊断信息和包含地表语义信息的第一特征图，并根据诊断信息和第一特征图，生成地表目标区域的作业规划，实现了结合地表目标区域的诊断信息和地表语义进行作业规划的处理方式，使得用户能够直接获得与诊断信息对应的作业规划，与需要人工确定诊断结果对应的区域，以及需要人工确定针对诊断结果对应区域的作业规划相比，节约了人力成本。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的结合多光谱和地表语义的作业规划方法的应用场景示意图；
图2为本申请一实施例提供的结合多光谱和地表语义的作业规划方法的流程示意图；
图3为本申请另一实施例提供的结合多光谱和地表语义的作业规划方法的流程示意图；
图4为本申请一实施例提供的作业位置与作业量的关系示意图一；
图5为本申请一实施例提供的作业位置与作业量的关系示意图二；
图6为本申请又一实施例提供的结合多光谱和地表语义的作业规划方法的框图；
图7为本申请一实施例提供的结合多光谱和地表语义的作业规划装置的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供的结合多光谱和地表语义的作业规划方法可以应用于需要诊断地表对象的场景，该方法具体可以由结合多光谱和地表语义的作业规划执行。该方法的应用场景可以如图1所示，具体的，结合多光谱和地表语义的作业规划装置11可以从其他装置/设备12获得多光谱图像，并对多光谱图像采用本申请实施例提供的结合多光谱和地表语义的作业规划方法进行处理。对于结合多光谱和地表语义的作业规划装置11与其他装置/设备12通讯连接的具体方式，本申请可以不做限定，例如可以基于蓝牙接口实现无线通讯连接，或者基于RS232接口实现有线通讯连接。
需要说明的是，对于包括作业规划装置11的设备的类型，本申请实施例可以不做限定，该设备例如可以为台式机、一体机、笔记本电脑、掌上电脑、平板电脑、智能手机、带屏遥控器、无人机、地面端设备等。
需要说明的是，图1中以作业规划装置11从其他装置或设备获得多光谱图像为例，可替换的，作业规划装置11可以通过其他方式获得多光谱图像，示例性的，作业规划装置11可以拍摄获得多光谱图像。
本申请实施例提供的结合多光谱和地表语义的作业规划方法，通过根据地表目标区域的多光谱图像，获得地表目标区域的诊断信息和包含地表语义信息的第一特征图，并根据诊断信息和第一特征图，生成地表目标区域的作业规划，实现了结合地表目标区域的诊断信息和地表语义进行作业规划的处理方式，节约了人力成本。
下面结合附图，对本申请的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
图2为本申请一实施例提供的结合多光谱和地表语义的作业规划方法的流程示意图，本实施例的执行主体可以为结合多光谱和地表语义的作业规划 装置，具体可以为该作业规划装置的处理器。如图2所示，本实施例的方法可以包括：
步骤201，获得地表目标区域的多光谱图像。
本步骤中，所述地表目标区域是指需要进行对象诊断的地表区域，地表目标区域中包括待诊断对象。根据诊断场景不同，待诊断对象可以不同。示例性的，当为农田诊断场景时，待诊断对象可以为农田。当为树木诊断场景时，待诊断对象可以为树木。
所述多光谱图像是由多个波段对所述地表目标区域进行反复拍摄而得到的图像，该多个波段包括可见光波段和临近可见光的波段。需要说明的是，所述多光谱图像的波段数可以为几个、十几个、几十个、几百个甚至上千个。当多光谱图像的波段数较多时，即波段的分辨率较高时，多光谱图像也可以理解为高光谱图像。
需要说明的是，对于获得多光谱图像的具体方式，本申请不做限定。可选的，可以通过无人机上搭载的多光谱相机，拍摄获得所述多光谱图像。
步骤202，根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图。
本步骤中，由于地表区域中地表对象在不同状态下对部分光谱的反射存在差异，因此根据多光谱图像中部分光谱的信息能够对地表对象进行诊断。示例性的，所述根据所述多光谱图像，获得所述地表目标区域的诊断信息具体可以包括：根据所述多光谱图像的部分光谱信息，计算地表目标区域的诊断信息。
示例性的，可以根据所述多光谱图像的部分光谱信息，按照预设植被因子计算公式计算得到地表目标区域的诊断信息。例如，可以根据多光谱图像的近红外波段与红外波段的信息，按照预设植被指数计算公式，计算归一化植被指数(Normalized Vegetation Index，NDVI)。其中，归一化植被指数是反映农作物长势和营养信息的重要参数之一，例如根据归一化植被指数,可以获得农作物对氮的需求量的诊断信息，对合理施用氮肥具有重要的指导作用。
需要说明的是，诊断信息的类型可以根据需求灵活实现。示例性的，诊断信息具体可以包括营养元素含量的诊断信息，例如氮元素含量的诊断信息。示例性的，诊断信息具体可以包括病虫害分布的诊断信息。
第一特征图的尺寸可以与所述多光谱图像的尺寸相同，例如，均为100乘 200。示例性的，第一特征图包含地表语义信息的具体方式可以为特征图中的像素值可以表征对应像素的地表语义，其中，地表语义可以包括能够识别出的地表对象类别。
其中，能够识别出的地表对象类别包括待诊断对象对应的类别，例如农田。可选的，能够识别出的地表对象类别还可以包括待诊断对象之外的其他类别，例如道路、建筑物、电线杆等。
例如，假设像素值为1可以表示农田、像素值为2可以表示道路、像素值为3可以表示建筑物，则根据多光谱图像所得到的第一特征图中，像素值为1的像素位置即为识别为农田的像素位置，像素值为2的像素位置即为识别为道路的像素位置，像素值为3的像素位置即为识别为建筑物的像素位置。
示例性的，可以基于地表对象的特征，对多光谱图像进行处理，识别出不同类别的地表对象，从而获得第一特征图。
步骤203，根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
本步骤中，由于第一特征图中包含了地表语义信息，因此根据第一特征图可以确定待诊断对象对应的区域，进一步结合地表目标区域的诊断信息从而能够待诊断对象对应区域的诊断信息。由于待诊断对象对应区域的诊断信息能够表征待诊断对象的问题状态，例如是否缺乏氮元素以及氮元素的需求程度，因此基于待诊断对象对应区域的诊断信息，能够确定针对诊断信息的作业规划。
需要说明的是，作业规划的具体类型可以根据需求灵活实现。
本实施例中，通过根据地表目标区域的多光谱图像，获得地表目标区域的诊断信息和包含地表语义信息的第一特征图，并根据诊断信息和第一特征图，生成地表目标区域的作业规划，实现了结合地表目标区域的诊断信息和地表语义进行作业规划的处理方式，使得用户能够直接获得与诊断信息对应的作业规划，与需要人工确定诊断结果对应的区域，以及需要人工确定针对诊断结果对应区域的作业规划相比，节约了人力成本。
图3为本申请另一实施例提供的结合多光谱和地表语义的作业规划方法的流程示意图，本实施例在图2所示实施例的基础上，主要描述了根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划的可选实现方式。如图3所示，本实施例的方法可以包括：
步骤301，获得地表目标区域的多光谱图像。
需要说明的是，步骤301与步骤201类似，在此不再赘述。
步骤302，根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图。
本步骤中，示例性的，所述根据所述多光谱图像获得所述地表目标区域的诊断信息，具体可以包括：使用预训练的第一神经网络模型处理所述多光谱图像，以生成表征所述地表目标区域各个位置点的诊断信息的第二特征图。
第二特征图的尺寸可以与所述多光谱图像的尺寸相同，例如，均为100乘200。示例性的，第二特征图表征地表目标区域各个位置点的诊断信息的具体方式可以为特征图中的像素值可以表征对应像素的诊断信息，以氮元素为例像素值可以表征氮元素含量，以病虫害为例像素值可以表征病虫害状态。
示例性的，使用预训练的第一神经网络模型处理所述多光谱图像，以生成表征所述地表目标区域各个位置点的诊断信息的第二特征图，具体可以包括如下步骤A1和步骤A2。
步骤A1，将所述多光谱图像输入所述第一神经网络模型，得到所述第一神经网络模型的模型输出结果。
其中，所述第一神经网络模型的模型输出结果可以包括多个输出通道分别输出的置信度特征图，该多个输出通道可以与根据诊断信息的取值范围确定的多个目标取值(取值子范围)一一对应，单个的置信度特征图的像素值用于表征像素的诊断信息是所述目标取值的概率。例如，假设诊断信息的取值范围为1至10，多个通道可以分别与数值1至数值10一一对应，且对应数值i的输出通道输出置信度特征图i，i等于1、2、……10，则置信度特征图i中的像素值可以表征像素的诊断信息是i的概率。
步骤B2，根据所述第一神经网络模型的模型输出结果，得到所述第二特征图。
示例性的，可以将与该多个输出通道一一对应的多个置信度特征图中，同一像素位置像素值最大的置信度特征图对应的目标取值，作为所述像素位置的诊断信息，以得到第二特征图。
假设，所述第一神经网络模型的输出通道的个数为10，10个置信度特征图分别为置信度特征图1至置信度特征图10，且置信度特征图1对应目标取值i。例如，当置信度特征图1中像素位置(100,100)的像素值是70，置信度特征图 2中像素位置(100,100)的像素值是50，置信度特征图3中像素位置(100,100)的像素值是20，置信度特征图4至置信度特征图10中像素位置(100,100)的像素值均是20时，可以确定像素位置(100,100)对应取值1，即像素位置(100,100)的诊断信息取值1。
由于第一神经网络模型能够以像素级粒度确定诊断信息，因此通过使用第一神经网络模型处理多光谱图像以生成地表目标区域的诊断信息，有利于提高获得诊断信息的精度。
示例性的，所述第一神经网络模型具体可以为卷积神经网络(Convolutional Neural Networks，CNN)模型。
示例性的，所述根据所述多光谱图像获得包含地表语义信息的第一特征图，具体可以包括：使用预训练的第二神经网络模型处理所述多光谱图像，获得包含地表语义信息的第一特征图。示例性的，使用预训练的第二神经网络模型处理所述多光谱图像，获得包含地表语义信息的第一特征图具体可以包括如下步骤B1和步骤B2。
步骤B1，将所述多光谱图像输入所述第二神经网络模型，得到所述第二神经网络模型的模型输出结果。
其中，所述第二神经网络模型的模型输出结果可以包括多个输出通道分别输出的置信度特征图，该多个输出通道可以与多个地表对象类别一一对应，单个地表对象类别的置信度特征图的像素值用于表征像素是所述地表对象类别的概率。例如，假设能够识别3个地表对象类别，分别为农田、道路和建筑物，且对应农田的输出通道输出置信度特征图1、对应道路的输出通道输出置信度特征图2、对应建筑物的输出通道输出置信度特征图3，则置信度特征图1中的像素值可以表征像素是农田的概率，置信度特征图2中的像素值可以表征像素是道路的概率，置信度特征图3中的像素值可以表征像素是建筑物的概率。
步骤B2，根据所述第二神经网络模型的模型输出结果，得到所述第一特征图。
示例性的，可以将与该多个输出通道一一对应的多个置信度特征图中同一像素位置像素值最大的置信度特征图对应的地表对象类别，作为所述像素位置的地表对象类别，从而得到所述第一特征图。
假设，所述第二神经网络模型的输出通道的个数为4，4个置信度特征图 分别为置信度特征图1至置信度特征图4，且置信度特征图1对应农田、置信度特征图2对应道路、置信度特征图3对应建筑物、置信度特征图4对应“其他”。例如，当置信度特征图1中像素位置(100,100)的像素值是70，置信度特征图2中像素位置(100,100)的像素值是50，置信度特征图3中像素位置(100,100)的像素值是20，置信度特征图4中像素位置(100,100)的像素值是20时，可以确定像素位置(100,100)是农田。又例如，当置信度特征图1中像素位置(100,80)的像素值是20，置信度特征图2中像素位置(100,80)的像素值是30，置信度特征图3中像素位置(100,80)的像素值是20，置信度特征图4中像素位置(100,80)的像素值是70时，可以确定像素位置(100,80)是其他，即不是农田、道路和建筑物中的任意一种。
示例性的，所述第二神经网络模型具体可以为卷积神经网络模型。
示例性的，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，具体可以包括：根据所述多光谱图像中的部分光谱信息，获得包含地表语义信息的第一特征图。例如，使用预训练的第二神经网络模型处理所述多光谱图像中的部分光谱信息，获得包含地表语义信息的第一特征图。通过根据多光谱图像中的部分光谱信息获得第一特征图，可以减少计算量，节省计算资源。示例性的，用于获得第一特征图的所述部分光谱信息可以包括红(red，R)光谱、绿(green，G)光谱和蓝(blue，B)光谱的信息。
示例性的，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，具体可以包括：根据所述多光谱图像以及所述地表目标区域的深度图(Depth Map)，获得包含地表语义信息的第一特征图。通过在获得第一特征图时基于地表目标区域的深度图，可以实现在进行地表对象类别识别时考虑地表对象的高度因素，以提高识别的准确性，例如，根据深度图可以区分树木和草地。
步骤303，根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系。
本步骤中，作业位置用于表征地表目标区域中需要进行农机作业的待诊断对象的位置点，例如农业无人机进行农药喷洒作业的位置点。由于第一特征图中的像素点能够与地表目标区域中的位置点对应，因此作业位置可以与第一特征图的像素点对应。
需要说明的是，地表目标区域的诊断信息可以包括地表目标区域中待诊 断对象对应位置点的诊断信息，还可以包括其他地表对象对应位置的诊断信息，由于第一特征图包含了地表语义信息，能够区分出待诊断对象和其他地表对象，因此将地表目标区域的诊断信息和第一特征图结合可以获得待诊断对象的诊断信息，从而能够确定出作业位置与作业量的对应关系。
在所述诊断信息包括至少一个区域分别的诊断信息时，示例性的，步骤303具体可以包括如下步骤C1和步骤C2。
步骤C1，根据所述至少一个区域分别的诊断信息以及所述第一特征图，确定作业位置与诊断信息的对应关系。
其中，所述至少一个区域可以包括待诊断对象所在的区域，还可以包括其他地表对象所在的区域。由于第一特征图包含了地表语义信息，因此将至少一个区域的诊断信息与第一特征图结合，能够获得待诊断对象的位置点(即，作业位置)与诊断信息的对应关系。
示例性的，步骤C1具体可以包括：从第一特征图中筛选出地表对象类别为待诊断对象的目标像素点，建立所述目标像素点对应的作业位置与所述目标像素点所属区域的诊断信息的对应关系，得到作业位置与诊断信息的对应关系。
步骤C2，根据作业位置与诊断信息的对应关系以及不同诊断信息等级与作业量的对应关系，按照作业量线性过渡的策略，得到作业位置与作业量的对应关系。
示例性的，步骤C2具体可以包括：将作业位置对应的诊断信息与不同诊断信息等级与作业量的对应关系进行匹配，建立作业位置与其对应诊断信息所匹配的作业量的对应关系，以得到初始的作业位置与作业量的对应关系，按照作业量线性过渡策略对初始的作业位置与作业量的对应关系进行调整，得到最终的作业位置与作业量的对应关系。
具体的，根据一个诊断信息可以确定其所属的诊断信息等级，进一步的根据其所属的诊断信息等级以及不同诊断信息等级与作业量的对应关系可以确定其对应的作业量。
其中，对诊断信息的取值范围可以划分程度等级，即划分诊断信息等级。以氮元素含量为例，可以将氮元素含量的取值范围划分为4个子范围，分别对应4个等级，例如氮元素含量子范围1对应极端缺乏等级，氮元素含量子范围2对应严重缺乏等级，氮元素含量子范围3对应较缺乏等级，氮元素含量子范围 4对应健康等级。以病虫害状态为例，可以将病虫害状态的取值范围划分为4个子范围，分别对应4个等级，例如病虫害状态子范围1对应极端严重等级，病虫害状态子范围2对应严重等级，病虫害状态子范围3对应较严重等级，病虫害状态子范围4对应健康等级。需要说明的是，对于获得诊断信息等级的划分方式，本申请不做限定，例如可以预设诊断信息等级的划分，或者由用户划分诊断信息等级。
对不同诊断信息等级可以设置对应的作业量。以氮元素含量为例，极端缺乏等级对应的作业量可以为每平米100克氮肥，严重缺乏等级对应的作业量可以为每平米70克氮肥，较缺乏等级对应的作业量可以为每平米40克氮肥。健康等级对应的作业量可以为每平米0克氮肥。需要说明的是，这里的氮肥作业量仅为举例说明，不作为农业生产依据，具体施肥量需要农作物实际需求设置。需要说明的是，对于不同诊断信息等级与作业量的对应关系，本申请不做限定，例如可以预设诊断信息等级与作业量的对应关系，或者由用户设置诊断信息等级与作业量的对应关系。
通过作业量线性过渡的策略，可以使得相邻作业位置之间的作业量线性过渡，避免相邻作业位置之间作业量的较大变化。假设作业位置之间的相邻关系如图4所示，且作业位置1-作业位置5属于一个区域，作业位置6-作业位置10属于另一个区域。当不采用作业量线性过渡的策略时，如图4所示，作业位置1-作业位置5的作业量为100，作业位置6-作业位置10的作业量为60。当采用作业量线性过渡的策略时，如图5所示，作业位置1至作业位置9的作业量依次递减5，实现了作业量线性的线性过渡。需要说明的是，图5仅作为线性过渡策略的举例说明。
由于农机在作业过程中通常无法及时响应作业量的较大变化，通过按照作业量线性过渡的策略得到作业位置与作业量的对应关系，能够避免作业量的较大变化，提高了作业规划的合理性。
在所述诊断信息为表征所述地表目标区域各个位置点的诊断信息的第二特征图时，示例性的，步骤303具体可以包括如下步骤D1和步骤D2。
步骤D1，根据所述第一特征图以及所述第二特征图，确定作业位置与诊断信息的对应关系。
其中，由于所述第二特征图能够表征所述地表目标区域各个位置点的诊断信息，第一特征图包含了地表语义信息，因此将第二特征图与第一特征图 结合，能够获得待诊断对象的位置点(即，作业位置)与诊断信息的对应关系。
示例性的，步骤D1具体可以包括：从第一特征图中筛选出地表对象类别为待诊断对象的目标像素点，建立所述目标像素点对应的作业位置与第二特征图中所述目标像素点对应的诊断信息的对应关系，从而得到了作业位置与诊断信息的对应关系。
步骤D2，根据所述作业位置与诊断信息的对应关系以及不同诊断信息与作业量的对应关系，得到作业位置与作业量的对应关系。
示例性的，步骤D2具体可以包括：将作业位置对应的诊断信息，与不同诊断信息与作业量的对应关系进行匹配，建立作业位置与其对应诊断信息所匹配的作业量的对应关系，从而得到了作业位置与作业量的对应关系，
其中，不同诊断信息与作业量之间存在对应关系，通过不同诊断信息与作业量的对应关系，实现了诊断信息粒度的作业量对应，与诊断信息等级粒度的作业量对应相比，能够提高作业量的精度。需要说明的是，对于不同诊断信息与作业量的对应关系，本申请不做限定，例如可以预设诊断信息与作业量的对应关系，或者由用户设置诊断信息与作业量的对应关系。
需要说明的是，步骤D1和步骤D2是采用先将第一特征图与诊断信息结合，然后确定作业量的方式，可替换的，也可以先根据诊断信息确定作业量，然后再与第一特征图结合。
步骤304，根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划。
本步骤中，示例性的，步骤304具体可以包括：在目标图像中标注所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划图。示例性的，所述目标图像包括下述中的一种或多种：全黑图像、全白图像、所述多光谱图像或所述多光谱图像中部分光谱信息对应的图像。其中，全黑图像可以为各像素的R值、G值和B值均为0的图像，全白图像可以为各像素的R值、G值和B值均为255的图像。所述多光谱图像中部分光谱信息对应的图像具体可以为多光谱图像对应的RGB图像。
通过在目标图像中标注所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划图，实现了通过作业规划图的方式直观表示出地表目标区域的作业位置与作业量的对应关系。
示例性的，所述诊断信息包括病虫害分布的诊断信息，所述作业规划图为施药规划图。通过作业规划图可以帮助用户对地表对象(例如农田)进行精准作业，解决了用多少农药的问题，一方面可以降低农药使用量，减低生产成本，另一方面能够避免由于过量使用农药对环境的污染。
示例性的，所述诊断信息包括营养元素含量的诊断信息，所述作业规划图为施肥规划图。通过作业规划图可以帮助用户对地表对象(例如农田)进行精准作业，解决了用多少肥料的问题，一方面可以降低肥料使用量，减低生产成本，另一方面能够避免由于过量使用肥料对环境的污染。
示例性的，步骤304具体可以包括：根据所述作业位置与作业量的对应关系，生成规划的作业路线以及所述作业路线对应的作业参数。实现了自动化规划作业路线及作业参数，有利于进一步节约人力能本。对于农业无人机而言，作业路线具体可以为飞行路线，作业参数除了包括作业量还可以包括飞行高度、飞行速度等。
示例性的，本实施例的方法还可以包括：将所述作业规划发送至农业无人机。通过将作业规划发送至农业无人机，使得农业无人机能够根据所生成的作业规划进行作业。
示例性的，本实施例的方法还可以包括：展示所述作业规划。通过展示作业规划使得用户通过包括作业规划装置11的设备的显示屏获知地表目标区域的作业规划，提高了用户获知作业规划的便利性。
本实施例中，通过根据地表目标区域的多光谱图像，获得地表目标区域的诊断信息和包含地表语义信息的第一特征图，根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系，根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划，实现了结合地表目标区域的诊断信息和地表语义进行作业规划的处理方式，使得用户能够直接获得与诊断信息对应的作业规划，节约了人力成本。
上述作业规划图也可以理解为针对待诊断对象所存在问题的处方图，即作业处方图，以诊断信息包括营养元素含量或病虫害状态，且待诊断对象为农作物为例。如图6所示，第一步，可以获取数据，例如可以利用无人机挂载多光谱相机对农作物进行测绘，获得农作物的多光谱信息，然后通过图像拼接得到整块农田完整的多光谱图像，另外，根据无人机测绘还可以得到深度图。第二步，利用多光谱图像和深度图，输入到训练好的第二卷积神经网络 模型中，预测农田、道路、地面、水面等类别和其可信度得分(相当于前述的置信度特征图)，将两者融合能够得到最终的农田语义地图(相当于前述的第一特征图)，从而解决了到哪些区域施肥打药的问题。第三步，将多光谱图像输入到训练好的第一卷积神经网络模型得到植物营养元素或病虫害分布的诊断信息，或者也可以通过植被因子计算公式得到。第四步，将农田语义地图和植物营养元素或病虫害相结合，得到农田区域的营养元素图或病虫害图，之后划分等级，以病虫害分布为例，分为极端严重、严重、较轻和健康四个等级，每个等级设置对应的农药用量，生成最终的农田施肥/施药处方图。
农田施肥/施药处方图还可以用于农田喷洒区域的分割。在一个可选的实施例中，根据多光谱获得农田施肥/施药处方图，将作业区域分割为若干块，并根据分块的情况来规划农业无人机作业情况。例如，根据所述处方图得到农田作业区域中各个分块所需喷洒的农药/肥料品种和用量，然后规划农业无人机进行作业的，作业可以是单台农业无人机通过负载不同的药箱实现，也可以通过多台无人机负载不同的药箱实现。在作业时，可以控制多台无人机协同作业，例如控制两台无人机的作业区域重叠，以加重某个区域的喷洒药量。
需要说明的是，第二步和第三步并没有先后顺序的限制。
图7为本申请一实施例提供的结合多光谱和地表语义的作业规划装置的结构示意图，如图7所示，该装置700可以包括：处理器701和存储器702。
所述存储器702，用于存储程序代码；
所述处理器701，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
获得地表目标区域的多光谱图像；
根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
本实施例提供的结合多光谱和地表语义的作业规划装置，可以用于执行前述方法实施例的技术方案，其实现原理和技术效果与方法实施例类似，在此不再赘述。
另外，本申请实施例还提供一种农业无人机，所述农业无人机基于上述结合多光谱和地表语义的作业规划方法获得的作业规划，在所述目标区域执行作业。
本申请实施例还提供一种无人机，搭载有多光谱图像获取装置，所述无人机包括处理器和存储器，所述存储器中包含有指令，当所述无人机在目标区域飞行时调用所述指令以执行如下步骤：
获得地表目标区域的多光谱图像；
根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
本申请实施例还提供一种地面端设备，用于与搭载有多光谱图像获取装置的无人机通信连接，所述无人机在目标区域飞行并获得地表目标区域的多光谱图像；
所述地面端设备具有存储器和处理器，所述存储器中存储有指令，所述指令被调用以执行如下步骤：
根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (57)

  1. 一种结合多光谱和地表语义的作业规划方法，其特征在于，包括：
    获得地表目标区域的多光谱图像；
    根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
    根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述多光谱图像，获得所述地表目标区域的诊断信息，包括：
    根据所述多光谱图像的部分光谱信息，计算所述地表目标区域的诊断信息。
  3. 根据权利要求1所述的方法，其特征在于，所述根据所述多光谱图像，获得所述地表目标区域的诊断信息包括：
    使用预训练的第一神经网络模型处理所述多光谱图像，以生成表征所述地表目标区域各个位置点的诊断信息的第二特征图。
  4. 根据权利要求1所述的方法，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    使用预训练的第二神经网络模型处理所述多光谱图像，获得包含地表语义信息的第一特征图。
  5. 根据权利要求1所述的方法，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    根据所述多光谱图像中部分光谱信息，获得包含地表语义信息的第一特征图。
  6. 根据权利要求5所述的方法，其特征在于，所述部分光谱信息包括红光谱、绿光谱和蓝光谱的信息。
  7. 根据权利要求1所述的方法，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    根据所述多光谱图像以及所述地表目标区域的深度图，获得包含地表语义信息的第一特征图。
  8. 根据权利要求1所述的方法，其特征在于，所述根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划，包括：
    根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系；
    根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划。
  9. 根据权利要求8所述的方法，其特征在于，所述诊断信息包括至少一个区域分别的诊断信息；
    所述根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系，包括：
    根据所述至少一个区域分别的诊断信息以及所述第一特征图，确定作业位置与诊断信息的对应关系；
    根据作业位置与诊断信息的对应关系，以及不同诊断信息等级与作业量的对应关系，按照作业量线性过渡的策略，得到作业位置与作业量的对应关系。
  10. 根据权利要求8所述的方法，其特征在于，所述诊断信息为表征所述地表目标区域各个位置点的诊断信息的第二特征图；
    所述根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系，包括：
    根据所述第二特征图以及所述第一特征图，确定作业位置与诊断信息的对应关系；
    根据所述作业位置与诊断信息的对应关系以及不同诊断信息与作业量的对应关系，得到作业位置与作业量的对应关系。
  11. 根据权利要求8所述的方法，其特征在于，所述根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划，包括：
    在目标图像中标注所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划图。
  12. 根据权利要求11所述的方法，其特征在于，所述目标图像包括下述中的一种或多种：
    全黑图像、全白图像、所述多光谱图像或所述多光谱图像中部分光谱信息对应的图像。
  13. 根据权利要求11所述的方法，其特征在于，所述诊断信息包括病虫害分布的诊断信息，所述作业规划图为施药规划图。
  14. 根据权利要求11所述的方法，其特征在于，所述诊断信息包括营养元素含量的诊断信息，所述作业规划图为施肥规划图。
  15. 根据权利要求8所述的方法，其特征在于，所述根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划，包括：
    根据所述作业位置与作业量的对应关系，生成规划的作业路线以及所述作业路线对应的作业参数。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    将所述作业规划发送至农业无人机。
  17. 根据权利要求1所述的方法，其特征在于，所述方法还包括：展示所述作业规划。
  18. 一种结合多光谱和地表语义的作业规划装置，其特征在于，包括：存储器和处理器；
    所述存储器，用于存储程序代码；
    所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
    获得地表目标区域的多光谱图像；
    根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
    根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
  19. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序包含至少一段代码，所述至少一段代码可由计算机执行，以控制所述计算机执行如权利要求1-17任一项所述的方法。
  20. 一种计算机程序，其特征在于，当所述计算机程序被计算机执行时，用于实现如权利要求1-17任一项所述的方法。
  21. 一种农业无人机，其特征在于，所述农业无人机基于权利要求1-17之一所述的方法获得的作业规划，在所述目标区域执行作业。
  22. 根据权利要求21所述的农业无人机，其特征在于，所述执行作业包括根据获得的目标区域的病虫害分布情况，进行农药喷洒作业。
  23. 根据权利要求21所述的农业无人机，其特征在于，所述执行作业包括根据获得的目标区域的营养元素含量情况，进行施肥作业。
  24. 一种无人机，搭载有多光谱图像获取装置，其特征在于，所述无人机包括处理器和存储器，所述存储器中包含有指令，当所述无人机在目标区域飞行时调用所述指令以执行如下步骤：
    获得地表目标区域的多光谱图像；
    根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
    根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
  25. 根据权利要求24所述的无人机，其特征在于，所述根据所述多光谱图像，获得所述地表目标区域的诊断信息，包括：
    根据所述多光谱图像的部分光谱信息，计算所述地表目标区域的诊断信息。
  26. 根据权利要求24所述的无人机，其特征在于，所述根据所述多光谱图像，获得所述地表目标区域的诊断信息包括：
    使用预训练的第一神经网络模型处理所述多光谱图像，以生成表征所述地表目标区域各个位置点的诊断信息的第二特征图。
  27. 根据权利要求24所述的无人机，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    使用预训练的第二神经网络模型处理所述多光谱图像，获得包含地表语义信息的第一特征图。
  28. 根据权利要求24所述的无人机，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    根据所述多光谱图像中部分光谱信息，获得包含地表语义信息的第一特征图。
  29. 根据权利要求28所述的无人机，其特征在于，所述部分光谱信息包括红光谱、绿光谱和蓝光谱的信息。
  30. 根据权利要求24所述的无人机，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    根据所述多光谱图像以及所述地表目标区域的深度图，获得包含地表语义信息的第一特征图。
  31. 根据权利要求24所述的无人机，其特征在于，所述根据所述诊断信 息和所述第一特征图，生成所述地表目标区域的作业规划，包括：
    根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系；
    根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划。
  32. 根据权利要求31所述的无人机，其特征在于，所述诊断信息包括至少一个区域分别的诊断信息；
    所述根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系，包括：
    根据所述至少一个区域分别的诊断信息以及所述第一特征图，确定作业位置与诊断信息的对应关系；
    根据作业位置与诊断信息的对应关系，以及不同诊断信息等级与作业量的对应关系，按照作业量线性过渡的策略，得到作业位置与作业量的对应关系。
  33. 根据权利要求32所述的无人机，其特征在于，所述诊断信息为表征所述地表目标区域各个位置点的诊断信息的第二特征图；
    所述根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系，包括：
    根据所述第二特征图以及所述第一特征图，确定作业位置与诊断信息的对应关系；
    根据所述作业位置与诊断信息的对应关系以及不同诊断信息与作业量的对应关系，得到作业位置与作业量的对应关系。
  34. 根据权利要求31所述的无人机，其特征在于，所述根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划，包括：
    在目标图像中标注所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划图。
  35. 根据权利要求34所述的无人机，其特征在于，所述目标图像包括下述中的一种或多种：
    全黑图像、全白图像、所述多光谱图像或所述多光谱图像中部分光谱信息对应的图像。
  36. 根据权利要求34所述的无人机，其特征在于，所述诊断信息包括病 虫害分布的诊断信息，所述作业规划图为施药规划图。
  37. 根据权利要求34所述的无人机，其特征在于，所述诊断信息包括营养元素含量的诊断信息，所述作业规划图为施肥规划图。
  38. 根据权利要求31所述的无人机，其特征在于，所述根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划，包括：
    根据所述作业位置与作业量的对应关系，生成规划的作业路线以及所述作业路线对应的作业参数。
  39. 根据权利要求38所述的无人机，其特征在于，所述方法还包括：
    根据所述作业规划，在所述目标区域执行作业。
  40. 根据权利要求24所述的无人机，其特征在于，所述方法还包括：展示所述作业规划。
  41. 一种地面端设备，用于与搭载有多光谱图像获取装置的无人机通信连接，所述无人机在目标区域飞行并获得地表目标区域的多光谱图像；其特征在于，所述地面端设备具有存储器和处理器，所述存储器中存储有指令，所述指令被调用以执行如下步骤：
    根据所述多光谱图像，获得所述地表目标区域的诊断信息和包含地表语义信息的第一特征图；
    根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划。
  42. 根据权利要求41所述的地面端设备，其特征在于，所述根据所述多光谱图像，获得所述地表目标区域的诊断信息，包括：
    根据所述多光谱图像的部分光谱信息，计算所述地表目标区域的诊断信息。
  43. 根据权利要求41所述的设备，其特征在于，所述根据所述多光谱图像，获得所述地表目标区域的诊断信息包括：
    使用预训练的第一神经网络模型处理所述多光谱图像，以生成表征所述地表目标区域各个位置点的诊断信息的第二特征图。
  44. 根据权利要求41所述的设备，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    使用预训练的第二神经网络模型处理所述多光谱图像，获得包含地表语义信息的第一特征图。
  45. 根据权利要求41所述的设备，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    根据所述多光谱图像中部分光谱信息，获得包含地表语义信息的第一特征图。
  46. 根据权利要求41所述的设备，其特征在于，所述部分光谱信息包括红光谱、绿光谱和蓝光谱的信息。
  47. 根据权利要求41所述的设备，其特征在于，所述根据所述多光谱图像，获得包含地表语义信息的第一特征图，包括：
    根据所述多光谱图像以及所述地表目标区域的深度图，获得包含地表语义信息的第一特征图。
  48. 根据权利要求41所述的设备，其特征在于，所述根据所述诊断信息和所述第一特征图，生成所述地表目标区域的作业规划，包括：
    根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系；
    根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划。
  49. 根据权利要求48所述的设备，其特征在于，所述诊断信息包括至少一个区域分别的诊断信息；
    所述根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系，包括：
    根据所述至少一个区域分别的诊断信息以及所述第一特征图，确定作业位置与诊断信息的对应关系；
    根据作业位置与诊断信息的对应关系，以及不同诊断信息等级与作业量的对应关系，按照作业量线性过渡的策略，得到作业位置与作业量的对应关系。
  50. 根据权利要求49所述的设备，其特征在于，所述诊断信息为表征所述地表目标区域各个位置点的诊断信息的第二特征图；
    所述根据所述诊断信息和所述第一特征图，确定作业位置与作业量的对应关系，包括：
    根据所述第二特征图以及所述第一特征图，确定作业位置与诊断信息的对应关系；
    根据所述作业位置与诊断信息的对应关系以及不同诊断信息与作业量的对应关系，得到作业位置与作业量的对应关系。
  51. 根据权利要求48所述的设备，其特征在于，所述根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划，包括：
    在目标图像中标注所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划图。
  52. 根据权利要求51所述的设备，其特征在于，所述目标图像包括下述中的一种或多种：
    全黑图像、全白图像、所述多光谱图像或所述多光谱图像中部分光谱信息对应的图像。
  53. 根据权利要求51所述的设备，其特征在于，所述诊断信息包括病虫害分布的诊断信息，所述作业规划图为施药规划图。
  54. 根据权利要求51所述的设备，其特征在于，所述诊断信息包括营养元素含量的诊断信息，所述作业规划图为施肥规划图。
  55. 根据权利要求48所述的设备，其特征在于，所述根据所述作业位置与作业量的对应关系，生成所述地表目标区域的作业规划，包括：
    根据所述作业位置与作业量的对应关系，生成规划的作业路线以及所述作业路线对应的作业参数。
  56. 根据权利要求55所述的设备，其特征在于，所述方法还包括：
    将所述作业规划发送至农业无人机。
  57. 根据权利要求41所述的设备，其特征在于，所述方法还包括：展示所述作业规划。
PCT/CN2019/115414 2019-11-04 2019-11-04 结合多光谱和地表语义的作业规划方法、装置及设备 WO2021087685A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2019/115414 WO2021087685A1 (zh) 2019-11-04 2019-11-04 结合多光谱和地表语义的作业规划方法、装置及设备
CN201980033739.3A CN112204569A (zh) 2019-11-04 2019-11-04 结合多光谱和地表语义的作业规划方法、装置及设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/115414 WO2021087685A1 (zh) 2019-11-04 2019-11-04 结合多光谱和地表语义的作业规划方法、装置及设备

Publications (1)

Publication Number Publication Date
WO2021087685A1 true WO2021087685A1 (zh) 2021-05-14

Family

ID=74004610

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/115414 WO2021087685A1 (zh) 2019-11-04 2019-11-04 结合多光谱和地表语义的作业规划方法、装置及设备

Country Status (2)

Country Link
CN (1) CN112204569A (zh)
WO (1) WO2021087685A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113436248A (zh) * 2021-06-18 2021-09-24 黑龙江惠达科技发展有限公司 计算农机作业面积的方法和装置
US20230077353A1 (en) * 2021-08-31 2023-03-16 University Of South Florida Systems and Methods for Classifying Mosquitoes Based on Extracted Masks of Anatomical Components from Images
CN115963857A (zh) * 2023-01-04 2023-04-14 广东博幻生态科技有限公司 一种基于无人机的农药喷洒方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022257139A1 (zh) * 2021-06-11 2022-12-15 深圳市大疆创新科技有限公司 植物状态确定方法、终端和计算机可读存储介质
CN113724210B (zh) * 2021-08-13 2024-08-06 广州华农大智慧农业科技有限公司 一种作物长势识别方法和系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106585992A (zh) * 2016-12-15 2017-04-26 上海土是宝农业科技有限公司 一种无人机智能识别、精确喷洒农药的方法及系统
CN108045582A (zh) * 2017-12-15 2018-05-18 佛山市神风航空科技有限公司 一种用于检测、防治农作物病害的主副无人机系统
CN108267175A (zh) * 2018-02-06 2018-07-10 首欣（北京）科技有限公司 一种基于无人机的农作物监测方法和装置
US20180307906A1 (en) * 2017-04-19 2018-10-25 Sentera, Llc Multiband filtering image collection and analysis
CN110297483A (zh) * 2018-03-21 2019-10-01 广州极飞科技有限公司 待作业区域边界获取方法、装置，作业航线规划方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106585992A (zh) * 2016-12-15 2017-04-26 上海土是宝农业科技有限公司 一种无人机智能识别、精确喷洒农药的方法及系统
US20180307906A1 (en) * 2017-04-19 2018-10-25 Sentera, Llc Multiband filtering image collection and analysis
CN108045582A (zh) * 2017-12-15 2018-05-18 佛山市神风航空科技有限公司 一种用于检测、防治农作物病害的主副无人机系统
CN108267175A (zh) * 2018-02-06 2018-07-10 首欣（北京）科技有限公司 一种基于无人机的农作物监测方法和装置
CN110297483A (zh) * 2018-03-21 2019-10-01 广州极飞科技有限公司 待作业区域边界获取方法、装置，作业航线规划方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113436248A (zh) * 2021-06-18 2021-09-24 黑龙江惠达科技发展有限公司 计算农机作业面积的方法和装置
US20230077353A1 (en) * 2021-08-31 2023-03-16 University Of South Florida Systems and Methods for Classifying Mosquitoes Based on Extracted Masks of Anatomical Components from Images
CN115963857A (zh) * 2023-01-04 2023-04-14 广东博幻生态科技有限公司 一种基于无人机的农药喷洒方法
CN115963857B (zh) * 2023-01-04 2023-08-08 广东博幻生态科技有限公司 一种基于无人机的农药喷洒方法

Also Published As

Publication number Publication date
CN112204569A (zh) 2021-01-08

Similar Documents

Publication Publication Date Title
WO2021087685A1 (zh) 结合多光谱和地表语义的作业规划方法、装置及设备
US11440659B2 (en) Precision agriculture implementation method by UAV systems and artificial intelligence image processing technologies
US10719787B2 (en) Method for mapping crop yields
Khan et al. Estimation of vegetation indices for high-throughput phenotyping of wheat using aerial imaging
CN111406261B (zh) 一种从大视野图像中检测被感染对象的计算机实现的方法
US10846843B2 (en) Utilizing artificial intelligence with captured images to detect agricultural failure
US20200193589A1 (en) Mapping field anomalies using digital images and machine learning models
Pilli et al. eAGROBOT—A robot for early crop disease detection using image processing
CN113228047A (zh) 利用多阶段、多尺度深度学习的植物病害检测
CN109478232A (zh) 自然环境中的杂草的识别
CN111008733B (zh) 一种作物生长管控方法和系统
WO2020103108A1 (zh) 一种语义生成方法、设备、飞行器及存储介质
WO2020103109A1 (zh) 一种地图生成方法、设备、飞行器及存储介质
JP2018046787A (ja) 農業管理予測システム、農業管理予測方法、及びサーバ装置
WO2021109120A1 (zh) 作物生长状况评估方法和装置
US20200134358A1 (en) Detecting infection of plant diseases with improved machine learning
CN112528912A (zh) 基于边缘计算的作物生长监测嵌入式系统及方法
JP2019153109A (ja) 農業管理予測システム、農業管理予測方法、及びサーバ装置
JP2020149201A (ja) 作物の倒伏リスク診断に用いる生育パラメータの測定推奨スポット提示方法、倒伏リスク診断方法、および情報提供装置
CN110472596A (zh) 一种农业精细化种植及灾害预防控制系统
CN110796148B (zh) 一种荔枝虫害监测识别系统和监测识别方法
JP2024144597A (ja) 作業マップ作成システム、作業マップの生成方法及び作業マップ生成プログラム
CN117575835A (zh) 基于无人机航拍的茶园长势监控系统及方法
CN112565726B (zh) 作业处方图的确定方法、作业控制方法及相关装置
Juan et al. Rapid density estimation of tiny pests from sticky traps using Qpest RCNN in conjunction with UWB-UAV-based IoT framework

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19952133

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19952133

Country of ref document: EP

Kind code of ref document: A1