WO2021056297A1

WO2021056297A1 - Image processing method and device, unmanned aerial vehicle, system and storage medium

Info

Publication number: WO2021056297A1
Application number: PCT/CN2019/107977
Authority: WO
Inventors: 龚云; 潘国秀
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-09-25
Filing date: 2019-09-25
Publication date: 2021-04-01
Also published as: CN112106346A

Abstract

An image processing method and device, an unmanned aerial vehicle, a system and a storage medium. The method comprises: acquiring the illumination intensity of each of a plurality of preset bands of the current environment (101); determining, according to the illumination intensity of each of the plurality of preset bands, the illumination intensity corresponding to at least one spectral band of a multispectral camera (102); and performing, according to the illumination intensity corresponding to the spectral band of the multispectral camera, image compensation processing on the current image acquired by the multispectral camera (103). By means of the image processing method and device, the unmanned aerial vehicle, the system and the storage medium, the imaging effect of an image can be improved.

Description

Image processing method, equipment, drone, system and storage medium

Technical field

The present invention relates to the field of images, in particular to an image processing method, equipment, unmanned aerial vehicle, system and storage medium.

Background technique

At present, when a drone is flying over an object, it can obtain an aerial image of the object. Due to some shooting reasons, the imaging effect of the image will be poor. In order to improve the imaging effect, the image can be compensated.

In order to effectively compensate for the aerial image, the UAV also needs to obtain multiple image compensation information when acquiring the aerial image, but not all the image compensation information can be obtained to compensate the image more accurately, which is easy to cause larger compensation error.

Summary of the invention

The present invention provides an image processing method, equipment, unmanned aerial vehicle, system and storage medium, which are used to solve the problem of more accurate image compensation.

The first aspect of the present invention is to provide an image processing method, including: acquiring the light intensity of multiple preset wavebands of the current environment; and determining at least the light intensity of the multispectral camera according to the light intensity of the multiple preset wavebands The light intensity corresponding to a spectral waveband; according to the light intensity corresponding to the spectral waveband of the multi-spectral camera, image compensation processing is performed on the current image acquired by the multi-spectral camera.

The second aspect of the present invention is to provide an image processing device, including: a light collection device for acquiring light intensity of multiple preset wavebands of the current environment; receiving light intensity of multiple incident wavebands; the image processing device It also includes: a memory and a processor, the memory is configured to store a computer program; the processor is configured to determine, according to the light intensity of the plurality of preset wavelength bands, at least one spectral band corresponding to the multispectral camera Light intensity; according to the light intensity corresponding to the spectral band of the multi-spectral camera, perform image compensation processing on the current image acquired by the multi-spectral camera.

The third aspect of the present invention is to provide an image processing device, including: an acquisition module for acquiring the light intensity of multiple preset wavebands of the current environment; The light intensity determines the light intensity corresponding to at least one spectral band of the multi-spectral camera; the compensation module is configured to perform image compensation on the current image acquired by the multi-spectral camera according to the light intensity corresponding to the spectral band of the multi-spectral camera deal with.

The fourth aspect of the present invention is to provide an unmanned aerial vehicle, including: a body and the image processing device according to the second aspect, the image processing device is provided on the body.

The fifth aspect of the present invention is to provide a computer-readable storage medium, the storage medium is a computer-readable storage medium, the computer-readable storage medium stores program instructions, and the program instructions are used in the first aspect. The image processing method described.

The sixth aspect of the present invention is to provide an image processing method, including: receiving, through a drone, the illumination intensity of multiple preset wavebands of the current environment and the illumination intensity corresponding to at least one spectral waveband of the multispectral camera, the at least The light intensity corresponding to a spectral waveband is determined according to the light intensity of the multiple preset wavebands; according to the light intensity corresponding to the spectral waveband of the multispectral camera, the current image acquired by the multispectral camera is imaged Compensation processing; when the processed image is an image of vegetation, the vegetation growth index is determined according to the band feature extraction of the processed image.

The seventh aspect of the present invention is to provide an image processing system, including: a memory, a processor, and a communication component. The memory is used to store a computer program; the communication component is used to receive the light intensity of multiple preset wavebands of the current environment and the light intensity corresponding to at least one spectral waveband of the multispectral camera through the drone, and the at least The light intensity corresponding to a spectral band is determined according to the light intensity of the multiple preset wavebands; the processor is configured to perform a measurement on the multi-spectral camera according to the light intensity corresponding to the spectral waveband of the multi-spectral camera. The current image acquired by the camera is subjected to image compensation processing; when the processed image is an image of vegetation, the vegetation growth index is determined according to the band feature extraction of the processed image.

The eighth aspect of the present invention is to provide a computer-readable storage medium, characterized in that the storage medium is a computer-readable storage medium, and the computer-readable storage medium stores program instructions, and the program instructions are used for Implement the image processing method described in the sixth aspect.

The ninth aspect of the present invention is to provide an image processing device, including: a receiving module for receiving, through the drone, the illumination intensity of multiple preset wavebands of the current environment and the illumination corresponding to at least one spectral waveband of the multispectral camera Intensity, the light intensity corresponding to at least one spectral band is determined according to the light intensity of multiple preset bands; the compensation module is used to image the current image acquired by the multi-spectral camera according to the light intensity corresponding to the spectral band of the multi-spectral camera Compensation processing; extraction module, used when the processed image is an image of vegetation, and determine the vegetation growth index according to the band feature extraction of the processed image.

The image processing method, equipment, drone, system and storage medium provided by the invention can improve the imaging effect of the image.

Description of the drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The exemplary embodiments and descriptions of the application are used to explain the application, and do not constitute an improper limitation of the application. In the attached picture:

FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of a process for determining light intensity according to an embodiment of the present invention;

3 is a schematic diagram of a specific flow of image compensation processing provided by an embodiment of the present invention;

4 is a schematic diagram of a specific flow of image compensation processing provided by an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention;

FIG. 7 is a schematic flowchart of an image processing method provided by an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an image processing system provided by an embodiment of the present invention.

detailed description

In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

In order to facilitate the understanding of the technical solutions and technical effects of this application, a brief description of the prior art is given below:

The drone can carry multiple imaging sensors. When the drone is flying above the object, the imaging sensor can generate the corresponding aerial image of the object through the reflected light of the object. At the same time, the UAV can also carry a light sensor. When the UAV is flying above the object, it can provide multiple real-time incident light intensity indexes of different bands. By selecting the real-time incident light intensity index, the aerial image can be compensated for compensation. Aerial image of the object behind. However, this compensation method is likely to cause large compensation errors, and the compensation effect often obtained is not ideal.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. As long as there is no conflict between the various embodiments, the following embodiments and the features in the embodiments can be combined with each other.

Fig. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present invention; the method 100 provided by an embodiment of the present application can be executed by a flying device, such as a drone, and the drone can be provided with a light collection device, such as Light sensor. Multiple image acquisition devices, such as multiple image sensors, can also be provided on the drone. Multiple image acquisition devices can be provided in the camera, and the camera can be provided on the drone. In multiple image acquisition devices, there may be several narrowband image sensors, and the narrowband wavebands corresponding to each narrowband image sensor may be completely different. The method 100 includes the following steps:

Step 101: Obtain the light intensity of multiple preset wavebands in the current environment.

Step 102: Determine the light intensity corresponding to at least one spectral waveband of the multispectral camera according to the light intensity of multiple preset wavebands.

Step 103: Perform image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera.

The image processing method provided by the present invention determines the light intensity corresponding to at least one spectral waveband of the multispectral camera according to the acquired light intensity of multiple preset wavebands; according to the light intensity corresponding to the spectral waveband of the multispectral camera, The current image acquired by the multispectral camera is subjected to image compensation processing, thereby solving the problem that the light intensity of the preset waveband cannot meet the more accurate compensation of the current image. At the same time, it can be selected from the light intensity of multiple preset wavebands. It can more accurately compensate the light intensity of the image.

According to the determined light intensity, performing image compensation processing on the current image can make the compensated image more accurately approach the real image, reduce compensation errors, and improve the imaging effect of the image. So as to further provide more accurate data for data analysis in subsequent images.

It should be noted that, in addition to being applied to drones, the embodiments of this application can also be applied to other equipment with shooting requirements, such as cameras, mobile phones, computers, monitoring equipment, etc., no matter what kind of execution device, the execution device Both can have a light collection device, and can also have multiple image collection devices as well.

For other execution devices, when the drone executes the embodiments of the present application, it can capture aerial images of objects, especially aerial images of vegetation, so that the distribution and growth conditions of vegetation can be analyzed. In addition, drones have better flexibility and automation compared to other execution equipment. And it can fly into a very harsh geographic environment to take images, which is not easy to do with other execution equipment.

The following is a detailed description of the above steps:

Among them, the band refers to the division of the electromagnetic spectrum, especially the division of the light spectrum, for example, the 840nm (nanometer) band. The preset waveband (also referred to as the incident waveband) refers to multiple known wavebands that are directly incident or irradiated by light waves in the current environment, for example, the 860nm waveband. In the embodiment of the present application, the preset waveband (that is, the light intensity of multiple preset wavebands is obtained from the light sensor) can be received by the light collection device provided on the drone. The light collection device can provide multiple preset wavebands of light intensity, such as 18, then when the light collection device acquires the preset waveband, it can directly obtain the waveband that meets the preset waveband length, for example, the preset 18 wavebands . Or when the light collecting device can collect a waveband within a range of waveband length, it can sequentially obtain 18 preset wavebands existing in the waveband length in the order of receiving. It should be understood that one preset waveband corresponds to one light intensity.

Illumination intensity refers to the energy of visible light received per unit area, referred to as illuminance; it can be used to indicate the intensity of illumination and the amount of illumination of the surface area of an object.

For example, when the drone is flying over the vegetation, it can receive the light intensity of 18 different light waves in the natural environment in real time through the light sensor set on the drone, such as the light intensity of 860nm and the light intensity of 810nm. ……Wait.

Among them, a multi-spectral camera refers to a camera that can receive multiple preset spectral bands (that is, spectral bands), and the camera can perform imaging according to the received multiple preset spectral bands. The received spectral band (also called the received reflection band) refers to the light wave irradiated on an object, such as vegetation, and the light wave band reflected by the vegetation, such as the 840nm (nanometer) band. In the embodiment of the present application, the light intensity of the spectral waveband can be obtained by using an image acquisition device provided in a multispectral camera, such as a narrowband image sensor. Each image acquisition device corresponds to a spectral band, which can also be referred to as the working band of the image acquisition device.

Specifically, FIG. 2 shows a method 200 for determining light intensity. The method for determining light intensity corresponding to multiple spectral bands may include: Step 201: When multiple preset wavebands in the current environment and the multi-spectral camera receive When the two spectral bands do not match, two first preset wavebands are selected from a plurality of preset wavebands; Step 202: Determine the first light intensity corresponding to the first spectral waveband of the multispectral camera according to the two first preset wavebands.

Among them, the first spectral band belongs to one of the multiple spectral bands, and its light intensity is the first light intensity. The first spectral band does not match each of the plurality of preset bands. The first preset waveband is at least one waveband among a plurality of preset wavebands, and the first preset waveband does not match the first spectral waveband.

It should be understood that the above-mentioned mismatch may be a complete mismatch or an incomplete match, that is, a partial mismatch and a partial match exist at the same time. When it does not match, the first preset waveband is different from the first spectral waveband.

For example, according to the foregoing, the UAV compares the narrowband bands (that is, spectral bands) corresponding to 5 narrowband image sensors with 18 different preset bands, when there are at least one narrowband band and 18 narrowband bands in the 5 narrowband bands. Each of the different preset wavebands is different. For example, the narrowband waveband 840nm does not exist in the 18 different preset wavebands. The UAV selects the light intensity of any two preset wavebands from the light intensity of 18 different preset wavebands, such as the light intensity of 860nm and the light intensity of 810nm. Determine the average value of the light intensity of the two preset wavebands as the first light intensity.

In addition, in order to improve the accuracy of image compensation, the method for determining the first light intensity may include the following two:

1) The linear interpolation method can be used to calculate the first light intensity according to the two first preset wavebands.

2) Determine the first light intensity according to the ratio of the difference between the light intensity of the two first preset wavebands and the difference between the two first preset wavebands.

Among them, the linear interpolation method can be determined by the following formula 1):

Among them, x is the first spectral band, x ₁ , x ₂ are two different preset bands, f(x) is the first light intensity corresponding to the first spectral band, f(x ₁ ) is x ₁ The light intensity corresponding to the preset waveband x ₁ _{, f(x 2} ) represents the light intensity corresponding to the preset wave band x _2. And the preset band x ₁ ≠ the preset band x ₂ .

After the above formula 1) is transformed, the light intensity corresponding to the determined spectral band is determined by the following formula 2):

According to the foregoing, the UAV compares the narrowband bands (that is, spectral bands) corresponding to the 5 narrowband image sensors with 18 different preset bands. When there are at least one narrowband band and 18 different bands in the 5 narrowband bands Each of the preset wavebands is different. For example, the narrowband waveband 840nm does not exist in 18 different preset wavebands. Then the drone selects the light intensity of any two different preset wavebands from the light intensity of 18 different preset wavebands, such as the light intensity of 860nm and the light intensity of 810nm. The UAV determines the value of the first light intensity f(x) in the narrowband band of 840nm through the above formula 2).

It should be noted that, in addition to the linear interpolation method to determine the light intensity of the narrowband band, it can also be determined by other methods, such as curve interpolation. The above embodiments are only for the purpose of explanation, and do not limit the present invention. Other implementations that can determine the light intensity corresponding to the spectral waveband of the multispectral camera according to the light intensity of the preset waveband fall within the protection scope of the present invention.

In addition, the method of determining the first light intensity according to the ratio in the manner 2) can also be implemented through the specific implementation manner of the manner 1), which will not be repeated here.

The above-mentioned implementation basis for determining the light intensity has been able to determine the light intensity in the spectral band with a relatively precise degree. In order to determine the light intensity of the spectral band more accurately, when selecting the two first preset bands, you can choose the one closest to the first spectral band (that is, among the multiple bands, the two first preset bands are the closest The first spectral band).

For example, according to the foregoing, after determining that the narrowband band 840nm does not exist in 18 different preset bands, the UAV can select the two presets closest to 840nm from the light intensity of the 18 different preset bands. Bands, 860nm and 810nm. Then, determine the value of the first light intensity f(x) in the narrowband wavelength band of 840nm through the above formula 2).

In addition, when the drone determines that at least one narrowband band exists in 18 different preset bands, determining the light intensity corresponding to multiple spectral bands includes: obtaining the first spectral band from a multi-spectral camera; When the second preset waveband in the preset waveband matches the first spectral waveband of the multispectral camera, the first light intensity of the first spectral waveband is directly determined according to the second light intensity of the second preset waveband.

Wherein, if the second preset waveband among the multiple preset wavebands of the current environment matches the first spectral waveband of the multispectral camera, the second preset waveband is the same as the first spectral waveband.

For example, according to the foregoing, if the UAV determines that the narrowband band 840nm exists in 18 different preset bands, that is, the UAV receives the light intensity of the preset band 840nm through the light sensor, then the UAV The light intensity of the preset wavelength band of 840 nm is directly obtained as the light intensity of the narrowband wavelength band of 840 nm.

Since the monochrome image generated by the narrowband image sensor may have the problem of bad pixels, and the lack of clarity of each object in the image, it is necessary to compensate the monochrome image in order to facilitate the subsequent extraction of various information in the image. .

Wherein, the light intensity corresponding to the spectral band of the multispectral camera may refer to the light intensity determined according to the foregoing embodiment, such as the light intensity determined by linear interpolation. It can also refer to determining the light intensity and the received light intensity of the spectral band, and the received light intensity is used to determine the image before compensation, that is, the current image.

Wherein, the method of acquiring the current image may include: the multispectral camera receives the light intensity of multiple spectral bands, and obtains the light intensity of at least one preset spectral band from the received light intensity of the multiple spectral bands; One light intensity generates at least one image; according to the at least one image, the current image is acquired (that is, the second light intensity corresponding to the first spectral band is acquired through the multispectral camera, and the current image is acquired according to the second light intensity).

Wherein, according to at least one image, the way to obtain the current image may include:

1). Directly use at least one image as the current image. For example, for the same vegetation, 5 images are generated, and these 5 images are directly used as a current image without combining.

2). Combine at least one image into the current image. For example, for different vegetations in the same area, and different vegetations are adjacent to each other, 5 images are generated, then these 5 images can be formed into a continuous image according to the adjacent relationship, so that these 5 images are formed into 1 current image .

For example, according to the foregoing, the drone flies over the vegetation, and receives the light intensity of 20 vegetation reflection bands through the 5 narrowband image sensors in its multi-spectral camera. The narrowband band corresponding to each narrowband image sensor can be different. The narrowband band corresponding to the A narrowband image sensor is 450nm (that is, the spectral band), the narrowband band corresponding to the B narrowband image sensor is 560nm, and the narrowband band corresponding to the C narrowband image sensor is 640nm. The narrowband waveband corresponding to the D narrowband image sensor is 730nm, and the narrowband waveband corresponding to the E narrowband image sensor is 840nm. Each narrowband image sensor recognizes the corresponding narrowband spectral band and the light intensity of the spectral band. Take the 450nm narrowband image sensor corresponding to the narrowband A as an example to illustrate the process of obtaining a vegetation image. The narrowband image sensor A recognizes the light intensity at the 450nm spectral band, and converts the light intensity into electric charge through the many photosensitive units of the image sensor. In the process of converting the charge into a digital signal, each photosensitive unit can also be called a pixel unit. According to these digital signals, the corresponding monochrome image in TIFF format is determined, such as a red image of vegetation growing in aerial photography. By analogy, each narrowband image sensor generates a corresponding monochrome image of vegetation aerial growth. At this time, the 5 vegetation aerial growth monochrome images can be directly used as one target image, waiting for image compensation for each monochrome image, or they can be used as five target images. Alternatively, multiple images can be combined according to the spatial position of each image sensor to synthesize a complete vegetation aerial growth monochrome image. For example, if five narrowband image sensors are arranged horizontally in sequence, the monochrome images generated by each image sensor are arranged horizontally to synthesize a complete monochrome image.

It should be understood that only one narrowband image sensor can be provided in the drone, but when only one narrowband image sensor is provided, the monochrome image obtained by the narrowband image sensor is directly used as the image that needs to be compensated later, and there is no need to perform image processing. synthesis.

In addition, the narrowband image sensor may be a 2 million pixel global shutter image sensor, and may have a certain spatial arrangement layout.

Specifically, image compensation processing for the current image may include the following two types:

Manner 1) and Fig. 3 show a method 300 of image compensation processing, which includes: Step 301: Compensate for the second light intensity according to the first light intensity; Step 302: Compensate for the current image according to the compensated second light intensity Perform image compensation processing.

For example, according to the foregoing, after the UAV determines the light intensity corresponding to the narrowband waveband, it can directly use the value of the light intensity as the digital signal obtained by the corresponding narrowband image sensor (for example, a narrowband image sensor with a narrowband waveband of 840nm). Data, that is, compensated digital signal. The compensated digital signal is combined with the original digital signal corresponding to the narrowband 840nm (ie, the digital signal corresponding to the band before image compensation), and a monochrome image is generated according to the data of the combined digital signal.

Manner 2) and Figure 4 show a method 400 of image compensation processing, which includes: Step 401: Obtain the second light intensity corresponding to the first spectral band from the multispectral camera; Step 402: According to the first light intensity and the second light intensity Intensity, perform image compensation processing on the current image acquired by the multispectral camera.

For example, according to the foregoing, the drone can generate the current image according to the acquired second light intensity. The UAV directly generates a compensated monochrome image from the compensated digital signal data, and combines the compensated monochrome image with the current image to complete the image compensation. It can also be implemented according to the specific implementation in the above manner 1), which will not be repeated here.

It should be understood that the second light intensity corresponding to the first spectral waveband can be used to generate the current image, and the first light intensity corresponding to the first spectral waveband refers to the light intensity used to compensate the current image.

After the image is compensated, the compensated image can be more accurate, especially closer to the real image, or the real appearance of the object, especially the real aerial appearance of the object. After obtaining the compensated image, the compensated image can be processed differently according to different subsequent application scenarios. For example, in an application scenario of vegetation growth, the vegetation growth condition can be determined according to the compensated image. It is also possible to identify and track things to be monitored on the compensated image in the monitoring application scenario. It is also possible to retouch the compensated image in the application scene of photography. It is worth noting that in some application scenes, such as monitoring application scenes and photographing application scenes, it is possible to further perform color image processing on the compensated monochrome image, that is, to de-monochrome the compensated monochrome image. Color processing to obtain a compensated color image.

In the embodiment of the present application, an application scenario of vegetation growth is taken as an example to describe the image processing after compensation:

The processing method may include: when the processed image is an image of vegetation, the vegetation growth index is determined according to the extraction of the band characteristics of the processed image.

Among them, the vegetation growth index (NDVI, Normalized Difference Vegetation Index) refers to the growth state of vegetation (one of the important parameters reflecting vegetation growth and nutritional information), vegetation coverage, and the elimination of some radiation errors. It can reflect the background influence of the plant canopy, such as soil, wet ground, snow, dead leaves, roughness, etc., and is related to vegetation cover.

The extraction process can be as follows. After the UAV obtains the compensated vegetation aerial monochromatic image, extracts the spectral features of the monochromatic image, and determines the NDVI based on the spectral features. Among them, NDVI is the sum of the difference between the reflection value of the near-infrared waveband and the reflection value of the red light waveband.

It should be noted that in addition to the drone that can directly determine the NDVI, the NDVI can also be determined by other devices. When the NDVI is determined by other devices, such as image recognition devices (specifically, computers and servers), no The human-computer sends the compensated vegetation aerial monochrome image to the image recognition device through the network, and the image recognition device determines the NDVI. The specific implementation of the determination process is the same as the specific implementation of the drone to determine the NDVI. Go into details again.

The embodiments of the present application may be applicable to different drone models, and the NDVI obtained by aerial photography of the same vegetation at different flight times by different drones is the same. Therefore, it is possible to detect the NDBVI of a certain vegetation obtained in a certain aerial photography according to this characteristic. The detection process may include: determining multiple vegetation growth indexes corresponding to the same vegetation through different flight equipment and/or different flight times; If multiple vegetation growth indexes are the same, confirm that the vegetation growth index is correct. When it is determined that multiple NDVI of the same vegetation are the same, the NDVI is correct.

FIG. 5 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention; the device 500 can be applied to flying equipment, such as a drone, and the device 500 can execute the above-mentioned image processing method. Wherein, the device 500 includes: an acquisition module 501, a determination module 502, and a compensation module 503. The following is a detailed description of the functions of each module:

The obtaining module 501 is configured to obtain the light intensity of multiple preset wavebands in the current environment.

The determining module 502 is configured to determine the light intensity corresponding to at least one spectral waveband of the multispectral camera according to the light intensity of multiple preset wavebands.

The compensation module 503 is configured to perform image compensation processing on the current image acquired by the multi-spectral camera according to the light intensity corresponding to the spectral band of the multi-spectral camera.

It should be noted that, for parts that are not described in detail in this embodiment, reference may be made to the relevant description of the embodiment shown in the image processing method above, and details are not repeated here.

In a possible design, the structure of the image processing apparatus 500 shown in FIG. 5 may be implemented as an electronic device, and the electronic device may be an image processing device, such as various devices such as a mobile phone, a tablet computer, and a server. As shown in FIG. 6, the image processing device 600 may include: one or more processors 601, one or more memories 602, and a light collecting device 603. Wherein, the memory 602 is used to store a program that supports the electronic device to execute the image processing method provided in the embodiment shown in FIG. 1 to FIG. 4 above. The electronic device may also include a camera 604; the camera includes an image acquisition device; and the processor 601 is configured to execute a program stored in the memory 602. Specifically, the program includes one or more computer instructions, where one or more computer instructions can implement the following steps when executed by the processor 601:

Run the computer program stored in the memory to realize: determine the light intensity corresponding to at least one spectral band of the multi-spectral camera according to the light intensity of multiple preset bands; according to the light intensity corresponding to the spectral band of the multi-spectral camera, The acquired current image is subjected to image compensation processing.

The light collecting device 603 is configured to obtain the light intensity of multiple preset wavebands in the current environment; and receive the light intensity of multiple incident wavebands.

Specifically, the processor 601 is specifically configured to: when multiple preset wavebands of the current environment and multiple spectral wavebands received by the multispectral camera do not match, select two first preset wavebands from the multiple preset wavebands , And determine the first light intensity corresponding to the first spectral waveband of the multispectral camera according to the two first preset wavebands.

Further, the image processing device 600 further includes: a communication component; the communication component is configured to obtain the second light intensity corresponding to the first spectral band through the multi-spectral camera, and obtain the current image according to the second light intensity.

The processor 601 is specifically configured to: compensate the second light intensity according to the first light intensity; and perform image compensation processing on the current image according to the compensated second light intensity.

Specifically, the communication component is used to obtain the second light intensity corresponding to the first spectral band from the multi-spectral camera.

The processor 601 is specifically configured to: perform image compensation processing on the current image acquired by the multispectral camera according to the first light intensity and the second light intensity.

Further, the processor 601 is further configured to: use a linear interpolation method to calculate the first light intensity according to the two first preset wavebands.

Specifically, the processor 601 is specifically configured to determine the first light intensity according to the ratio of the difference between the light intensity of the two first preset wavebands and the difference between the two first preset wavebands.

Further, among the multiple wavebands, the two first preset wavebands are closest to the first spectral waveband.

Further, the communication component is also used to: obtain the first spectral band from the multispectral camera;

The processor 601 is specifically configured to: when the second preset waveband of the multiple preset wavebands of the current environment matches the first spectral waveband of the multispectral camera, directly determine the second light intensity of the second preset waveband The first light intensity in a spectral band.

Specifically, the light intensity of a plurality of preset wavebands is obtained from the light sensor.

Further, the processor 601 is further configured to: when the processed current image is an image about vegetation, determine the vegetation growth index according to the band feature extraction of the processed image.

Further, the processor 601 is further configured to: determine multiple vegetation growth indices corresponding to the same vegetation through different flying equipment and/or different flight times; when the multiple vegetation growth indices are the same, determine that the vegetation growth index is correct.

In addition, an embodiment of the present invention provides a computer-readable storage medium, the storage medium is a computer-readable storage medium, the computer-readable storage medium stores program instructions, and the program instructions are used to implement the images shown in FIGS. 1 to 4 above. Approach.

An embodiment of the present invention provides an unmanned aerial vehicle; specifically, the unmanned aerial vehicle includes a body and the image processing device shown in FIG. 6, and the image processing device is arranged on the body.

Further, the UAV also includes: a multi-spectral camera; a multi-spectral camera, used to provide the image processing device with a current image and used to determine the light intensity of the current image; the light intensity is determined by the spectral band obtained by the multi-spectral camera.

FIG. 7 is a schematic flowchart of an image processing method provided by an embodiment of the present invention; the method 700 provided by an embodiment of the present application is executed by an image processing system with computing capabilities, such as a computer and a server, etc., or a virtual server, Cloud server, etc. The method 700 includes the following steps:

701: Receive the light intensity of multiple preset wavebands of the current environment and the light intensity corresponding to at least one spectral waveband of the multispectral camera through the drone, and the light intensity corresponding to at least one spectral waveband is based on the light intensity of the multiple preset wavebands definite.

702: Perform image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera.

703: When the processed image is an image of vegetation, the vegetation growth index is determined according to the band feature extraction of the processed image.

It should be noted that the specific implementation manners of step 701 to step 703 have been described in detail in the foregoing, and will not be repeated here. For illustration only, the method 700 can be executed by a device, such as a computer. When the computer is executed, it needs to receive the light intensity of multiple preset wavebands and the light intensity of at least one spectral waveband sent by the drone. The light intensity of multiple preset wavebands and the light intensity of at least one spectral waveband are obtained by the drone through the specific implementation described above. After receiving this information, the computer can perform image compensation and determine the vegetation growth index.

FIG. 8 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention; the device 800 can be applied to image processing systems with computing capabilities, such as computers and servers, and also virtual servers, cloud servers, etc. The device 800 can execute the above-mentioned image processing method. Wherein, the device 800 includes: a receiving module 801, a compensation module 802, and an extraction module 803. The following is a detailed description of the functions of each module:

The receiving module 801 is configured to receive the light intensity of multiple preset wavebands of the current environment and the light intensity corresponding to at least one spectral waveband of the multispectral camera through the drone, and the light intensity corresponding to the at least one spectral waveband is based on the multiple presets The light intensity of the band is determined.

The compensation module 802 is configured to perform image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera.

The extraction module 803 is used to determine the vegetation growth index according to the extraction of the band characteristics of the processed image when the processed image is an image about vegetation.

In a possible design, the structure of the image processing apparatus 800 shown in FIG. 8 can be implemented as an electronic device, and the electronic device can be an image processing system, such as various devices such as a mobile phone, a tablet computer, and a server. As shown in FIG. 9, the image processing system 900 may include: one or more processors 901, one or more memories 902, and one or more communication components 903. Wherein, the memory 902 is used to store a program that supports the electronic device to execute the image processing method provided in the embodiment shown in FIG. 7. The processor 901 is configured to execute a program stored in the memory 902. Specifically, the program includes one or more computer instructions, and when one or more computer instructions are executed by the processor 901, the following steps can be implemented:

Run the computer program stored in the memory to achieve: according to the light intensity corresponding to the spectral band of the multi-spectral camera, perform image compensation processing on the current image acquired by the multi-spectral camera; when the processed image is an image of vegetation, according to the processed image To extract the band features of the image to determine the vegetation growth index.

The communication component 903 is configured to receive the light intensity of multiple preset wavebands of the current environment and the light intensity corresponding to at least one spectral waveband of the multispectral camera through the drone, and the light intensity corresponding to the at least one spectral waveband is based on the multiple presets The light intensity of the band is determined.

In addition, an embodiment of the present invention provides a computer-readable storage medium. The storage medium is a computer-readable storage medium. The computer-readable storage medium stores program instructions. The program instructions are used to implement the image processing method of FIG. 7 described above.

The technical solutions and technical features in each of the above embodiments can be singly or combined in case of conflict with the present invention, as long as they do not exceed the cognitive scope of those skilled in the art, they all belong to the equivalent embodiments within the protection scope of this application. .

In the several embodiments provided by the present invention, it should be understood that the related detection device (for example: IMU) and method disclosed may be implemented in other ways. For example, the embodiments of the remote control device described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components. It can be combined or integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, remote control devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. , Including several instructions to make a computer processor (processor) execute all or part of the steps of the method described in each embodiment of the invention. The aforementioned storage media include: U disk, mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes.

The above are only the embodiments of the present invention, which do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the description and drawings of the present invention, or directly or indirectly applied to other related technologies In the same way, all fields are included in the scope of patent protection of the present invention.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention. range.

Claims

An image processing method, characterized in that it comprises:

Obtain the light intensity of multiple preset bands in the current environment;

Determine the light intensity corresponding to at least one spectral waveband of the multispectral camera according to the light intensity of the multiple preset wavebands;

Perform image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera.
The method according to claim 1, wherein determining the light intensity corresponding to the multiple spectral bands of the multi-spectral camera comprises:

When the multiple preset wavebands of the current environment do not match the multiple spectral wavebands received by the multispectral camera, selecting two first preset wavebands from the multiple preset wavebands, And the first light intensity corresponding to the first spectral waveband of the multispectral camera is determined according to the two first preset wavebands.
The method according to claim 2, wherein the method further comprises:

Acquiring the second light intensity corresponding to the first spectral band through the multispectral camera, and acquiring the current image;

The step of performing image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera includes:

Compensate the second light intensity according to the first light intensity;

Perform image compensation processing on the current image according to the compensated second light intensity.
The method according to claim 2, wherein the step of performing image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera comprises:

Acquiring the second light intensity corresponding to the first spectral band from the multispectral camera;

According to the first light intensity and the second light intensity, image compensation processing is performed on the current image acquired by the multispectral camera.
The method according to claim 2, wherein the method further comprises:

Using a linear interpolation method, the first light intensity is calculated according to the two first preset wavebands.
The method according to claim 2, wherein the determining the first light intensity corresponding to the first spectral waveband of the multispectral camera according to the two first preset wavebands comprises:

The first light intensity is determined according to the ratio of the difference between the light intensity of the two first preset wavebands and the difference between the two first preset wavebands.
The method according to claim 2, wherein, among the plurality of wavebands, the two first preset wavebands are closest to the first spectral waveband.
The method according to claim 1 or 2, wherein the method further comprises:

Acquiring the first spectral band from the multispectral camera;

Wherein, determining the light intensity corresponding to the multiple spectral bands of the multi-spectral camera includes:

When the second preset waveband among the plurality of preset wavebands of the current environment matches the first spectral waveband of the multispectral camera, directly according to the second light intensity of the second preset waveband Determine the first light intensity of the first spectral band.
The method according to claim 1, wherein the light intensity of the plurality of preset wavebands is obtained from a light sensor.
The method according to claim 1, wherein the method further comprises:

When the processed current image is an image about vegetation, the vegetation growth index is determined according to the band feature extraction of the processed image.
The method according to claim 10, wherein the method further comprises:

Determine multiple vegetation growth indexes corresponding to the same vegetation through different flight equipment and/or different flight times;

In the case that the multiple vegetation growth indexes are the same, it is determined that the vegetation growth index is correct.
An image processing device, characterized in that it comprises:

A light collection device for acquiring the light intensity of multiple preset wavebands in the current environment; receiving the light intensity of multiple incident wavebands;

The image processing device further includes: a memory and a processor, and the memory is used to store a computer program;

The processor is configured to determine the light intensity corresponding to at least one spectral waveband of the multispectral camera according to the light intensity of the multiple preset wavebands; according to the light intensity corresponding to the spectral waveband of the multispectral camera, The current image acquired by the multispectral camera is subjected to image compensation processing.
The device according to claim 12, wherein the processor is specifically configured to:

When the multiple preset wavebands of the current environment do not match the multiple spectral wavebands received by the multispectral camera, selecting two first preset wavebands from the multiple preset wavebands, And the first light intensity corresponding to the first spectral waveband of the multispectral camera is determined according to the two first preset wavebands.
The device according to claim 13, wherein the image processing device further comprises: a communication component;

The communication component is configured to obtain the second light intensity corresponding to the first spectral band through the multispectral camera, and obtain the current image according to the second light intensity;

The processor is specifically configured to: compensate the second light intensity according to the first light intensity;

Perform image compensation processing on the current image according to the compensated second light intensity.
The device according to claim 13, wherein the communication component is configured to obtain the second light intensity corresponding to the first spectral band from the multispectral camera;

The processor is specifically configured to: perform image compensation processing on the current image acquired by the multispectral camera according to the first light intensity and the second light intensity.
The device according to claim 13, wherein the processor is further configured to:

Using a linear interpolation method, the first light intensity is calculated according to the two first preset wavebands.
The device according to claim 13, wherein the processor is specifically configured to:

The first light intensity is determined according to the ratio of the difference between the light intensity of the two first preset wavebands and the difference between the two first preset wavebands.
The device according to claim 2, wherein, among the plurality of wavebands, the two first preset wavebands are closest to the first spectral waveband.
The device according to claim 1 or 2, wherein the communication component is further configured to: obtain the first spectral band from the multispectral camera;

The processor is specifically configured to: when a second preset waveband of the plurality of preset wavebands of the current environment matches the first spectral waveband of the multispectral camera, directly according to the first spectral waveband The second light intensity of the two preset wavebands determines the first light intensity of the first spectral waveband.
The device according to claim 1, wherein the light intensity of the plurality of preset wavebands is obtained from a light sensor.
The device according to claim 1, wherein the processor is further configured to:

When the processed current image is an image about vegetation, the vegetation growth index is determined according to the band feature extraction of the processed image.
The device according to claim 10, wherein the processor is further configured to:

Determine multiple vegetation growth indexes corresponding to the same vegetation through different flight equipment and/or different flight times;

In the case that the multiple vegetation growth indexes are the same, it is determined that the vegetation growth index is correct.
An unmanned aerial vehicle, characterized by comprising: a body and the image processing device according to claims 12-22, the image processing device being arranged on the body.
The drone of claim 23, wherein the drone further comprises: a multispectral camera;

The multi-spectral camera is used to provide a current image to the image processing device and used to determine the light intensity of the current image; the light intensity is determined by the spectral band obtained by the multi-spectral camera.
A computer-readable storage medium, wherein the storage medium is a computer-readable storage medium, the computer-readable storage medium stores program instructions, and the program instructions are used to implement any one of claims 1-11 The image processing method described in the item.
An image processing method, characterized in that it comprises:

The UAV receives the light intensity of multiple preset wavebands of the current environment and the light intensity corresponding to at least one spectral waveband of the multispectral camera, and the light intensity corresponding to the at least one spectral waveband is based on the multiple preset wavebands. The light intensity is determined;

Performing image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera;

When the processed image is an image of vegetation, the vegetation growth index is determined according to the band feature extraction of the processed image.
An image processing system, which is characterized by comprising: a memory, a processor, and a communication component;

The memory is used to store a computer program;

The communication component is used to receive the light intensity of multiple preset wavebands of the current environment and the light intensity corresponding to at least one spectral waveband of the multispectral camera through the drone, and the light intensity corresponding to the at least one spectral waveband is based on the The light intensity of the multiple preset wavebands is determined;

The processor is configured to perform image compensation processing on the current image acquired by the multispectral camera according to the light intensity corresponding to the spectral band of the multispectral camera;

When the processed image is an image of vegetation, the vegetation growth index is determined according to the band feature extraction of the processed image.
A computer-readable storage medium, wherein the storage medium is a computer-readable storage medium in which program instructions are stored, and the program instructions are used to implement the image processing of claim 26 method.