JPWO2018180954A1 - Image processing device, growth survey image creation system and program - Google Patents

Abstract

An object of the present invention is to provide an image processing apparatus, a growth research image creation system, and a program that can correct the brightness of each pixel in an aerial image and accurately analyze the growth state of a plant. An aerial image using input means (communication I / F 24) for inputting an aerial image of a plant in a field, captured using the aerial image acquisition device 1A, and imaging information at the time of capturing the aerial image Correction value calculating means (control section 21) for calculating a correction value of the luminance value of each pixel in the correction section, and correction means (control section 21) for correcting the luminance value of each pixel in the aerial image using the correction value. The imaging information includes imaging position information and sun position information, the imaging position information includes latitude and longitude information where the imaging device exists, and the sun position information includes sun azimuth information and sun altitude information. .

Description

  The present invention relates to an image processing device, a growth survey image creation system, and a program.

2. Description of the Related Art Conventionally, there has been known a technique of acquiring an aerial image of a field photographed from the sky by remote sensing, performing image analysis processing using the aerial image, and conducting research on the growth of crops in the field.
For example, Patent Literature 1 discloses a technique of acquiring an aerial image of a field using a satellite and calculating a Normalized Difference Vegetation Index (NDVI) representing the degree of crop growth based on the acquired aerial image. I have. In recent years, two types of digital cameras for visible light imaging and near-infrared light imaging are mounted on a drone to acquire an aerial image of a field and perform similar monitoring.

  Here, since the amount of light reflection by the plant is used for calculating the normalized vegetation index, in order to accurately grasp the growth state of the crop in the field, the reflection of the light by the plant in the shooting area is required. It is necessary to obtain a large amount of data with high accuracy.

JP 2017-35055 A

By the way, when a tall plant such as paddy rice is photographed from the sky, luminance unevenness occurs for each pixel in the aerial photographed image due to the positional relationship with the imaging device.
Regarding this, the case of paddy rice monitoring will be described with reference to FIG. As shown in FIG. 5, sunlight can be approximated to parallel light, so that it can be considered that all plants (here, rice) in the field are irradiated at the same incident angle. On the other hand, the direction of the line segment (the direction of the line of sight) connecting the imaging device and the subject differs for each rice plant according to the positional relationship with the imaging device. That is, the amount of sunlight reflected by each rice body differs, such as a rice body R1 that enters at an angle at which sunlight is directed toward the line of sight of the imaging device and a rice body R3 that enters at an angle at which the sunlight is backlit. Therefore, the brightness varies in the aerial image.

  As in Patent Literature 1, since the aerial image captured by a satellite has a wide imaging range, the variation in brightness in the aerial image does not significantly affect image analysis. In addition, since the height of the imaging device is high in the first place, there is not so much difference in the amount of reflection for each rice body as described above.

  However, for example, when aerial photographing is performed from a short distance using a drone, the influence of the above-described variation in brightness increases. Due to such luminance unevenness, the accuracy in performing the calculation using the photographing data is reduced.

  The present invention has been made in view of the above problems, and has an image processing apparatus, a growth research image creation system, and a program that correct the brightness of each pixel in an aerial image and can accurately analyze the growth state of a plant. The purpose is to provide.

In order to solve the above problem, an image processing apparatus according to claim 1 is provided.
Input means for inputting an aerial image of a plant in a field, which is imaged using an imaging device,
A correction value calculation unit that calculates a correction value of a luminance value of each pixel in the aerial image using imaging information at the time of capturing the aerial image,
Correction means for correcting the brightness value of each pixel in the aerial image using the correction value,
The imaging information includes imaging position information and sun position information,
The imaging position information includes latitude and longitude information where the imaging device exists,
The solar position information includes solar azimuth information and solar altitude information.

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect,
The imaging information includes weather information.

According to a third aspect of the present invention, in the image processing apparatus according to the first or second aspect,
The correction value calculation unit sets the point having the highest luminance with respect to the imaging position information as a first reference point, and performs the correction in accordance with a distance of each pixel in the aerial image from the first reference point. It is characterized in that a value is calculated.

According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects,
It is characterized by comprising a solar position calculating means for calculating the solar position information based on the date and time at the time of shooting and the latitude and longitude information.

According to a fifth aspect of the present invention, in the image processing apparatus according to any one of the first to fourth aspects,
The imaging information includes image direction information that is direction information of each pixel with respect to a second reference point in the aerial image,
An image direction calculation unit that calculates the image direction information based on a shooting direction of the imaging device with respect to a ground surface is provided.

According to a sixth aspect of the present invention, in the image processing apparatus according to any one of the first to fifth aspects,
The imaging position information includes imaging altitude information in which the imaging device exists.

According to a seventh aspect of the present invention, in the image processing apparatus according to any one of the first to sixth aspects,
The correction value is calculated using a coefficient for multiplying or dividing a luminance value of each pixel of the aerial image.

The invention according to claim 8 is the image processing apparatus according to any one of claims 1 to 7,
The correction value is calculated using an offset value for adding or subtracting a luminance value of each pixel of the aerial image.

The growth survey image creation system according to claim 9,
An imaging device that acquires an aerial image of a plant in a field,
An image processing device according to any one of claims 1 to 8.

The program according to claim 10,
The computer of the image processing device
Input means for inputting an aerial image of a plant in a field, which is imaged using an imaging device,
A correction value calculation unit that calculates a correction value of a luminance value of each pixel in the aerial image using imaging information at the time of capturing the aerial image;
Correction means for correcting the luminance value of each pixel in the aerial image using the correction value,
Function as

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus which corrects the brightness for every pixel in an aerial image, and can analyze the growth state of a plant with high accuracy, a growth research image creation system, and a program can be provided.

It is a figure showing the system configuration of a growth research picture creation system. FIG. 2 is a diagram illustrating a functional configuration of the aerial image acquisition device of FIG. 1. FIG. 2 is a diagram illustrating a functional configuration of the image processing apparatus in FIG. 1. It is a figure explaining the imaging method of an aerial image. It is a figure showing an example of an aerial image. FIG. 3 is a diagram illustrating a cause of occurrence of luminance unevenness. It is a figure explaining image direction information. FIG. 9 is a diagram illustrating an example of a method for calculating correction data. FIG. 9 is a diagram illustrating an example of a method for calculating correction data. FIG. 9 is a diagram illustrating an example of a method for calculating correction data. FIG. 3 is a diagram illustrating luminance unevenness correction according to the sun altitude. FIG. 3 is a diagram illustrating luminance unevenness correction according to the sun altitude. FIG. 3 is a diagram illustrating luminance unevenness correction according to the sun altitude. FIG. 4 is a diagram illustrating luminance unevenness correction according to weather. FIG. 4 is a diagram illustrating luminance unevenness correction according to weather. It is a flowchart which shows a growth research image creation process. It is a flowchart which shows the detail of step S2 of FIG. It is a flowchart which shows the detail of step S22 of FIG. It is a flowchart which shows the detail of step S23 of FIG. It is a flowchart which shows the detail of step S25 of FIG. It is a figure showing the effect of the present invention.

[Embodiment]
Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings, but the present invention is not limited thereto.

<Configuration of the growth survey image creation system 100>
FIG. 1 shows an example of the overall configuration of a growth survey image creation system 100.
The growth survey image creation system 100 acquires an aerial image of a field by remote sensing, corrects the luminance value in the acquired aerial image to an appropriate value, and allows a quantitative evaluation of the growth state of the crop to be performed. This is a system that generates an image.

  The growth survey will be described here by taking paddy rice monitoring as an example, but is not limited to this. For example, the present invention can be applied to monitoring of upland rice, wheat, and the like.

As shown in FIG. 1, the growth survey image creation system 100 includes an aerial image acquisition device 1A and an image processing device 2A connected so as to be able to transmit and receive data.
The aerial image acquisition device 1A and the image processing device 2A can be paired using wireless communication that can be used for each. The wireless communication includes wireless LAN (Wi-Fi) or Bluetooth (registered trademark). And so on.

The aerial image acquisition device 1A implements a function as an imaging device. The aerial image acquisition device 1A acquires a plurality of aerial images captured by remote sensing so as to cover the entire field while flying over the field and transmits the image to the image processing apparatus 2A.
The remote sensing refers to a technology that includes a platform including an unmanned aerial vehicle such as a drone or a manned aircraft such as an airplane, a helicopter, and a balloon, and an observation device such as a digital camera, and observes the ground from above. Here, description will be made assuming that a drone is used as a platform.

FIG. 2 illustrates a functional configuration example of the aerial image acquisition device 1A.
As illustrated in FIG. 2, the aerial image acquisition device 1A includes a control unit 11, an imaging unit 12, a flying unit 13, a positioning unit 14, a communication I / F 15, a storage unit 16, and the like. Connected through.

The control unit 11 includes a CPU (Central Processing Unit) and a RAM (Random Access Memory).
), And executes various processes in cooperation with various programs stored in the storage unit 16 to totally control the operation of the aerial image acquisition apparatus 1A.
For example, the control unit 11 controls the imaging unit 12 in response to a shooting instruction from the transmitter, which is received via the communication I / F 15, and causes the aerial image to be shot. The control unit 11 controls the flying unit 13 in accordance with the flight instruction from the transmitter, which is received via the communication I / F 15, and causes the aerial image acquisition device 1A to fly.

  The imaging unit 12 is a digital camera including a CCD (Charge Coupled Device) sensor and the like, and acquires an aerial image according to an instruction from the control unit 11. The imaging unit 12 receives the reflected light of sunlight reflected by the rice body via the lens, captures an image formed on the imaging surface by the lens, and generates digital image data of an aerial image. Note that the imaging unit 12 captures an image of a point directly below the aerial image acquisition device 1A.

The flight unit 13 includes a rotor, a motor, a battery, an ESC (Electronic Speed Control), and the like, and causes the aerial image acquisition device 1A to fly according to an instruction from the control unit 11.
A plurality of rotors are provided on the main body of the aerial image acquisition device 1A, and each rotor is rotationally driven by a motor to apply lift to the aerial image acquisition device 1A. The motor rotates by the voltage provided by the battery, and the rotation speed is changed by voltage control by the ESC. The rotation speed of the rotor changes in accordance with the output of the motor, and the flight direction of the aerial image acquisition device 1A is changed by the rotation speed difference between the rotors.

  The positioning unit 14 includes a three-dimensional positioning device such as a GPS, and acquires information on the latitude, longitude, and altitude of the aerial image acquisition device 1A. In addition, the positioning unit 14 includes a gyro sensor and detects the inclination and rotation of the aerial image acquisition device 1A, and the control unit 11 controls the aerial attitude of the aerial image acquisition device 1A based on the inclination and rotation, and will be described later. Thus, the photographing direction of the aerial image acquisition device 1A with respect to the ground surface is calculated together with the three-dimensional positioning information by GPS.

The communication I / F 15 is an interface for transmitting and receiving data to and from an external device.
The communication I / F 15 is communicably connected to a transmitter such as a remote controller that can control the flight of the aerial image acquisition device 1A and the aerial image capturing, and receives a flight instruction and an image received via the remote controller. The instruction is transmitted to the control unit 11.
In addition, it is connected to the image processing apparatus 2A so as to be able to transmit and receive data, and transmits a digitalized aerial image to the image processing apparatus 2A under the control of the control unit 11.

The storage unit 16 is configured by, for example, a hard disk drive (HDD) or a nonvolatile semiconductor memory. The storage unit 16 stores various programs and various data as described above.
In addition, the aerial image acquisition device 1A may include a LAN adapter and a router, and may be configured to be connected to an external device via a communication network such as a LAN.

The image processing device 2A corrects luminance unevenness in the aerial image transmitted from the aerial image acquisition device 1A, and generates an image capable of quantitatively analyzing the growing situation of the crop in the field.
FIG. 2 shows a functional configuration example of the image processing apparatus 2A.
As shown in FIG. 2, the image processing apparatus 2A includes a control unit 21, an operation unit 22, a display unit 23, a communication I / F 24, a storage unit 25, and the like. I have.

The control unit 21 includes a CPU (Central Processing Unit) and a RAM (Random Access Memory).
), And executes various processes in cooperation with various programs stored in the storage unit 25 to control the operation of the image processing apparatus 2A as a whole.
For example, the control unit 21 performs an image analysis process in cooperation with an image processing program stored in the storage unit 25, and functions as a correction value calculation unit, a correction unit, a sun position calculation unit, and an image direction calculation unit. To achieve.

  The operation unit 22 includes a keyboard provided with character input keys, numeric input keys, various function keys, and the like, and a pointing device such as a mouse. Is output to the control unit 21 as an input signal.

  The display unit 23 includes a monitor such as a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal Display). The display unit 23 displays various screens according to an instruction of a display signal input from the control unit 21. Implement the function as a means.

  The communication I / F 24 is an interface for transmitting and receiving data to and from external devices such as the aerial image acquisition device 1A. The communication I / F 24 implements a function as an aerial image input unit.

The storage unit 25 is configured by, for example, a hard disk drive (HDD) or a nonvolatile semiconductor memory. The storage unit 25 stores various programs and various data as described above.
In addition, the image processing apparatus 2A may include a LAN adapter and a router, and may be configured to be connected to an external device via a communication network such as a LAN.

<Luminance unevenness in aerial image>
Hereinafter, the brightness unevenness occurring in the aerial image in the growth survey image creating system 100 will be described.

FIG. 4A shows an example of an aerial image capturing method.
In the aerial image I of the whole field, a plurality of aerial images i are continuously laminated and synthesized. The aerial image i is one image captured by the aerial image acquisition device 1A at a time. In the case of FIG. 4A, the aerial image acquisition device 1A continuously captures the aerial image i while flying in the direction of X in the figure from the lower right end to the upper right end of the figure. Subsequently, the aerial image acquisition device 1A moves in the Y direction in the figure, and continuously captures the aerial image i while flying from the upper end to the lower end in the X direction in the figure. By repeating the above and photographing up to the left end of the figure, photographing of the entire field is completed.

Here, FIG. 4B shows an example of an aerial image is taken in fine weather. As shown in the figure, the aerial image is is an aerial image due to the difference in the amount of light reflected by rice plants, assuming that the sun exists in the sky above the left side of the figure and sunlight S is irradiated from the direction of the arrow. A so-called luminance unevenness occurs in which the luminance varies for each pixel in the is.
A possible cause of this phenomenon will be described with reference to FIG.

When capturing a three-dimensional object from above, the imaging unit 12 of the aerial image acquisition device 1A has a different line segment direction (hereinafter, referred to as a line-of-sight direction) connecting the imaging unit and the subject for each pixel in the aerial image. . On the other hand, since sunlight can be approximated to parallel light, it enters all pixels in the aerial image from the same direction.
Therefore, as shown in FIG. 5, with respect to the right end of the photographing area, the incident direction of the sunlight S is the same as the line-of-sight direction D1 of the imaging unit 12, that is, the angle at which the light is directed toward the rice body R1 as the subject. Becomes When the subject is a three-dimensional object having a height such as a rice body, when the sunlight S is irradiated at an angle of normal light, the sunlight S is reflected on the left side surface of the rice body R1. The amount of received light increases.
On the other hand, since the line-of-sight direction D2 of the imaging unit 12 is directly below the center of the imaging region, the incident angle of sunlight is different from the line-of-sight direction D2 of the imaging unit 12. Since the rice body R2 is located directly below the aerial image acquisition device 1A, the imaging unit 12 receives the sunlight S reflected on the upper surface of the rice body R2, but the amount of received light is smaller than that of the rice body R1. I do.
Further, the incident direction of the sunlight S is different from the line of sight D3 of the imaging unit 12 and is incident on the left end of the photographing area at an angle which is a backlight that is incident from behind the rice body R3 as the subject. Therefore, the amount of light received by the imaging unit 12 is the smallest in the imaging area.
As described above, since the direction of the line of sight to the subject is different in the imaging region, it is considered that the distribution of brightness changes in the aerial image, that is, luminance unevenness occurs. The brightness of each pixel in the aerial image is brighter nearer to the brightest point (rice R1 in FIG. 5) and darker farther away.

  Here, the luminance value of each pixel in the aerial image is corrected using imaging information, that is, (1) imaging position information, (2) sun position information, (3) image direction information, and (4) weather information. be able to.

(1) Imaging position information The luminance value of each pixel in the aerial image is affected by the imaging position information. The imaging position information includes latitude / longitude information of the position where the aerial image acquisition device 1A is present and imaging height information.
The latitude / longitude information is used for calculating the sun position information, and the imaging altitude information is used for calculating the correction data, as described later.

(2) Sun position information The luminance value of each pixel in the aerial image is affected by the sun position information. The solar position information includes solar azimuth information and solar altitude information.
When the sun azimuth is different, the direction in which the brightest point exists with respect to the aerial image acquisition device 1A is different. Therefore, the distribution of brightness in the image fluctuates depending on the sun azimuth.
Further, as will be described in detail later, as the sun altitude is lower, the brightest point when viewed from the aerial image acquisition device 1A is farther from a point directly below the aerial image acquisition device 1A. Since the luminance of each pixel in the aerial image depends on the distance from the brightest point, the luminance value in the aerial image varies depending on the sun altitude.
The solar position information can be specified by month, day, time, and the latitude and longitude information described above.

(3) Image direction information The image direction information is information indicating in which direction each pixel in the aerial image is located when viewed from the image center.
This will be described with reference to FIG. It is assumed that the sun is located southwest of the image center O. Therefore, the point having the highest luminance (first reference point; hereinafter, referred to as bright reference point P) exists in the northeast direction with respect to the image center O.

Consider two aerial images, aerial image i1 and aerial image i2, taken by the same aerial image acquisition device 1A. The aerial image i2 is a direction in which the aerial image acquisition device 1A is rotated by 90 degrees with respect to the ground surface from the time of capturing the aerial image i1 (a direction in which the camera of the imaging unit 12 is rotated by 90 degrees with respect to the ground surface). Shooting with
As described above, since the brightness value of each pixel in the aerial image is brighter as it is closer to the bright reference point P, in the case of FIG. 6, the brightness value of the pixel on the northeast side in the aerial image is higher. Accordingly, the pixels in the aerial image i1 are brightest near the vertex 4, while the pixels in the aerial image i2 are brightest near the vertex 3. Therefore, since the luminance value of each pixel in the aerial image is determined by the relative positional relationship from the bright reference point P, even if the pixels are on the same coordinates, the orientation of the aerial image acquisition device 1A with respect to the ground surface ( It can be seen that the luminance value differs depending on the (photographing direction), and the brightness distribution (luminance unevenness) of the entire aerial image also fluctuates accordingly.

Therefore, in order to calculate the value of the luminance unevenness in the aerial image, in addition to the sun azimuth information, each pixel in the aerial image is positioned at a specific coordinate (second reference point; here, image center O). On the other hand, it is necessary to obtain image direction information indicating which direction the image is located.
For example, in the example of FIG. 6, the target pixel n on the aerial image i1 is located in the east direction with respect to the image center O, and thus the image direction information of the target pixel n is “east”. On the other hand, since the target pixel n on the aerial image i2 is located in the north direction with respect to the image center O, the image direction information of the target pixel n is “north”.
As described above, the luminance unevenness in the aerial image can be determined by adding the image direction information held by each pixel and the above-described sun azimuth information. Note that the image direction information of each pixel can be uniquely obtained if the shooting direction of the aerial image acquisition device 1A is clear.

(4) Weather Information The luminance value of each pixel in the aerial image is affected by the weather information. The weather information is information indicating whether the weather is clear or cloudy.
As will be described in detail later, the amount of sunlight reaching the ground decreases in cloudy weather (a situation where clouds are uniformly distributed over the entire sky) compared to when the weather is fine. FIG. 4B shows an example of an aerial photograph image ic photographed in cloudy weather. Even if the aerial image ic is an image of the same field as the aerial image is taken in fine weather in FIG. 4B, the brightness value of each pixel in the aerial image in cloudy weather is better than in fine weather. Become smaller. That is, the absolute value of the luminance in the entire aerial image varies depending on the weather.

  As described above, in the present embodiment, the luminance unevenness and the absolute value of the luminance in the aerial image are corrected in consideration of the four elements (1) to (4).

<Method of correcting uneven brightness in aerial image>
Hereinafter, a method for correcting the uneven brightness in the aerial image described above in the growth survey image creating system 100 will be described.

  As described above, when the incident angle of sunlight and the line of sight of the imaging unit 12 match, the brightness becomes highest. Therefore, in the growth survey image creating system 100 according to the present embodiment, the luminance value of each pixel in the aerial image is corrected based on the distance from the bright reference point P.

FIG. 7 shows a method of calculating a correction value of a luminance value.
As described below, the method of calculating the correction value includes (1) calculating the coordinates of the bright reference point, (2) calculating the coordinates of each pixel, (3) deriving the correction data, and (4) calculating the correction value. , And 4 steps are included.

(1) Calculation of Coordinates of Bright Reference Point FIGS. 7A and 7B are explanatory diagrams of a method of calculating the coordinates of the bright reference point P.
First, the distance D [pix] of the bright reference point P from the image center O is calculated. As illustrated in FIG. 7A, it is _assumed that the incident angle of the sunlight S is θ _sun , the angle of view of the imaging unit 12 is θ _fov , and the number of horizontal pixels is w [pix]. Note that the number of horizontal pixels is a value that fluctuates according to the imaging height information, as is apparent from the figure.
The bright reference point P is defined as one point on the ground surface where the line of sight of the imaging unit 12 is at the same angle as the incident angle θ _{sun of the} sunlight S. Therefore, the distance D from the image center O of the bright reference point P (immediately below the aerial image acquisition device 1A) is obtained by the following equation (1).
D = (tan θ _sun / tan θ _{fov / 2} ) * w / 2 (1)

Subsequently, the coordinates of the light reference point P are calculated. FIG. 7B is a diagram of the ground surface viewed from above. As shown in the figure, the X-axis direction and the Y-axis direction are determined with the image center O as the origin, and the coordinates of the bright reference point P are set to (xp, yp).
The distances Dx [pix] and Dy [pix] of the bright reference point P from the X-axis direction and the Y-axis direction are respectively obtained by the following equation (2).
Dx = Dcos θ _p (2-1)
Dy = Dsin θ _p (2-2)
When the distances Dx and Dy are converted into coordinates, the coordinates (xp, yp) of the bright reference point P are obtained.

(2) Calculation of the distance of each pixel from the bright reference point The luminance value of each pixel varies according to the distance from the bright reference point P. Therefore, in order to calculate the correction value of the luminance value, it is necessary to find the distance of each pixel from the bright reference point P.
FIG. 7C is an explanatory diagram of a method of calculating the distance of each pixel from the bright reference point P. Similarly to FIG. 7B, the X-axis direction and the Y-axis direction are determined with the image center O as the origin, and the coordinates of the target pixel C are set to (xc, yc).
The distance dc [pix] of the target pixel C from the bright reference point P is obtained by the following equation (3).
dc = {(xc−xp) ² + (yc−yp) ² } (3)

(3) Derivation of Correction Data Using the distance d of each pixel described above from the light reference point P, the correction data v (x, y) of the luminance value of each pixel is obtained by the following equation (4). .
v (x, y) = Kexp (−αd) * W + β (4)
That is, the above equation (4) indicates that the correction data v (x, y) depends on the distance d of each pixel from the bright reference point P.
Here, W is a weather parameter (coefficient) for multiplying or dividing the correction data v (x, y) according to the weather, and β is added or corrected to the correction data v (x, y) according to the solar altitude information. This is an offset value for performing the subtraction.
Here, α is a parameter corresponding to the solar altitude, and K is a constant calculated from the actual data.

Here, the correction according to the solar altitude information will be described.
As shown in FIG. 8A, it is assumed that, when sunlight Sa is irradiated, the coordinates of the light reference point P1 overlap the position of the rice body R1. When the sun direction is the same and the sun altitude is lower than this and the sunlight S2 is radiated, the line of sight of the imaging unit 12 parallel to the incident angle of the sunlight Sb shifts to the right side in the figure and becomes bright. The reference point P2 overlaps the position of the rice body R4. That is, the lower the sun altitude, the farther the bright reference point is from immediately below the aerial image acquisition device 1A.

FIG. 8B is a diagram of the ground surface viewed from above. As shown in the figure, the X-axis direction and the Y-axis direction are determined with the image center O as the origin, and the coordinates of the bright reference point P1 are set to (xp, yp). The distances of the pixels of interest a and b in the aerial image i from the image center O are da and db, respectively.
In the graph of FIG. 8C, the horizontal axis represents the distance D from the image center O, and the vertical axis represents the luminance value L. Since the distances from the bright reference point of the target pixels a and b are different between the case where the sunlight Sa is irradiated and the case where the sunlight Sb is irradiated, the luminance values are also different. Therefore, by subtracting a predetermined value (β) from the luminance value L when the sunlight Sa is irradiated, the luminance value L when the sunlight Sb is irradiated can be obtained.
The value corresponding to the subtraction performed at this time is the above-described offset value β.

Next, the correction according to the weather will be described.
FIG. 9 shows a comparison of brightness values between a sunny day and a cloudy day (a situation where clouds are uniformly distributed over the sky) at the same sun position.
FIG. 9A is a diagram of the ground surface viewed from above. As shown in the figure, the X-axis direction and the Y-axis direction are determined with the image center O as the origin, and the coordinates of the bright reference point P are set to (xp, yp). The distances of the pixels of interest a and b in the aerial image i from the image center O are da and db, respectively.
In the graph of FIG. 9B, the horizontal axis indicates the distance D from the image center O, and the vertical axis indicates the luminance value L. Even if the azimuth and altitude of the sun are the same, the brightness differs between the same pixels because the brightness value L is higher in clear weather than in cloudy weather. Therefore, by dividing the brightness value L in fine weather by the predetermined value (W), the brightness value L in cloudy weather can be obtained.
The value corresponding to the division performed at this time is the coefficient W described above.

(4) Calculation of Correction Value The calculation is performed using the correction data v (x, y) obtained as described above, and the correction value of the luminance value L is calculated for each pixel.
As described above, if the coordinates of the image center O are (0, 0) and the coordinates of the pixel of interest C are (xc, yc) in FIG. 7C, the correction equations for the image center O and the pixel of interest C are v ( 0,0) and v (xc, yc).
Based on this, the correction value Lnew of the luminance value L at the target pixel C is calculated. The brightness value Lnew is obtained by the following equation (5).
Lnew = L * v (xc, yc) / v (0, 0) (5)
That is, a difference between the correction data of the image center O and the correction data of the target pixel C is reflected as a coefficient on the luminance value L of the target pixel C, thereby obtaining a correction value Lnew.
By performing the above calculation for all the pixels in the aerial image, an aerial image in which luminance unevenness is eliminated can be obtained.

<Operation of the growth survey image creation system 100>
Hereinafter, the operation of the growth survey image creation system 100 for correcting the uneven brightness generated in the aerial image described above and generating the growth survey image showing the entire field will be described.
FIG. 10 shows a flowchart of a growth survey image creation process in the image processing device 2A. The growth survey image creation processing shown in FIG. 10 is executed in cooperation with the control unit 21 and a program stored in the storage unit 25.

First, when a plurality of aerial images captured so as to cover the entire field are input from the aerial image acquisition device 1A via the communication I / F 24 (step S1), the aerial image correction processing is performed. (Step S2). The aerial image correction processing is performed on each of the plurality of aerial images input to the image processing apparatus 2A.
FIG. 11 shows a detailed flow of the process in step S2. The process in step S2 is executed in cooperation with the control unit 21 and a program stored in the storage unit 25.

  The aerial image correction processing includes the steps of acquiring imaging position information (step S21), acquiring solar position information (step S22), acquiring image direction information (step S23), and acquiring weather information (step S24). Then, a correction value is calculated based on the information (step S25).

In step S21, the control unit 21 acquires, as the imaging position information, the latitude and longitude information and the imaging height information of the point where the aerial image acquisition device 1A exists.
In detail, the control unit 21 acquires the three-dimensional positioning information measured by the positioning unit 14 at the time of capturing the aerial image from the aerial image acquisition device 1A via the communication I / F 24. Since the aerial image acquisition device 1A captures an area immediately below, the latitude and longitude obtained here are regarded as the latitude and longitude of the center of the image of the aerial image (the point immediately below the aerial image acquisition device 1A). be able to. The imaging height information is a height from a point directly below the aerial image acquisition device 1A.

In step S22, sun position information is obtained.
FIG. 12 shows a detailed flow of the process in step S22. The process in step S22 is executed in cooperation with the control unit 21 and a program stored in the storage unit 25.
In step S22, first, the control unit 21 acquires date and time information (step S221). The date and time information may refer to the system time managed by the image processing apparatus 2A, or may allow the user to input the date and time via the operation unit 22.

Next, the control unit 21 calculates a solar altitude and a solar azimuth based on the information acquired in step S221 and the latitude and longitude information (step S222). The obtained solar altitude and solar azimuth are altitudes as viewed from the center of the aerial image.
As described above, solar position information (solar altitude information and solar azimuth information) is obtained.

In step S23, image direction information is obtained.
FIG. 13 shows a detailed flow of the process in step S23. The process of step S23 is executed in cooperation with the control unit 21 and a program stored in the storage unit 25.
In step S23, first, the control unit 21 acquires shooting direction information (step S231). The shooting direction information is the direction (shooting direction) of the aerial image acquisition device 1A with respect to the ground surface at the time of shooting the aerial image. The control unit 21 obtains three-dimensional positioning information by GPS, tilt information by a gyro sensor, and the like measured by the positioning unit 14 by the communication I / F 24, and obtains shooting direction information based on the information.

Next, the control unit 21 calculates the direction of each pixel (step S232). As described above, when the shooting direction information is specified, the direction of each pixel with respect to the center of the image is uniquely determined by adding the shooting direction information to the sun azimuth information acquired in step S22.
As described above, image direction information is obtained.

  In step S24, weather information is obtained. The weather information is obtained by allowing the user to input whether the weather is clear or cloudy via the operation unit 22.

In step S25, a correction value is calculated.
FIG. 14 shows a detailed flow of the process in step S25. The process in step S25 is executed in cooperation with the control unit 21 and a program stored in the storage unit 25.
In step S25, the control unit 21 calculates the coordinates of the bright reference point (step S251), and then calculates the distance of each pixel from the bright reference point (step S252). Here, the coordinates may be calculated based on the center of the image as described above, or may be calculated based on an arbitrary point in the aerial image. In the present embodiment, the image center is used as a reference.

Subsequently, the control unit 21 derives correction data (step S253). Here, the control unit 21 calculates an offset value based on the sun position information obtained in step S22, calculates a weather parameter W based on the weather information obtained in step S24, and based on these. Deriving correction data.
Next, the control unit 21 calculates a correction value of each pixel based on the correction data (Step S254).
The correction value is calculated as described above.

When the correction processing of the aerial image has been completed for all the aerial images as described above, the process proceeds to step S3 in FIG. 10, and the control unit 21 combines the all aerial images. Here, an operation of bonding the images to form a continuous image is performed based on the three-dimensional positioning information of the aerial image acquisition device 1A at the time of capturing each aerial image.
As described above, the growth survey image creation processing is completed, and the growth survey image of the entire field in which the variation in luminance has been eliminated is completed.

<Effects of the growth survey image creation system 100>
The effect of the present invention will be described with reference to FIG.
In the graph of FIG. 15, the horizontal axis represents the coordinates based on the image center, and the vertical axis represents the pixel value, and indicates the pixel value in a linear image area connecting the bright reference point and the image center.
As shown in the graph, the correction data is larger at coordinates farther from the image center and closer to the bright reference point.
As a result, it can be seen that the luminance of the pixels near the bright reference point was reduced by the correction, and the entire aerial image had uniform luminance.

[Other embodiments]
As mentioned above, although it explained concretely based on embodiment concerning the present invention, the above-mentioned embodiment is a suitable example of the present invention, and is not limited to this.

  For example, in the above embodiment, the weather parameter is a coefficient for multiplying or dividing the correction data. However, the present invention is not limited to this, and may be an offset value for performing addition or subtraction. Similarly, the parameter relating to the solar altitude is an offset value for performing addition or subtraction on the correction data, but may be a coefficient for performing multiplication or division. In addition, the formula used in the above embodiment may be appropriately changed so as to be suitable for calculation of correction data.

  In addition, the correction data is calculated using an equation having an offset value related to the weather parameter and the solar altitude, but is not limited thereto. It may be calculated by using this.

  Further, in the above-described embodiment, the description has been given of the case where a single digital camera is mounted as the imaging unit. However, by mounting two types of digital cameras for visible light shooting and near infrared light shooting, NDVI can be calculated and used for growth studies.

  In the above-described embodiment, the altitude of the aerial image acquisition device 1A has been described as being used for calculating correction data on the assumption that the altitude changes during imaging. When the altitude of 1A is fixed, the imaging altitude information does not affect the calculation of the correction data. That is, since the value of w in the above equation (1) is common to all aerial images, there is no need to calculate the above equation (1) for each aerial image, and the processing can be simplified. .

  Further, in the above-described embodiment, the example in which the image direction information is acquired at the time of calculating the correction value has been described on the assumption that the photographing direction by the aerial image acquisition device 1A is different for each aerial image. When the direction is fixed, it is not necessary to obtain the image direction information. That is, in this case, since the relative positional relationship of each pixel with respect to the bright reference point is common to all aerial images, the processing can be simplified.

  In addition, the detailed configuration and detailed operation of each device constituting the growth survey image creating system 100 can be appropriately changed without departing from the spirit of the invention.

  INDUSTRIAL APPLICATION This invention can be utilized for an image processing apparatus, a growth research image creation system, and a program.

1A Aerial image acquisition device (imaging device)
11 control unit 12 imaging unit 13 flight unit 14 positioning unit 15 communication I / F
16 storage unit 17 bus 2A image processing device 21 control unit (correction value calculation unit, correction unit, sun position calculation unit, image direction calculation unit)
22 operation part 23 display part 24 communication I / F (input means)
25 storage unit 26 bus 100 growth survey image creation system P light reference point (first reference point)
O Image center (second reference point)

Claims (10)

Input means for inputting an aerial image of a plant in a field, which is imaged using an imaging device,
A correction value calculation unit that calculates a correction value of a luminance value of each pixel in the aerial image using imaging information at the time of capturing the aerial image,
Correction means for correcting the brightness value of each pixel in the aerial image using the correction value,
The imaging information includes imaging position information and sun position information,
The imaging position information includes latitude and longitude information where the imaging device exists,
An image processing device, wherein the sun position information includes solar azimuth information and solar altitude information.   The image processing device according to claim 1, wherein the imaging information includes weather information.   The correction value calculation unit sets the point having the highest luminance with respect to the imaging position information as a first reference point, and performs the correction in accordance with a distance of each pixel in the aerial image from the first reference point. The image processing apparatus according to claim 1, wherein the value is calculated.   4. The image processing apparatus according to claim 1, further comprising: a solar position calculating unit configured to calculate the solar position information based on a date, a time, and the latitude and longitude information at the time of shooting. 5. The imaging information includes image direction information that is direction information of each pixel with respect to a second reference point in the aerial image,
The image processing apparatus according to any one of claims 1 to 4, further comprising an image direction calculation unit configured to calculate the image direction information based on a shooting direction of the imaging device with respect to a ground surface.   The image processing device according to claim 1, wherein the imaging position information includes imaging altitude information where the imaging device exists.   The image processing device according to claim 1, wherein the correction value is calculated using a coefficient for multiplying or dividing a luminance value of each pixel of the aerial image.   The image processing device according to claim 1, wherein the correction value is calculated using an offset value for adding or subtracting a luminance value of each pixel of the aerial image. An imaging device that acquires an aerial image of a plant in a field,
A growth survey image creation system, comprising: the image processing device according to claim 1. The computer of the image processing device
Input means for inputting an aerial image of a plant in a field, which is imaged using an imaging device,
A correction value calculation unit that calculates a correction value of a luminance value of each pixel in the aerial image using imaging information at the time of capturing the aerial image;
Correction means for correcting the luminance value of each pixel in the aerial image using the correction value,
Program to function as

Images