KR102164522B1 - Method for estimation of depth of water using hyperspectral image - Google Patents

Abstract

The present invention relates to a method of estimating the depth of water using a hyperspectral image. Specifically, after acquiring a hyperspectral image using an unmanned aerial vehicle, by setting a regression equation using a band ratio, high accuracy with only the hyperspectral image It is about a method to estimate the depth of the water.

Description

Method for estimation of depth of water using hyperspectral image}

The present invention relates to a method of estimating the depth of water using a hyperspectral image. Specifically, after acquiring a hyperspectral image using an unmanned aerial vehicle, by setting a regression equation using a band ratio, high accuracy with only the hyperspectral image It is about a method to estimate the depth of the water.

3D modeling of information outside the water is relatively easy using various imaging equipment, but 3D modeling that reflects information in the water, especially a wide range of water depths, is a very difficult task.

The most common way to measure depth is to use ultrasound. However, this method has low precision due to various external factors such as reflection due to obstacles and is not possible at low depths. In low water depths, a human dive or measuring method using a tape measure is mainly used. Even in this case, it is practically impossible to measure a wide range.

As an alternative to this, in order to measure the depth of water, a method of securing a hyperspectral image of the water body and using it has been discussed. This is because when a hyperspectral sensor is used, it is possible to secure a lot of information up to several hundred times compared to photographing with a general image sensor, including the visible light region, as well as the ultraviolet region and the infrared region.

In order to check the depth of water using hyperspectral information, a database is needed that matches what the actual bands by wavelength mean. For example, if you know the value of 8 bands (412, 490, 512, 555, 660, 680, 765, 875nm) by the Goestationary Ocean Color Imager (GOGI) launched in 2010 The database on whether or not each value represents a sea color (海色) is already secured. Therefore, when the hyperspectral information for each point is confirmed by photographing the superspectral image of the ocean, the sea color is estimated without direct observation by only the hyperspectral image by checking the value at the wavelength and the sea color matched thereto from the database. The same method can be applied to depth.

In addition, since observation values for each wavelength at all points cannot be confirmed and databaseized, a regression equation is used. That is, if a regression equation is constructed using a plurality of data, the actual value can be estimated by using the band for each wavelength identified in the hyperspectral image by using the regression equation even at a location that is not in a database.

By combining these methods, it is theoretically possible to estimate the depth of water using only hyperspectral images. When a value for a specific wavelength is extracted from the hyperspectral image, the depth can be estimated by using a database that is previously matched with the water depth and stored, or by substituting it into a regression equation.

However, it is known that there is a certain level of error in the method of estimating the depth of water using a photographic value of a specific wavelength in this manner. As a cause of such errors, errors occurring in the hyperspectral information photographing step (for example, a situation in which the photographed value does not represent the actual position value due to the influence of reflected light), and errors occurring in the image correction step (for example, Errors in geometric correction, errors in image registration, etc.), errors in the database (for example, when the photographed value is correct but the identification information is incorrect), and other external factors are presented.

Moreover, when the water depth is estimated through a regression equation constructed using a value with low accuracy, this error gradually increases.

In addition, it is difficult to secure a hyperspectral image of a wide area.

In order to secure a hyperspectral image of a wide area, it is common to shoot with a manned aircraft or an artificial satellite. In this case, the cell size becomes excessively large, such as 10m to 50m, making precise measurement impossible. For example, when measuring the depth of water, it is measured in units of 100m ^{2 on the} surface, so the precision decreases.

In order to solve this problem, attempts have been made to mount hyperspectral sensors on unmanned aerial vehicles such as drones that can fly at close range. In this case, the cell size is reduced, so that the accuracy is increased, but there is another problem that the accuracy is lowered due to problems such as shaking during flight that the unmanned aerial vehicle has. To prevent this, it is possible to consider installing a high-performance gimbal that can compensate for shake, but the weight of the high-performance gimbal is quite large, which weakens the drone's flight performance.

Moreover, since the hyperspectral sensor basically performs line scanning to secure hyperspectral information, software for matching a plurality of hyperspectral information is essential. However, the existing hyperspectral information matching software lacks its accuracy, and thus, an image obtained by matching a plurality of hyperspectral information may provide information on a point slightly deviated from the actual photographing point.

Accordingly, there is a trend of increasing demand for a new method of identifying depth that can reduce errors and improve accuracy.

Review related patent documents.

Korean Patent Registration No. 10-1621354 discloses a depth measurement method using hyperspectral images. Since all of the above-described problems are included, the accuracy is low.

Korean Patent Registration No. 10-1463351 discloses a technique for increasing the speed of information calculation by detecting a specific region in a hyperspectral image by using a band ratio and excluding the detected region in a hyperspectral image. The present invention described later and the use of a band ratio are common only, and the objects, configurations, and effects of the overall invention are different.

Korean Patent Registration No. 10-1619836 discloses a drone using a gimbal and a shock absorber for a camera for shooting hyperspectral images. It is advantageous in that an image that compensates for the shaking of a drone is secured, but the image matching technology by line scanning cannot be disclosed.

Korean Patent Registration No. 10-1806488 discloses a technology for identifying a dark body by mounting a hyperspectral image sensor on a drone. Korean Patent Registration No. 10-1744662 discloses a technology for classifying plants using hyperspectral images. To this end, standard data for hyperspectral images for various dark bodies or plants must be secured in advance, and the object to be photographed is identified by comparing the photographed images with them. However, it does not solve the fundamental problems found in the shooting stage, such as the problem of lowering accuracy due to the shaking of the drone and increasing the efficiency of image matching.

Korean Patent Registration No. 10-1689197 discloses a gimbal assembly that can be attached to a drone. The assembly disclosed herein is composed of a first unit capable of rotation/reverse rotation, a second unit capable of elevating and descending, and a third unit that rotates them, thereby compensating for shaking of the aircraft. However, when configured in this way, the weight of the gimbal will increase excessively, resulting in a problem that the flying performance of the drone is degraded.

Korean Patent Publication No. 10-2014-0014819 discloses a method of identifying data in the form of an artificial intelligence algorithm by applying an index/query technique to data identified in a hyperspectral image. The prerequisite for using this method is the accuracy of hyperspectral images, but it cannot even suggest a method to increase the accuracy of images captured in an environment such as an unmanned aerial vehicle.

US Patent No. 7,181,055 proposes an algorithm for matching hyperspectral images. Here, the reflectivity image is utilized to increase the matching accuracy.

(Patent Document 1) Korean Patent Registration No. 10-1463351

(Patent Document 2) Korean Patent Registration No. 10-1463351

(Patent Document 3) Korean Patent Registration No. 10-1619836

(Patent Document 4) Korean Patent Registration No. 10-1806488

(Patent Document 5) Korean Patent Registration No. 10-1744662

(Patent Document 6) Korean Patent Registration No. 10-1689197

(Patent Document 7) Korean Laid-Open Patent No. 10-2014-0014819

(Patent Document 8) US Patent No. 7,181,055

The present invention was devised to solve the above problems. In particular, in the case of estimating the depth of water using hyperspectral images, various problems that may occur are to be solved.

Specifically, it is intended to solve the problem that the accuracy of image matching is lowered due to the characteristic of the line-scanned hyperspectral sensor, and to solve the problem of image shaking caused by being photographed in an unmanned aerial vehicle.

In addition, to obtain accurate hyperspectral information and to propose a novel hyperspectral information identification method to prevent the situation in which the database for the corresponding captured value does not reflect accurate identification information even though the captured value of a specific wavelength is accurate. do.

An embodiment of the present invention for solving the above problems is, (a) while flying along the set flight path, using the hyperspectral sensor 140, line scanning (line scanning) Acquiring hyperspectral information for each preset image unit; (b) performing, by the geometric correction module 250, performing geometric correction on the hyperspectral information obtained for each image unit by using position information of each pixel; (c) obtaining, by the image matching module 260, a hyperspectral image of the collection area by matching the hyperspectral information obtained for each image unit on which the geometric correction has been performed; (d) inputting a depth observation value (Y) of each of a plurality of predetermined points into the hyperspectral observation value input module 310; (e) The band ratio calculation module 320, at each of the plurality of predetermined points, from the hyperspectral image obtained in step (c), the hyperspectral photographing value (I), which is a pixel value for a plurality of wavelengths (x). (x)), select an arbitrary pair of two wavelengths (λ ₁ , λ ₂ ) among a number of wavelengths (x) included in the hyperspectral photographing value, and select the hyperspectral photographing value (I( determining λ ₁ ), I(λ ₂ )); (f) The band ratio calculation module 320 divides any one of the identified two hyperspectral photographing values (I(λ ₁ ), I(λ ₂ )) into the other and divides the band ratio (X Calculating ); And (g) the regression analysis module 330 assumes that the depth observation value (Y) and the calculated band ratio (X) are in a linear relationship, and the depth observation value (Y) at the plurality of points. And generating a regression equation (Y=aX+b) by calculating a and b of a regression equation (Y=aX+b) using the calculated band ratio (X), and the generated A depth estimation method using hyperspectral images is provided, which estimates water depth from hyperspectral images using a regression equation.

In addition, after the step (g), (h) the regression analysis module 330, of the plurality of wavelengths included in the hyperspectral photographing value, other than the pair of two wavelengths in the step (e) Generating a regression equation for each of a plurality of pairs of two wavelengths by selecting a wavelength of, confirming the hyperspectral photographing value for this, and repeating steps (f) to (g); (i) selecting, by the regression equation determination module 340, a regression equation having the highest correlation among the plurality of generated regression equations; And (j) determining, by the regression equation determination module 340, the two wavelengths applied to the band ratio (X) used in the regression equation selected in step (g) as a pair of optimal band ratio wavelengths. It is preferable to include.

In addition, after the step (j), (k) the observation value estimation module 350 corresponds to the pair of optimal band ratio wavelengths determined in step (j) from the hyperspectral photographing values of points other than the plurality of points. Checking the two hyperspectral photographing values, dividing one of them into the other and calculating a band ratio of the corresponding point; And (l) the observation value estimation module 350, inputting the calculated band ratio to the regression equation selected in step (i), and estimating the depth observation value of the corresponding point.

In addition, the step (l) may include calculating, by the observation value estimation module 350, a regular expression index (NDWI) using the intensity of light for a wavelength corresponding to near infrared rays in the hyperspectral image; And outputting, by the observation value estimation module 350, a water depth as 0 when the calculated regular vegetation index (NDWI) is less than 0.

In addition, in the step (i), the regression equation determination module 340 calculates correlation coefficients for the plurality of generated regression equations, so that each wavelength in the pair of two wavelengths is converted to the X axis and the Y axis. Generating a correlation coefficient map, wherein in step (i), the regression equation determination module 340 determines a point with the highest correlation coefficient using the correlation coefficient map generated in step (i). It is preferable to include the step of determining and determining each wavelength constituting the corresponding point as a pair of the optimal band ratio wavelengths.

In addition, in the step (c), (c11) the correspondence point confirmation module 262, in any two hyperspectral information (A, B) obtained for each image unit on which the geometric correction has been performed, With respect to the XY coordinates (x, y) of any one pixel of the hyperspectral information (A) and the pixel value (I(x, y)) at the corresponding coordinates, one of the other hyperspectral information (B) Calculate a normalized correlation of the XY coordinates (x', y') of the pixels and the pixel values (I(x', y')) at the corresponding coordinates, but any one of the hyperspectral information (A) By calculating all the normal correlation coefficients of each of the pixels of the other hyperspectral information (B) for each of all pixels of the two hyperspectral information (A, Confirming the corresponding point of B); (c12) image matching the two hyperspectral information (A, B) based on the identified corresponding point by the image matching module 260; And (c13) the image matching module 260 performs steps (c11) to (c12) for each pair of hyperspectral information for each image unit obtained in step (a), and image matching It is preferable to include the step of obtaining a hyperspectral image of the collected area.

In addition, before the step (c11), (c21) the integral image calculation module 261, in the hyperspectral information obtained for each image unit on which the geometric correction is performed, for each pixel, any one reference coordinate The method further includes calculating an integral image pixel value based on, wherein the pixel value in the step (c11) is preferably an integral image pixel value calculated in the step (c21).

In addition, before the step (c21), the (c31) normalization module 240 sets the XY coordinates of the pixels of the hyperspectral information of any one image unit as an integer order pair (x, y), and the set XY The XY coordinates (x, y) are calculated so that the unit distance between the ordered pairs (x, y) of each integer is √2 by calculating the average coordinates of the coordinates and setting the calculated average coordinates as the origin (0, 0). It is preferable to further include the step of normalizing the hyperspectral information of any one of the image units by moving to the normalized XY coordinates (x', y').

In addition, after the step (c31), (c32) the normalization module 240 performs XY coordinates (x, y) before normalization of any one of the hyperspectral information and XY coordinates (x', y') after normalization. Calculating homography (H), which is a determinant including a parameter, using the value of And (c33) normalizing the other hyperspectral information by using the calculated homography (H).

In addition, in the step (c32), homogeneous coordinates are utilized so that the XY coordinates before normalization are (x, y, 1) and are referred to as determinant A, and the XY coordinates after normalization are (x', y', As 1), it is referred to as the determinant A', and the homography (H) is calculated using the following equation, but using a plurality of XY coordinates, and in step (c33), the other hyperspectral information is transferred to the determinant A. By substituting, it is preferable that the determinant A'which is a normalized XY coordinate is calculated.

Another embodiment of the present invention for solving the above problems provides a program that is stored in a computer-readable recording medium and executes the above-described method, and a computer-readable recording medium in which the code is recorded.

The method according to the present invention obtains excellent hyperspectral images with high accuracy and precision by matching hyperspectral information secured in small image units (line units) with high accuracy, and based on this, the depth of water is determined with high accuracy and precision. Can be estimated. In particular, it is possible to effectively correct the shake of an unmanned aerial vehicle without using a high-weight, high-performance gimbal. Used in conjunction with the above-described image matching technology, it allows obtaining excellent hyperspectral images with high synergistic effects.

In addition, conventionally, only one wavelength with a value identified as the closest to the wavelength value of an actual object is used, so if the matching is incorrect or there is an error in the measured value, the accuracy of the actual estimated depth is low. However, according to the present invention, the photographed value of the wavelength identified as the closest to the value of the wavelength of the actual water depth is used as the numerator, and the photographed value of the identification as the farthest from the value of the actual water depth is used as the denominator. Unlike matching by wavelength, since the depth value is estimated through a double matching process, accuracy can be improved.

In addition, this method is computer-programmed with a non-stop concept as a single program, and when the depth of observation of multiple points and the captured hyperspectral image are input, the hyperspectral image is appropriately preprocessed and After processing, the regression equation is automatically determined based on this, and the depth can be automatically estimated.

Particularly, in the past, it was necessary to perform complex calculations and data transmission while returning to and from a number of programs, but all of these problems are solved, and the computation time that took several days can be reduced by about 50% or more by using the integral image.

1 shows a system for carrying out the method according to the invention.
2 is a flow chart illustrating an image processing method among methods according to the present invention.
3 is a flow chart illustrating an image analysis method among methods according to the present invention.
4 is a photograph for explaining an example of an unmanned aerial vehicle used in the method according to the present invention.
5 is a schematic diagram illustrating a method of obtaining hyperspectral information by an unmanned aerial vehicle in the method according to the present invention.
6 is a schematic diagram for explaining geometric correction in an image processing method among methods according to the present invention.
7 is a schematic diagram for explaining image matching in an image processing method among methods according to the present invention.
8 is a schematic diagram for explaining a method of using a normal correlation coefficient for image matching in an image processing method among methods according to the present invention.
9 is a schematic diagram illustrating a method of using an integral image during image matching in an image processing method among methods according to the present invention.
10 is a schematic diagram illustrating a normalization step in an image processing method among methods according to the present invention.
11 is a diagram illustrating a band ratio of an image analysis method among methods according to the present invention.
12 is a diagram for explaining a process in which regression analysis is performed using a band ratio and an observed depth of water in an image analysis method according to the present invention.
13 shows a correlation map set by an image analysis method among methods according to the present invention.
14 illustrates a method of selecting a regression equation using a correlation map by an image analysis method among methods according to the present invention.
15 shows a screen of a program in which the method according to the present invention is implemented.
16 shows a screen in which a final result is output through candidate determination after completing the depth estimation by applying the water body determination step in the method according to the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Hereinafter, a method of estimating depth, which is an observation value, with high accuracy without actually measuring by using hyperspectral images will be described. However, if the concept of the method described below and described in the claims is applied, it will be possible to identify objects other than the depth of water (for example, sea color, dark body, vegetation, etc.), and this process is the same or equal scope. If made according to the concept within, it will be understood by those of ordinary skill in the art that the scope of the rights thereto may also be included within the scope of the present invention as claimed in the claims below.

Description of the system

The system in which the method according to the present invention is performed includes an unmanned aerial vehicle 100 equipped with a hyperspectral sensor 140 to capture an hyperspectral image, and an image processing unit that performs processing to improve the accuracy of the captured hyperspectral image. It may be divided into 200 and an image analysis unit 300 that performs a function of estimating water depth based on the processed hyperspectral image.

The unmanned aerial vehicle 100 may be, for example, a drone, but may be any type of aircraft flying at a short distance from the surface.

The unmanned aerial vehicle 100 includes a gimbal 110 that has a gyro sensor and an acceleration sensor to move the camera 130 in the opposite direction of movement, a shock absorber 120 provided in the gimbal 110 to absorb shock, It includes a camera 130 on which the hyperspectral sensor 140 is mounted, and a communication unit 150 capable of transmitting the hyperspectral information obtained from the hyperspectral sensor 140 to the image processing unit 200.

The gimbal 110 may use any structure that can be secured, but in consideration of weight, 1, 2, or 3 axis gimbal is preferable to a 4 axis or 5 axis gimbal. However, the limitation of shake correction of the 1st, 2nd, and 3rd axis gimbal can be overcome by compensation of the shock absorber 120.

In addition, the unmanned aerial vehicle 100 may further include a GPS sensor for checking location information.

An example of such an unmanned aerial vehicle 100 is shown in FIG. 4.

The image processing unit 200 includes a communication module 210, a flight path setting module 220, a photographed image input module 230, a normalization module 240, a geometric correction module 250, and an image matching module 260. And a hyperspectral image output module 270.

The communication module 210 communicates with the communication unit 150 of the unmanned aerial vehicle 100 by wire or wirelessly, so that the photographed image input module 230 receives the hyperspectral information obtained from the hyperspectral sensor 140. The information to be received may include location information.

The flight path setting module 220 performs a function of setting the flight path of the unmanned aerial vehicle 100 to secure the image based on the collection area to which the hyperspectral information is to be collected.

The normalization execution module 240 performs a normalization function of the secured hyperspectral information. In all geometric transformations, rotation, size, and preservation of parallelism occur with respect to the origin (0, 0), so it is necessary to normalize each ordered pair of XY coordinates present in each image.

The geometric correction performance module 250 performs geometric correction to compensate for distortion caused by geometric conditions such as a curvature of the ground from the obtained hyperspectral information.

The image matching module 260 performs a function of matching a plurality of secured hyperspectral information. To this end, an integral image calculation module 261 and a correspondence point confirmation module 262 are provided.

The hyperspectral image output module 270 performs a function of outputting a hyperspectral image obtained as a normalized, geometrically corrected, and image-matched final result.

A method of performing specific functions of each module will be described later.

The image analysis unit 300 includes a hyperspectral observation value input module 310, a band ratio calculation module 320, a regression analysis module 330, a regression equation determination module 340, and an observation value estimation module 350. do.

The hyperspectral observation value input module 310 performs a function of inputting a depth (hereinafter referred to as “depth observation value”), which is an actual hyperspectral observation value for a plurality of predetermined points. That is, the actual depth of water to check the correlation with the photographed value secured in the hyperspectral image is input.

The band ratio calculation module 320 calculates a band ratio using a photographing value (eg, a pixel value) secured from the hyperspectral image from the image processing unit 200.

The regression analysis module 330 calculates a regression equation by performing a regression analysis of the observed depth value and the band ratio.

The regression equation determination module 340 performs a function of determining a regression equation having the highest correlation by calculating a correlation between a plurality of regression equations, and determining an optimal band ratio through this.

Observation value estimation module 350, using the regression equation determined in the regression equation determination module 340, when a specific hyperspectral image is input (that is, when an arbitrary band ratio is input), based on the observation value It estimates the actual depth and identifies the actual object based on this.

A method of performing specific functions of each module will be described later.

Description of the method

The method according to the invention will be described. An image processing method with reference to FIG. 2 and an image analysis method with reference to FIG. 3 will be described.

First, an image processing method will be described with reference to FIGS. 2 and 5 to 10.

The flight path setting module 220 sets the flight path of the unmanned aerial vehicle 100 using the collection area (S110). The collection area will be the water body for which the water depth is to be estimated and the area around it. When the hyperspectral sensor 140 performs line scanning, as shown in FIG. 5, the angle of view of the unmanned aerial vehicle 100 with respect to a collection area that is a target for obtaining hyperspectral information ) In consideration, the flight path of the unmanned aerial vehicle 100 is set. The setting method may be any according to the prior art, and a detailed description thereof will be omitted.

Then, while flying along the set flight path, the unmanned aerial vehicle 100 acquires hyperspectral information for each preset image unit through line scanning using the hyperspectral sensor 140 (S120). Here, the image unit is preset by a manufacturer according to the characteristics of the camera 130 or the hyperspectral sensor 140, and hyperspectral information may be obtained in a different manner according to each feature.

With respect to the hyperspectral information identified in this way, geometric correction is performed (S140), and image matching is performed (S150), so that a hyperspectral image that is an image analysis target is secured.

Explain a specific method.

Geometric correction is made by the geometric correction performing module 250 (see FIG. 6). Specifically, geometric correction is performed using the location information of each pixel on the hyperspectral information obtained for each image unit. Any conventionally known method may be applied.

Image registration is a process of matching a plurality of hyperspectral information using a certain reference point (see FIG. 7 ). At least two pairs of a plurality of hyperspectral information are photographed so that they partially overlap each other.A predetermined reference point is found, and two pairs of hyperspectral information are superimposed based on this reference point, and this process is performed by each pair of all hyperspectral information. By repeating at, a hyperspectral image is obtained. That is, the basis of image registration is a process of superimposing two hyperspectral information.

In the present invention, focusing on the "correspondence point extraction algorithm", image matching is performed accordingly. This is achieved by extracting a template from one of the two hyperspectral information with overlapping regions, and finding the point with the highest correlation from the other hyperspectral information. It is an algorithm that sets as the reference point of overlap, that is, the matching point. This is shown in Figure 8.

Specifically, image registration of two hyperspectral pieces of information A and B with overlapping regions (indicated by image1 and image2 in Fig. 8) will be described.

First, the correspondence point identification module 262, in any two superspectral information (A, B) overlapping, the XY coordinates (x, y) of any one pixel of any one hyperspectral information (A), and the corresponding For the pixel value (I(x, y)) at the coordinates, the XY coordinates (x', y') of any one pixel of the other hyperspectral information (B) and the pixel value (I( Calculate the normalized correlation of x', y')). This process, as described later, is the XY coordinates (x') of all pixels of the template of the hyperspectral information (B), which are different for each of the XY coordinates (x, y) of all pixels of the template of the hyperspectral information (A). , y') are all matched to achieve multiple times.

The general formula for calculating the normal correlation coefficient is the same as Equation 1 below, but in the present invention, in order to secure higher accuracy, Equation 2 using more pixel values is used.

The XY coordinates of the hyperspectral information (A) are expressed as (x, y), and the XY coordinates of the hyperspectral information (B) are expressed as (x', y'). The top bar means the average value (this is the same for all formulas below). n is the total number of pixels, I(x, y) is the pixel value corresponding to (x, y) in the XY coordinates, the pixel value of the hyperspectral information (A) is expressed as I ₁ and the hyperspectral information (B) The pixel value of is represented by I ₂ . W is the horizontal pixel size of the template, and H is the vertical pixel size of the template.

In this way, if the normal correlation coefficient is calculated for every pixel coordinate on the template and expressed as XY coordinates, a correlation map as shown in the lower right corner of FIG. 8 can be calculated. In the correlation coefficient map, the XY coordinate of the location having the maximum value, that is, the point where the value of the normal correlation coefficient is large, is identified as the corresponding point of the two hyperspectral information (A, B).

Now, the image matching module 260 performs image matching of the two hyperspectral information A and B based on the corresponding point.

The image matching module 260 performs image registration by performing this process for each pair of all hyperspectral information for each image unit, and through this, a hyperspectral image in which all captured images are matched is obtained.

However, in this case, since a considerable number of operations are performed, the operation speed may be slowed according to the performance of the image processing unit 200. Accordingly, in another embodiment of the present invention, in order to implement a faster operation speed, image matching may be performed in a state in which the amount of calculation is greatly reduced by using an integral image. In the integrated image, a reference coordinate is set, and all pixel values of each pixel are summed using a direction based on this, and then used as a pixel value required for a normal correlation coefficient calculation as shown in Equation 2.

With reference to FIG. 9, the concept of using the integral image will be described. The left side shows a general image. The number recorded for each pixel is a pixel value. The right side shows an integral image using the left image. In the integral image here, after setting "2" in the upper left corner as a reference coordinate, it is an integral image formed by summing each value in the left and lower directions. For example, "13" in the 2nd line and 3rd column is a value of 2+1+5+0+3+2.

If you want to obtain the sum of the pixel values of the nine center pixels in the corresponding image, the general image on the left must perform a total of nine operations. However, in the integral image on the right, it is sufficient to perform four operations. In other words, by generating an integral image, the amount of computation is reduced to 50% or less.

Specifically, the method of using the integral image in the hyperspectral image is as follows. First, a new pixel value (II(x, y)) according to the integral image is set for each pixel using Equation 3.

Each symbol is as indicated in Equation 2. Now using this, the normal correlation coefficient described in Equation 2 can be calculated by Equation 4.

Here, the value of the standard deviation σ used for the denominator can be calculated through Equation 5. As stated above, that I means the pixel value.

In this way, once the integral image is created and placed, the use of Equation 3 to Equation 5 eliminates the need to use a large amount of calculation of Equation 2 for each operation, so that the normal correlation coefficient can be quickly calculated. It is preferable because it can be.

Meanwhile, as described above, in all geometric transformations, since rotation, size, and preservation of parallelism occur based on the origin (0, 0), in an embodiment of the present invention, the XY coordinates present in each image After normalization of each ordered pair is performed, geometric correction and image matching may be performed.

Specifically, the normalization module 240 sets the XY coordinates of the pixels of the hyperspectral information of any one image unit as an integer order pair (x, y), calculates the average coordinates of the set XY coordinates, and the calculation The XY coordinates (x, y) are normalized to the XY coordinates (x', y) so that the unit distance between the ordered pairs (x, y) of each integer is √2 by setting the average coordinates as the origin (0, 0). By moving to'), the hyperspectral information of any one image unit is normalized. This concept is illustrated in FIG. 10.

More specifically, this operation may use Equation 6.

Here, A is an XY coordinate in hyperspectral information to be normalized (ie, not normalized), and A'is a normalized XY coordinate. As for the coordinates here, it is preferable to use a homogeneous coordinate system for parameter calculation to be described later. That is, the XY coordinates are referred to as the determinant A'as (x, y, 1), and the XY coordinates after normalization are referred to as the determinant A'as (x', y', 1). It is expressed as a kind of vector. In other words, in the homogeneous coordinate system, the (x, y) coordinate can be understood as a line made of countless points.

H is a homography, a matrix composed of parameters for normalization (h ₁₁ ~ h ₃₃ ). That is, if homography (H) is calculated, all hyperspectral information can be easily normalized.

For the calculation of homography (H), Equation 6 is used, but any one normalized value is required. That is, for any one of the hyperspectral information, after normalizing separately using the method described with reference to FIG. 10, homography H is calculated using the value. For example, if you know four ordered pairs of x and y of the XY coordinates before normalization, and the normalized values of the four ordered pairs are known, Equation 7 and Equation 8 can be derived by using Equation 6 And, accordingly, each parameter (h ₁₁ ~ h ₃₃ ) can be checked.

When homography (H) is calculated by Equation 8, now, by substituting the XY coordinates of all pixels of all hyperspectral images into Equation 6, normalization can be performed with only a small amount of calculation without the need to separately calculate the normalized value. Do.

In this way, the presence of homography H means that not only coordinate changes between corresponding points are possible, but also all points existing in the coordinate system of the two hyperspectral information can be converted to each other. Therefore, when coordinate transformation is performed by normalizing the other hyperspectral information based on any one of the two hyperspectral information, the two hyperspectral images are moved to the same coordinate system.

Next, an image analysis method will be described with reference to FIGS. 3 and 11 to 16.

The hyperspectral image in which the above-described image processing method has been performed is secured. In addition, in order to estimate the depth, the depth observation value Y, which is each hyperspectral observation value, must be separately confirmed for each of a plurality of predetermined points (S210). That is, the depth observation value Y of each of a plurality of predetermined points is input to the hyperspectral observation value input module 310. Here, for the accuracy of the regression analysis, it is preferable that the number of points at which the observed depth value (Y) is confirmed is larger.

Next, a photographed image having hyperspectral information is input to the photographed image input module 210, and a hyperspectral photographing value (which is a pixel value for a plurality of wavelengths x) at each of a plurality of predetermined points where the depth observation value was inputted ( I(x)) is confirmed (S220).

Next, the band ratio calculation module 320 selects a pair of arbitrary two wavelengths (λ ₁ , λ ₂ ) among a plurality of wavelengths (x) included in the hyperspectral photographing value, and hyperspectral photographing values for each (I(λ ₁ ), I(λ ₂ )) is checked (S230). At this time, the band ratio X is calculated by dividing any one of the identified two hyperspectral photographing values I(λ ₁ ) and I(λ ₂ ) by the other (S240) (see FIG. 11). If this is expressed by an equation, it is shown in Equation 9 below.

In other words, the band ratio is calculated for each point to confirm the depth observation value (Y) and the band ratio (X), which are performed individually for each of the plurality of points.

Next, the regression analysis module 330 assumes that the observed depth value (Y) and the calculated band ratio (X) are in a linear relationship, and the observed depth value (Y) identified in step S210 at each of a plurality of points By using the band ratio X calculated in step S240, a and b of the regression equation (Y=aX+b) are calculated to generate a regression equation (Y=aX+b) (S250). Here, a specific formula for calculating a is the same as in Equation 10 below, and when the value of a is calculated, the value of b is naturally calculated. n is the number of depth observations, X is the band ratio, and Y is the depth observation.

12 shows an example. Here, a regression equation calculated using the band ratio (X) at five points (P1, P2, P3, P4, P5) (i.e., n=5) and the observed depth value (Y) is shown in the lower right. .

Meanwhile, this process may be performed for each pair while changing the two wavelengths selected in step S230. Even if the predetermined points are the same and the observed depth values Y at each point are the same, if two wavelengths are selected differently, the final calculated regression equation is different.

Specifically, the regression analysis module 330 first selects an arbitrary wavelength other than the selected pair of two wavelengths, checks the hyperspectral photographing value for this, and repeats the steps S240 to S250, a plurality of regression equations Generates (S260).

Next, the regression equation determination module 340 calculates the correlation coefficients of the plurality of generated regression equations, and selects a regression equation having the highest correlation coefficient among them (S270). At this time, the two wavelengths applied to the band ratio (X) used in the selected regression equation are determined as a pair of optimal band ratio wavelengths.

For example, if the number of wavelengths included in the hyperspectral information is N, the number of pairs of two wavelengths is _N C _{2 due} to the combination, so the regression equation and the correlation coefficient are calculated as much as the number, The optimal regression equation and band ratio and their wavelength will be calculated.

Meanwhile, this method can be achieved using a correlation coefficient map. An example of the correlation coefficient map is shown in FIG. 13 (note that this is a concept different from the numerical value applied to the correlation coefficient map of the image processing method described in FIG. 8). Here, the regression equation determination module 340 calculates correlation coefficients for a plurality of generated regression equations, but in the pair of two wavelengths, one wavelength is the X axis and the other wavelength is the Y axis. The generated correlation coefficient map is shown.

Next, the regression equation determination module 340 determines a point having the highest correlation coefficient using the correlation coefficient map, and determines each wavelength constituting the point as a pair of the optimal band ratio wavelengths.

Fig. 14 shows the concept of automating this approach. By extracting the photographed value for each wavelength, which is a spectral characteristic, each band ratio is calculated through this, and by regression analysis using the depth observation value, a correlation coefficient map is generated, and by calculating the correlation coefficient for each pair of wavelengths, The optimal band ratio wavelength and the optimal regression equation are determined.

15 shows an output screen of a program implementing this. The correlation coefficient map on the right is shown, and it can be seen that the correlation coefficient and regression equation for each wavelength are automatically calculated below it.

Through such a process, when the optimum regression equation and the optimum band ratio are determined, the depth observation value can be estimated using only the wavelength and photographing value of the hyperspectral information included in the hyperspectral image even at a position where there is no depth observation value.

For example, when the observation value estimation module 350 calculates a band ratio by checking two hyperspectral photographing values corresponding to a pair of optimal band ratio wavelengths among photographed values included in the hyperspectral image, it is optimally regressed. By inputting it into the equation, you can estimate the depth observation at that point with high accuracy.

On the other hand, in an embodiment of the present invention, a step of preventing a phenomenon in which the depth observation value, which is the final result, is distorted due to vegetation or the like may be further included. To this end, the observation value estimation module 350 calculates a normal vegetation index (NDWI) by using the intensity of light for a wavelength corresponding to the near infrared (NIR) in the hyperspectral image, and the calculated normal vegetation index (NDWI) is 0. If it is smaller, it is not judged as a water body (ie, it is judged as vegetation, etc.). In this case, it is possible to output that the water depth is 0 or not the water body in the program.

The regular vegetation index (NDWI) is calculated by the following formula. Here, NIR is the intensity of light at a wavelength corresponding to near-infrared rays among the values included in the hyperspectral information of the hyperspectral image, and Green is the intensity at the wavelength corresponding to green among the values included in the hyperspectral information of the hyperspectral image. It is the intensity of light.

The regular expression index (NDWI), which is the value calculated by Equation 11 above, has a value in the range of -1 to 1. If the normal vegetation index (NDWI) is more than 0, it is determined as a water body, and the estimated depth observation value in the previous step can be output as it is, but if it is less than 0, it is determined that it is not a water body and the depth is output as 0 or is not a water body. Print. In other words, it compensates for the fact that the water depth was partially calculated due to distortion caused by vegetation and the like.

By performing these steps, the accuracy of estimating depth observation values can be further increased.

The left side of FIG. 16 shows an image captured by a general camera, and the right side shows the result of performing the method according to the present invention. By using the regular vegetation index (NDWI), the water chain part (blue part) and the non-water chain part were distinguished, and in the water chain part, all depths of each point were calculated with high accuracy by an estimation method using a regression equation. By comparing this with actual data, it was possible to verify that the accuracy of the method according to the present invention is high.

In the above, the present specification has been described with reference to the embodiments shown in the drawings so that those skilled in the art can easily understand and reproduce the present invention, but this is only exemplary, and those skilled in the art can use various modifications and equivalents from the embodiments of the present invention. It will be appreciated that embodiments are possible. Therefore, the scope of protection of the present invention should be determined by the claims.

100: unmanned vehicle
110: Gimbal
120: shock absorber
130: camera
140: hyperspectral sensor
150: communication department
200: image processing unit
210: communication module
220: flight path setting module
230: photographed image input module
240: normalization execution module
250: geometric correction performance module
260: image matching module
261: integral image operation module
262: correspondence point confirmation module
270: hyperspectral image output module
300: image analysis unit
310: Hyperspectral observation value input module
320: band ratio calculation module
330: regression analysis module
340: regression determination module
350: observation value estimation module

Claims (13)

(a) the unmanned aerial vehicle 100, while flying along a set flight path, using the hyperspectral sensor 140 to obtain hyperspectral information for each preset image unit through line scanning;
(b) performing, by the geometric correction module 250, performing geometric correction on the hyperspectral information obtained for each image unit by using position information of each pixel;
(c) The image matching module 260 acquires a hyperspectral image of a collection area to which the hyperspectral information is to be acquired by matching the hyperspectral information obtained for each image unit on which the geometric correction has been performed. Step to do;
(d) inputting a depth observation value (Y) of each of a plurality of predetermined points into the hyperspectral observation value input module 310;
(e) The band ratio calculation module 320, at each of the plurality of predetermined points, from the hyperspectral image obtained in step (c), the hyperspectral photographing value (I), which is a pixel value for a plurality of wavelengths (x). (x)), select an arbitrary pair of two wavelengths (λ ₁ , λ ₂ ) among a number of wavelengths (x) included in the hyperspectral photographing value, and select the hyperspectral photographing value (I( determining λ ₁ ), I(λ ₂ ));
(f) The band ratio calculation module 320 divides any one of the identified two hyperspectral photographing values (I(λ ₁ ), I(λ ₂ )) into the other and divides the band ratio (X Calculating ); And
(g) The regression analysis module 330 assumes that the depth observation value (Y) and the calculated band ratio (X) are in a linear relationship, and the depth observation value (Y) at the plurality of points and And generating a regression equation (Y=aX+b) by calculating a and b of a regression equation (Y=aX+b) using the calculated band ratio (X),
Estimating the depth of water from the hyperspectral image using the generated regression equation,
Depth estimation method using hyperspectral images.
The method of claim 1,
After the step (g),
(h) The regression analysis module 330 selects a wavelength other than the pair of two wavelengths in step (e) among the plurality of wavelengths included in the hyperspectral photographing value, Generating a regression equation for each of a plurality of pairs of two wavelengths by checking the spectroscopic image value and repeating steps (f) to (g);
(i) selecting, by the regression equation determination module 340, a regression equation having the highest correlation between the band ratio (X) and the depth observation value (Y) among the plurality of generated regression equations ;
(j) further comprising the step of determining, by the regression equation determination module 340, two wavelengths applied to the band ratio (X) used in the regression equation selected in step (g) as a pair of optimal band ratio wavelengths And
Estimating the depth of water from the hyperspectral image using the selected regression equation,
Depth estimation method using hyperspectral images.
The method of claim 2,
After step (j),
(k) The observation value estimation module 350 checks two hyperspectral photographing values corresponding to the pair of optimal band ratio wavelengths determined in step (j) from the hyperspectral photographing values of points other than the plurality of points. And calculating a band ratio of a corresponding point by dividing one of them into the other; And
(l) the observation value estimation module 350 further comprising the step of estimating the observed depth value of the corresponding point by inputting the calculated band ratio to the regression equation selected in step (g),
Depth estimation method using hyperspectral images.
The method of claim 3,
The step (l),
Calculating, by the observation value estimation module 350, a regular vegetation index (NDWI) using the intensity of light for a wavelength corresponding to near-infrared rays in the hyperspectral image; And
Further comprising the step of outputting, by the observation value estimation module 350, a water depth as 0 when the calculated regular vegetation index (NDWI) is less than 0,
Depth estimation method using hyperspectral images.
The method of claim 4,
The regular expression index (NDWI) is calculated by the following formula,

Here, NIR is the intensity of light at a wavelength corresponding to near infrared rays among values included in the hyperspectral information, and Green is the intensity of light at a wavelength corresponding to green among values included in the hyperspectral information,
Depth estimation method using hyperspectral images.
The method of claim 3,
The step (i),
The regression equation determination module 340, by calculating the correlation coefficients for the plurality of generated regression equations, generating a correlation coefficient map in which each wavelength is an X-axis and a Y-axis in a pair of two wavelengths. Including,
Step (j),
The regression equation determination module 340 determines a point having the highest correlation coefficient using the correlation coefficient map generated in step (i), and sets each wavelength constituting the point to the optimal band ratio wavelength pair. Including the step of determining as,
Depth estimation method using hyperspectral images.
The method of claim 1,
The step (c),
(c11) In any two hyperspectral information (A, B) obtained for each image unit on which the geometric correction has been performed, the corresponding point identification module 262 With respect to the XY coordinates (x, y) of the pixel of and the pixel value (I(x, y)) at that coordinate, the XY coordinates (x', y) of any one pixel of the other hyperspectral information (B) ') and the normalized correlation of the pixel values (I(x', y')) at the corresponding coordinates, but the other one for each of all pixels of the one hyperspectral information (A) By calculating all of the normal correlation coefficients of each of the pixels of the hyperspectral information (B) of, confirming the XY coordinates of the point where the value of the normal correlation coefficient is large as a corresponding point of the two hyperspectral information (A, B);
(c12) image matching the two hyperspectral information (A, B) based on the identified corresponding point by the image matching module 260; And
(c13) The image matching module 260 performs steps (c11) to (c12) for each pair of hyperspectral information for each image unit acquired in step (a), and image-matched Including the step of obtaining a hyperspectral image of the collection area,
Depth estimation method using hyperspectral images.
The method of claim 7,
Before step (c11),
(c21) The integral image calculation module 261, in the hyperspectral information obtained for each image unit on which the geometric correction is performed, for each pixel, an integral image based on any one reference coordinate. Further comprising the step of calculating the pixel value,
The pixel value in step (c11) is an integral image pixel value calculated in step (c21),
Depth estimation method using hyperspectral images.
The method of claim 8,
Before step (c21),
(c31) The normalization module 240 sets the XY coordinates of the pixels of the hyperspectral information of any one image unit as an integer order pair (x, y), calculates the average coordinates of the set XY coordinates, and the calculation The XY coordinates (x, y) are normalized to the XY coordinates (x', y) so that the unit distance between the ordered pairs (x, y) of each integer is √2 by setting the average coordinates as the origin (0, 0). '), further comprising the step of normalizing the hyperspectral information of any one image unit,
Depth estimation method using hyperspectral images.
The method of claim 9,
After the step (c31),
(c32) The normalization module 240 uses the values of the XY coordinates (x, y) before normalization of any one of the hyperspectral information and the XY coordinates (x', y') after normalization to determine the parameters. Calculating a homography (H) which is a determinant including; And
(c33) using the calculated homography (H), including the step of normalizing the other hyperspectral information,
Depth estimation method using hyperspectral images.
The method of claim 10,
In the step (c32), homogeneous coordinates are utilized so that the XY coordinates before normalization are (x, y, 1) and are referred to as determinant A, and the XY coordinates after normalization are (x', y', 1). It is referred to as the determinant A', and homography (H) is calculated using the following formula, but using a number of XY coordinates,

In the step (c33), by substituting the other hyperspectral information into the determinant A, the determinant A', which is a normalized XY coordinate, is calculated,
Depth estimation method using hyperspectral images.
Stored on a computer-readable recording medium
A program for executing the method according to any one of claims 1 to 11.
A computer-readable recording medium on which a program code for executing the method according to any one of claims 1 to 11 is recorded.

Images