JP2009295062A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently and automatically obtain the amount of positional deviation between two images taken on different times. <P>SOLUTION: A matching process unit 202 reads two images taken on different times (e.g., a pre-disaster image and a post-disaster image, which are images immediately before and immediately after a disaster, respectively.) from a memory 252; scans a first region, including a prescribed number of pixels on one image (e.g., the post-disaster image), in one pixel increments; and calculates a coefficient of correlation between the first region and a second region, residing at the same position as the first region on the other image (e.g., the pre-disaster image), for each scanning run across the first region, using a processor 251. A processor 203 for calculating the amount of positional deviation estimates the amount of positional deviation between the two images in a unit less than one pixel using the processor 251 on the basis of the coefficient of the correlation calculated by the matching process unit 202. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program. In particular, the present invention relates to a technique (change extraction system) for analyzing a plurality of image data captured from an artificial satellite or an aircraft and extracting a change in the image.

Patent Document 1, Patent Document 2, or Patent Document 3 discloses a method and apparatus for extracting changes in the ground surface from two image data (pair images) captured at different times.
JP-A-2005-78566 JP-A-2005-241886 JP 2007-114994 A

  It is desirable that the process of extracting image changes can be automatically performed without relying on human hands. However, in the apparatus of Patent Document 1, when the temporal imaging conditions such as season and time differ between the paired images, correction is performed in advance so that one of the images matches the imaging condition of the other image. It was necessary to keep it. Furthermore, since the method and apparatus of Patent Document 1 or Patent Document 2 does not include a means for performing registration (registration) processing of a pair image, it is necessary to perform registration manually in advance. . Furthermore, since the apparatus of Patent Document 1, Patent Document 2, or Patent Document 3 does not include means for performing cloud extraction processing from an image, a human can determine whether or not the cause of the extracted change is a cloud. It was necessary to interpret. Furthermore, since the apparatus of Patent Document 1, Patent Document 2, or Patent Document 3 does not include a means for performing texture classification processing, it is necessary for a human to interpret what kind of ground surface has changed. there were. That is, in the apparatus of Patent Document 1, Patent Document 2, or Patent Document 3, since manual work is required, it is difficult to automatically extract changes.

  For example, an object of the present invention is to efficiently and automatically obtain a positional deviation amount between two images taken at different times. Another object of the present invention is to quickly extract changes between two images taken at different times without requiring manual work, for example.

An image processing apparatus according to an aspect of the present invention includes:
An image processing apparatus for storing two images taken at different times in a memory in advance,
Each time the two images are read from the memory, a first region of a predetermined number of pixels is moved by one pixel or more in one image, and the first region is moved, the first region And a matching processing unit that calculates a correlation coefficient between the second region in the same position as the first region in the other image by a processor,
And a misregistration amount calculation processing unit that estimates a misregistration amount between the two images by a processor in units of less than one pixel based on the correlation coefficient calculated by the matching processing unit.

  According to one aspect of the present invention, in the image processing apparatus, the matching processing unit moves the first area in units of one pixel or more in one of two images taken at different times. Each time the first area is moved, a correlation coefficient between the first area and the second area at the same position as the first area in the other image is calculated. A displacement amount calculation processing unit estimates the displacement amount between the two images in units of less than one pixel based on the correlation coefficient, thereby automatically and efficiently calculating the displacement amount between the two images. It becomes possible to ask for.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an example of the configuration of a change extraction system 100 according to the present embodiment.

  The change extraction system 100 includes a stationary optical satellite 101 (an example of a flying object), a satellite image receiving center 102, a satellite image database 103, an image processing device 104, and a terminal device 105.

  The geostationary optical satellite 101 revolves around a circular orbit at an altitude of about 36,000 km above the equator in the same cycle as the rotation of the earth, and takes an image of the ground surface (satellite image) with an optical sensor (an example of an imaging device). It is a satellite. The stationary optical satellite 101 transmits the captured satellite image to the satellite image receiving center 102 by wireless communication. Note that the geostationary optical satellite 101 may be replaced with another flying object such as an orbiting satellite or an aircraft. The optical sensor may be replaced with another imaging device such as a synthetic aperture radar.

  The satellite image receiving center 102 includes equipment such as a communication antenna and is a facility for communicating with the stationary optical satellite 101 on a 24-hour basis. The satellite image receiving center 102 stores the satellite image received from the geostationary optical satellite 101 in the satellite image database 103.

  The satellite image database 103 is a file server equipped with a large-capacity hard disk drive (HDD) capable of storing satellite images for 3 to 4 months. The satellite image database 103 is connected to the satellite image receiving center 102, the image processing device 104, and the terminal device 105 via the network 106.

  The image processing apparatus 104 is an application server that has a CPU (Central Processing Unit), a memory, and the like, and responds to a processing request from the terminal device 105. The image processing apparatus 104 has a registration processing unit 141, a change extraction processing unit 142, a cloud extraction processing unit 143, and a texture classification processing unit 144 as its processing functions. In the figure, only one image processing apparatus 104 is shown, but a plurality of image processing apparatuses 104 may be provided.

  The terminal device 105 is a workstation equipped with a monitor, a keyboard, a mouse, and the like, and is a client that requests the image processing apparatus 104 for processing (service) and receives the request. Although only one terminal device 105 is shown in the figure, a plurality of terminal devices may be provided.

  As described above, in this embodiment, the image processing apparatus 104 performs image registration processing in the change extraction system 100 that analyzes a plurality of image data captured from an artificial satellite or an aircraft and extracts a change in the image. By automatically performing the change extraction process, the cloud extraction process, and the texture classification process, the burden on the image reader can be reduced.

  FIG. 2 is a block diagram illustrating an example of the configuration of the image processing apparatus 104.

  As described above, the image processing apparatus 104 includes the registration processing unit 141, the change extraction processing unit 142, the cloud extraction processing unit 143, and the texture classification processing unit 144. The image processing apparatus 104 includes hardware such as a processor 251, a memory 252, an input device 253, and an output device 254. The hardware is used by each unit of the image processing apparatus 104. The image processing apparatus 104 stores at least two images taken at different times in the memory 252 in advance. Specifically, the image processing apparatus 104 stores an image of the ground surface taken from above, that is, the above-described satellite image in the memory 252.

  The registration processing unit 141 reads two image data (pair images) taken at different times from the satellite images from the memory 252 and automatically aligns the images using the processor 251. As will be described later, the registration processing unit 141 uses the variance value and the edge amount as an index for selecting a small region for matching. The registration processing unit 141 includes a matching small region selection processing unit 201, a matching processing unit 202, a positional deviation amount calculation processing unit 203, and a resampling processing unit 204. The operation of each part will be described later.

  The change extraction processing unit 142 automatically extracts changes in the pair images using the processor 251. As will be described later, the change extraction processing unit 142 uses a variance value, an edge amount, and a correlation coefficient as an index of change extraction. The change extraction processing unit 142 includes a feature amount calculation processing unit 205. The operation of the feature amount calculation processing unit 205 will be described later.

  The cloud extraction processing unit 143 automatically determines from the cloud movement vector information in the pair image using the processor 251 whether or not the object shown in the pair image is a cloud. As will be described later, when the cloud extraction processing unit 143 determines whether or not the cloud is a cloud from the cloud movement vector, it uses a dispersion value, an edge amount, and a correlation coefficient as an index of vector reliability. The cloud extraction processing unit 143 includes a movement vector extraction processing unit 206, a movement amount processing unit 207, and a quality index calculation processing unit 208. The operation of each part will be described later.

  The texture classification processing unit 144 automatically classifies the ground surface of the image into one of a plant, a water area, a land, a mountain surface, and a cloud using the processor 251. As will be described later, the texture classification processing unit 144 uses a value of a red wavelength, an NDVI (Normalized / Difference / Vegetation / Index) value, and an index for classifying the texture into any one of plants, water, land, mountains and clouds. The saturation (Saturation) and lightness (Intensity) after HSI (Hue / Saturation / Intensity) conversion are used. The texture classification processing unit 144 includes an NDVI calculation processing unit 209 and an HSI conversion processing unit 210. The operation of each part will be described later.

  FIG. 3 is a diagram illustrating an example of a hardware configuration of the image processing apparatus 104.

  In FIG. 3, an image processing apparatus 104 is a computer, and includes a display device 901 having a CRT (Cathode / Ray / Tube) or LCD (liquid crystal display) display screen, a keyboard 902 (K / B), a mouse 903, and an FDD 904 ( Hardware such as a flexible disk, a drive, a CDD 905 (Compact, a disk, and a drive), and a printer device 906 are provided, and these are connected by a cable or a signal line.

  The image processing apparatus 104 includes a CPU 911 that executes a program. The CPU 911 is an example of the processor 251. The CPU 911 includes a ROM 913 (Read / Only / Memory), a RAM 914 (Random / Access / Memory), a communication board 915, a display device 901, a keyboard 902, a mouse 903, an FDD904, a CDD905, a printer device 906, and a magnetic disk. It is connected to the device 920 and controls these hardware devices. Instead of the magnetic disk device 920, a storage medium such as an optical disk device or a memory card reader / writer may be used.

  The RAM 914 is an example of a volatile memory. The storage media of the ROM 913, the FDD 904, the CDD 905, and the magnetic disk device 920 are an example of a nonvolatile memory. These are examples of the memory 252. The communication board 915, the keyboard 902, the mouse 903, the FDD 904, the CDD 905, and the like are examples of the input device 253. Further, the communication board 915, the display device 901, the printer device 906, and the like are examples of the output device 254.

  The communication board 915 is connected to a LAN (local area network) or the like. The communication board 915 is not limited to a LAN, but is the Internet, or an IP-VPN (Internet, Protocol, Private, Network), a wide area LAN, a WAN (Wide Area Network) such as an ATM (Asynchronous, Transfer, Mode) network, or the like. It does not matter if it is connected to. LAN, the Internet, and WAN are examples of the network 106.

  The magnetic disk device 920 stores an operating system 921 (OS), a window system 922, a program group 923, and a file group 924. The programs in the program group 923 are executed by the CPU 911, the operating system 921, and the window system 922. The program group 923 stores a program for executing a function described as “˜unit” in the description of the present embodiment. The program is read and executed by the CPU 911. The file group 924 includes data and information described as “˜data”, “˜information”, “˜ID (identifier)”, “˜flag”, and “˜result” in the description of this embodiment. Signal values, variable values, and parameters are stored as items of “˜file”, “˜database”, and “˜table”. The “˜file”, “˜database”, and “˜table” are stored in a storage medium such as a disk or a memory. Data, information, signal values, variable values, and parameters stored in a storage medium such as a disk or memory are read out to the main memory or cache memory by the CPU 911 via a read / write circuit, and extracted, searched, referenced, compared, and calculated. Used for processing (operation) of the CPU 911 such as calculation / control / output / printing / display. Data, information, signal values, variable values, and parameters are temporarily stored in the main memory, cache memory, and buffer memory during processing of the CPU 911 such as extraction, search, reference, comparison, calculation, control, output, printing, and display. Is remembered.

  In addition, the arrows in the block diagrams and flowcharts used in the description of this embodiment mainly indicate input / output of data and signals, and the data and signals are the memory such as the RAM 914, the flexible disk (FD) of the FDD 904, and the compact of the CDD 905. Recording is performed on a recording medium such as a disk (CD), a magnetic disk of the magnetic disk device 920, another optical disk, a mini disk (MD), or a DVD (Digital Versatile Disc). Data and signals are transmitted by a bus 912, a signal line, a cable, and other transmission media.

  In addition, what is described as “˜unit” in the description of this embodiment may be “˜circuit”, “˜device”, “˜device”, and “˜step”, “˜process”. , “˜procedure”, and “˜processing”. That is, what is described as “˜unit” may be realized by firmware stored in the ROM 913. Alternatively, it may be realized only by software, or only by hardware such as an element, a device, a board, and wiring, or a combination of software and hardware, and further by a combination of firmware. Firmware and software are stored as programs in a recording medium such as a magnetic disk, flexible disk, optical disk, compact disk, minidisk, or DVD. This program is read by the CPU 911 and executed by the CPU 911. That is, the program causes the computer to function as “to part” described in the description of the present embodiment. Alternatively, it causes the computer to execute the procedures and methods described in the description of the present embodiment.

  Hereinafter, the operation of the change extraction system 100 will be described with an example immediately after an earthquake occurs.

  The image reader who has received a report of the occurrence of the earthquake operates the keyboard and mouse of the terminal device 105 to input the disaster time and the estimated location of the epicenter, and requests the image processing device 104 to perform processing. Do.

  Next, based on the input information, the image processing apparatus 104 acquires pair images before and after the disaster from the satellite image database 103 via the network 106, and performs registration processing, change extraction processing, and cloud extraction processing. Then, the texture classification process is performed, and the processing result is provided to the terminal device 105. Each processing content will be described later.

  Next, the terminal device 105 displays the processing result provided from the image processing device 104 on the monitor. Here, the processing results are a pair image registration processing result, a change extraction processing result, a cloud extraction processing result, and a texture classification processing result. When displaying the processing result on the monitor, the images are displayed so as to overlap each other like a cell image (layer display).

  Finally, the image reader confirms the disaster area and the disaster situation based on the processing result displayed on the monitor. For example, in order to confirm the location extracted by the change extraction result, the layer is displayed so as to overlap the registration result. In addition, in order to confirm whether or not the cause extracted from the change extraction result is a cloud, the layer is displayed so as to overlap the cloud extraction result. Further, in order to confirm what kind of ground surface the place extracted by the change extraction result is, the layer is displayed so as to be superimposed on the texture classification result. The image reader then reports the confirmed disaster area and damage status to the disaster countermeasure department. If necessary, request an ambulance team to be dispatched.

  In this way, by using the change extraction system 100 immediately after the occurrence of a disaster and identifying the disaster area and the disaster situation at an early stage, a quick rescue operation can be performed.

  In the above example, the case of an earthquake has been described, but the same applies to other disasters such as a tsunami.

  Hereinafter, the contents of the operation of the registration processing unit 141 of the image processing apparatus 104, that is, the registration processing will be described. A flowchart showing an outline of the processing is shown in FIG. As illustrated in FIG. 4, the registration processing unit 141 inputs a pair image using the input device 253 and stores the pair image in the memory 252. Thereafter, the registration processing unit 141 performs matching small region selection processing (step S411), matching processing (step S412), misregistration amount calculation processing (step S413), resampling processing (for the pair images stored in the memory 252). Step S414) is performed by the processor 251. Then, the registration processing unit 141 stores the aligned pair images in the memory 252 and outputs them with the output device 254. Details are shown below.

  First, the matching small region selection processing unit 201 of the registration processing unit 141 performs a matching small region selection process (step S411). This process is performed only on one of the input pair images. Here, it is assumed that the imaged time is processed for an image after the disaster (post-disaster image). The specific procedure is as follows. First, the matching small area selection processing unit 201 divides the post-disaster image into N × N (N is an integer satisfying N> 0) small areas. Next, the matching small area selection processing unit 201 calculates a dispersion value (pixel value dispersion) and an edge amount in each of the divided small areas from the following expressions (1) and (2). Finally, the matching small area selection processing unit 201 selects a small area having the largest variance value and / or edge amount as a small area for matching. As a result, the correlation coefficient distribution is obtained for the most characteristic small region in the processing described later, and the accuracy of the registration processing is improved.

  In addition, a dispersion value is calculated | required by following Formula.

  The edge amount is obtained by the following equation.

  That is, the sum of the luminance values of the small area images edge-extracted by an edge extraction operator such as SOBEL is used as the edge amount.

  As described above, the matching small region selection processing unit 201 reads two images (for example, pre-disaster image and post-disaster image) taken at different times from the memory 252, and one image (for example, post-disaster image). ) Is divided into a plurality of (for example, N × N) regions each having a predetermined number of pixels, and the matching processing unit 202 determines whether or not the pixel value dispersion (variance value) and the edge amount in each region are based on at least one of them. A first area to be processed (for example, an area having the largest variance value and / or edge amount) is selected.

  Next, the matching processing unit 202 of the registration processing unit 141 performs a matching process (step S412). This process is performed using a normalized correlation method (area correlation method). The normalized correlation method is a method used to detect the same point (corresponding point) in two images, and quantitatively represents the degree of coincidence of small regions extracted from both images. A larger value of the relation number suggests that the two small regions match. The specific procedure is as follows. First, the matching processing unit 202 superimposes the small area for matching selected in the matching small area selecting process (step S411) on the pre-disaster image. Next, the matching processing unit 202 calculates a correlation coefficient from the following equation (3) by the normalized correlation method. Thereafter, the matching processing unit 202 moves the overlapping position up, down, left, and right by at least one pixel, calculates a correlation coefficient at each position, and creates a correlation coefficient distribution (correlation coefficient map).

  Note that the correlation coefficient of the normalized correlation method is obtained by the following equation.

  Next, the misregistration amount calculation processing unit 203 of the registration processing unit 141 performs misregistration amount calculation processing (step S413). This process is a process for calculating the degree of positional deviation between the pair images with an accuracy of a pixel unit or less. The specific procedure is as follows. First, the misregistration amount calculation processing unit 203 extracts the maximum value of the correlation coefficient from the correlation coefficient distribution created in the matching process (step S412). Next, the positional deviation amount calculation processing unit 203 extracts the correlation coefficient on both sides of the maximum value. Finally, the misregistration amount calculation processing unit 203 approximates the extracted three points to a quadratic polynomial (quadratic function) and obtains the peak position of the quadratic polynomial (the apex of the parabola of the quadratic function). The coordinates of this peak position are taken as the amount of positional deviation between the pre-disaster image and the post-disaster image. The amount of positional deviation is calculated in each of the X direction (horizontal direction) and the Y direction (vertical direction).

  As described above, the matching processing unit 202 reads two images (for example, pre-disaster image and post-disaster image) taken at different times from the memory 252, and uses one image (for example, post-disaster image). A first area composed of a predetermined number of pixels (for example, an area selected by the matching small area selection processing unit 201) is moved in units of 1 pixel or more (preferably, in units of one pixel), and the first area is moved. Each time, the processor 251 calculates a correlation coefficient between the first region and the second region at the same position as the first region in the other image (for example, the pre-disaster image). The misregistration amount calculation processing unit 203 estimates the misregistration amount between the two images by the processor 251 based on the correlation coefficient calculated by the matching processing unit 202 in units of less than one pixel.

  Note that the matching processing unit 202 associates the position (coordinates) of the first region with the correlation coefficient between the first region and the second region every time the first region is moved. Store in memory 252. At this time, data stored in the memory 252 (a combination of the position of the first region and the correlation coefficient) corresponds to the correlation coefficient map. The misregistration amount calculation processing unit 203 reads the position of the first region and the correlation coefficient from the memory 252 and calculates an approximate expression indicating the relationship between the position of the first region and the correlation coefficient by the processor 251. Using this approximate expression, the amount of displacement between the two images is estimated in units of less than one pixel. As described above, for example, the positional deviation amount calculation processing unit 203 sets the X coordinate of the point corresponding to the maximum value of the correlation coefficient in the first region, the X coordinate of the left and right points, and the respective points. The corresponding correlation coefficient is read from the memory 252 and an approximate quadratic curve obtained from the three X coordinates and the corresponding correlation coefficient is calculated. Then, the positional deviation amount calculation processing unit 203 estimates the X coordinate of the apex of this approximate quadratic curve (the point corresponding to the approximate value of the maximum correlation coefficient) as the positional deviation amount in the horizontal direction. Similarly, the misregistration amount calculation processing unit 203 calculates the Y coordinate of the point corresponding to the maximum value of the correlation coefficient in the first region, the Y coordinate of the upper and lower points, and the correlations corresponding to the respective points. The number is read from the memory 252 and an approximate quadratic curve obtained from the Y coordinate of the three points and the corresponding correlation coefficient is calculated. Then, the misregistration amount calculation processing unit 203 estimates the Y coordinate of the apex of the approximate quadratic curve (the point corresponding to the approximate value of the maximum correlation coefficient) as the misregistration amount in the vertical direction. By performing such processing, even when the matching processing unit 202 moves the first region in units of pixels, the positional deviation amount calculation processing unit 203 can estimate the positional deviation amount in units of less than one pixel (for example, The result is that the positional deviation amount is −0.1 pixel in the vertical direction and +0.2 pixel in the horizontal direction).

  Next, the resampling processing unit 204 of the registration processing unit 141 performs resampling processing (step S414). This process is a process of resampling one of the pair images and aligning the other image. Here, it is assumed that the post-disaster image is processed. The specific procedure is as follows. First, the resampling processing unit 204 generates an image obtained by shifting the post-disaster image by the positional deviation amount calculated in the positional deviation amount calculation process (step S413). Here, unsampled pixel values are interpolated by calculation from surrounding values. As an interpolation algorithm, Nearest Neighbor, Biliner, Bicubic, Lanczos, or the like can be used. Although the positional shift amount is estimated in units of less than one pixel in the positional shift amount calculation process (step S413), the estimated positional shift amount is an integral multiple of one pixel (for example, +2.0 pixels). Sometimes. In this case, since each pixel of the image shifted by the amount of positional deviation has already been sampled, pixel value interpolation processing is not necessary. On the other hand, when the estimated displacement amount is a fraction of one pixel (for example, +0.2 pixels), that is, when the estimated displacement amount unit is less than 1 pixel, Since each pixel of the image shifted by the shift amount is not sampled yet, the pixel value interpolation process as described above is required.

  Finally, the registration processing unit 141 cuts out and outputs the aligned pre-disaster image and post-disaster image to the same size.

  Hereinafter, the contents of the operation of the change extraction processing unit 142 of the image processing apparatus 104, that is, the change extraction process will be described. A flowchart showing an outline of the processing is shown in FIG. As illustrated in FIG. 5, the change extraction processing unit 142 reads and inputs a registered pair image from the memory 252, and then performs a feature amount calculation process (step S <b> 421) by the processor 251. Then, the change extraction processing unit 142 compares the obtained feature amounts to determine whether or not there is a change by the processor 251, stores the change extracted image in the memory 252, and outputs it by the output device 254. Details are shown below.

  First, the feature amount calculation processing unit 205 of the change extraction processing unit 142 performs a feature amount calculation process (step S421). This process is performed for both of the pair images. Although various feature amounts can be considered, here, the variance value, the edge amount, and the correlation coefficient are referred to as feature amounts. The specific procedure is as follows. First, the feature amount calculation processing unit 205 divides each pair image into N × N small regions. Next, the feature amount calculation processing unit 205 calculates the variance value and the edge amount in the respective divided small areas from the above-described equations (1) and (2). Next, the feature amount calculation processing unit 205 calculates the correlation coefficient of the small area at the same position between the pair images using the above-described equation (3).

  Next, the change extraction processing unit 142 compares the feature amounts. This process is a process of comparing the feature amounts obtained from the pair images to determine whether or not there is a change. The specific procedure is as follows. First, the change extraction processing unit 142 compares the variance value difference, the edge amount difference, and the correlation coefficient in the same small region of the pair image with respective threshold values. Each threshold value is stored in the memory 252 in advance and is read by the change extraction processing unit 142. Next, the change extraction processing unit 142 determines that the small area has changed when all the comparison results are equal to or greater than the threshold, and determines that there is no change otherwise. Thereafter, the change extraction processing unit 142 determines whether or not there is a change for all the small areas.

  Finally, the change extraction processing unit 142 images the presence / absence of a change and creates a change extraction image. Various imaging methods are conceivable, and there are a method of binarizing and displaying the presence / absence of change, a method of displaying by color coding according to the magnitude of change, and the like.

  The change extraction process described above is based on the premise that the above-described registration process is performed with high accuracy. However, the following device may be used in consideration of other cases. . In the flowchart of FIG. 5, a matching process (step S412) and a feature amount calculation process (step S421) are added immediately before the feature amount comparison. In the added matching process (step S412), the change extraction processing unit 142 creates a correlation coefficient distribution by the method described above. In the added feature amount calculation process (step S421), the change extraction processing unit 142 calculates the feature amount of the pre-disaster image in the small region with the highest correlation among the obtained correlation coefficient distributions. By doing in this way, even when the registration process is not performed with high accuracy, it is possible to suppress the influence of misalignment.

  As described above, the change extraction processing unit 142 estimates one of the two images (for example, the pre-disaster image and the post-disaster image) taken at different times by the positional deviation amount calculation processing unit 203. The processor 251 compares the feature amounts of the misaligned image and the other image (registered pair image), and extracts a change between the two images based on the comparison result.

  The contents of the operation of the cloud extraction processing unit 143 of the image processing apparatus 104, that is, the cloud extraction processing will be described below. A flowchart showing an outline of the processing is shown in FIG. As shown in FIG. 6, the cloud extraction processing unit 143 reads and inputs the registered pair image from the memory 252, and then performs a movement vector extraction process (step S 431), a movement amount calculation process (step S 432), and a quality index calculation process. (Step S433) is performed by the processor 251. Then, the cloud extraction processing unit 143 determines whether or not the object in the image is a cloud by using the obtained quality index and the movement amount as determination materials, and stores the cloud extracted image in the memory 252. At the same time, it is output by the output device 254. Details are shown below.

  First, the movement vector extraction processing unit 206 of the cloud extraction processing unit 143 performs a movement vector extraction process (step S431). This process is a process of extracting a cloud movement vector from a pair image. The specific procedure is as follows. First, the movement vector extraction processing unit 206 divides the post-disaster image into N × N small regions. Next, similarly to the matching process (step S412) described above, the movement vector extraction processing unit 206 performs matching between each small region and the pre-disaster image by the normalized correlation method of the above-described formula (3), Create a correlation coefficient distribution. Next, the movement vector extraction process part 206 extracts the position where a correlation coefficient becomes the maximum from each correlation coefficient distribution. Finally, the movement vector extraction processing unit 206 uses the position extracted from each correlation coefficient distribution, that is, the position in the pre-disaster image at the position where the correlation coefficient when matching is performed for each small region is the maximum value. The XY coordinates (start point) and the XY coordinates (end point) in the post-disaster image are obtained and used as the movement vector of the object in the image.

  Next, the movement amount processing unit 207 of the cloud extraction processing unit 143 performs a movement amount calculation process (step S432). This process is a process of calculating the magnitude of the movement vector obtained by the movement vector extraction process (step S431) described above. As a specific procedure, the movement amount processing unit 207 obtains the Euclidean distance from the start point to the end point of the movement vector, and sets it as the magnitude of the movement vector.

  Next, the quality index calculation processing unit 208 of the cloud extraction processing unit 143 performs a quality index calculation process (step S433). This process is a process of calculating an index for judging whether the quality of the movement vector obtained by the above-described movement vector extraction process (step S431) is good or bad. Various quality indexes can be considered, but here, the dispersion value, the edge amount, and the correlation coefficient are used as the quality indexes. The specific procedure is as follows. First, the quality index calculation processing unit 208 divides the post-disaster image into N × N small areas. Next, the quality index calculation processing unit 208 calculates the variance value and the edge amount in each small region from the above-described equations (1) and (2). Finally, the quality index calculation processing unit 208 performs matching between each small region and the pre-disaster image by the normalized correlation method of the above-described equation (3), similarly to the above-described matching processing (step S412). The maximum value is extracted from the correlation coefficient distribution. Note that the quality index calculation processing unit 208 may extract the maximum value from each correlation coefficient distribution using the correlation coefficient distribution created in the movement vector extraction process (step S431), or the movement vector. The maximum value of the correlation coefficient extracted from each correlation coefficient distribution in the extraction process (step S431) may be used as it is.

  Next, the cloud extraction processing unit 143 determines whether or not it is a cloud using the quality index and the movement amount as determination materials. The specific procedure is as follows. First, the cloud extraction processing unit 143 compares the quality index (dispersion value, edge amount, correlation coefficient) in one small region with each threshold value. Each threshold value is stored in advance in the memory 252 and is read out by the cloud extraction processing unit 143. Next, the cloud extraction processing unit 143 compares the movement amount with the threshold when all the comparison results are equal to or greater than the threshold. Next, the cloud extraction processing unit 143 determines that the small area is a cloud when the compared movement amount is equal to or greater than the threshold value. Thereafter, the cloud extraction processing unit 143 determines clouds for all the small regions. As described above, the determination rate of the cloud is determined only for the small region in which the attribute as the quality index is equal to or greater than a certain threshold value, so that the occurrence rate of the determination error can be suppressed. In addition, since it is assumed that there is no movement other than clouds in the image of the ground surface, it is further determined that only a small area whose movement amount is equal to or greater than a certain threshold is a cloud. Can be suppressed.

  Finally, the cloud extraction processing unit 143 images whether each small region is a cloud and creates a cloud extraction image. Various imaging methods are conceivable, and there are a method of binarizing and displaying the presence / absence of a cloud, a method of displaying by color coding according to the probability of being a cloud, and the like.

  As described above, in the cloud extraction processing unit 143, the positional deviation amount calculation processing unit 203 estimated one of two images taken at different times (for example, the pre-disaster image and the post-disaster image). An object that appears in a different position in the misaligned image and the other image (registered pair image) (for example, the maximum correlation coefficient when matching between the pre-disaster image and post-disaster image) When the distance (movement amount) between the different positions is equal to or greater than a predetermined threshold, it is determined that the object is a cloud.

  Hereinafter, the contents of the operation of the texture classification processing unit 144 of the image processing apparatus 104, that is, the texture classification processing will be described. A flowchart showing an outline of the processing is shown in FIG. As shown in FIG. 7, the texture classification processing unit 144 reads the registered image from the memory 252 and inputs it, and then performs an NDVI calculation process (step S441) and an HSI conversion process (step S442) by the processor 251. Then, the texture classification processing unit 144 classifies the texture (plant, water area, land, mountain surface, cloud) by the processor 251 using the obtained NDVI value and the HSI value as judgment materials, and stores the texture classification image in the memory 252. At the same time, it is output by the output device 254. Details are shown below.

  First, the NDVI calculation processing unit 209 of the texture classification processing unit 144 performs NDVI calculation processing (step S441). This process is a process of calculating a vegetation index by taking advantage of the feature that the green leaves of the plant absorb the wavelengths of the blue region and the red region and strongly reflect the wavelengths of the near infrared region. As a specific procedure, the NDVI calculation processing unit 209 calculates an NDVI value from the following equation (4).

  The NDVI value is obtained by the following equation.

  Next, the HSI conversion processing unit 210 of the texture classification processing unit 144 performs HSI conversion processing (step S442). In this process, an image having RGB (Red, Green, Blue) luminance values is converted into a developing system HSI space, and hue (Hue), saturation (Saturation), and brightness (Intensity) are calculated. is there. As a specific procedure, the HSI conversion processing unit 210 calculates brightness and saturation from the following formulas (5) and (6). Note that the hue is not calculated here because it is not used in subsequent processing.

  The brightness and saturation in the HSI conversion can be obtained by the following equations, respectively.

  Next, the texture classification processing unit 144 classifies the pixels into textures. This process is a process of classifying the obtained NDVI value and HSI value into any one of plants, water areas, land, mountains and clouds. The specific procedure is as follows. First, the texture classification processing unit 144 compares the NDVI value with a threshold value, and classifies the pixel as a plant when the result is equal to or greater than the threshold value. Next, when the pixel is not classified as a plant, the texture classification processing unit 144 compares the NDVI value with a threshold value, and classifies the pixel as a water area when the result is less than the threshold value. Next, when the pixel is not classified into the water area, the texture classification processing unit 144 compares the red wavelength value with a threshold value, and classifies the pixel as land when the result is less than the threshold value. Next, when the pixel is not classified as land, the texture classification processing unit 144 compares the S value and the I value with the respective threshold values, and when the S value is equal to or greater than the threshold value and the I value is less than the threshold value, Classify the pixel as a mountain surface. Finally, the texture classification processing unit 144 classifies pixels that have not been classified into the mountain surface into clouds. Thereafter, the texture classification processing unit 144 classifies all pixels into plants, water areas, land, mountains, or clouds. Each threshold value is stored in the memory 252 in advance, and is read by the texture classification processing unit 144.

  Finally, the texture classification processing unit 144 images the classification result and creates a texture classification image. Various imaging methods are conceivable, and there are a method of displaying in different colors according to the classification, a method of displaying different patterns according to the classification, and the like.

  As described above, the texture classification processing unit 144 estimates one of the two images (for example, the pre-disaster image and the post-disaster image) taken at different times by the positional deviation amount calculation processing unit 203. The processor 251 calculates the red wavelength value, the NDVI value, the S value, and the I value of each pixel of at least one of the image shifted in position, the other image, and the (registered pair image). Based on the wavelength value, the NDVI value, the S value, and the I value, it is determined whether each pixel is a plant, water area, land, mountain surface, or cloud.

  Hereinafter, the effect which this embodiment has will be described.

  According to the present embodiment, since the image processing apparatus 104 of the change extraction system 100 includes the registration processing unit 141 that performs registration processing of a pair image, a pair image having a shift in the captured position. It is possible to extract changes in between. Therefore, an effect of eliminating the need for manual work such as registering the pair images in advance can be obtained.

  Furthermore, according to the present embodiment, since the change extraction processing unit 142 included in the image processing apparatus 104 uses the variance value, the edge amount, and the correlation coefficient as the feature amount for change extraction, the pair image Even when the temporal imaging conditions such as season and time differ between the two, change extraction can be performed without being greatly influenced by the imaging conditions. Therefore, it is possible to obtain an effect of eliminating the need for manual work, such as performing correction so that one of the images matches the imaging condition of the other image in advance.

  Furthermore, according to the present embodiment, since the image processing apparatus 104 includes the cloud extraction processing unit 143 that performs cloud extraction processing from an image, even an image including a cloud can be changed without being greatly affected by the cloud extraction processing unit 143. Extraction is possible. Therefore, it is possible to obtain an effect of eliminating the need for manual work such as human interpretation of whether or not the cause of the change is due to the movement of the cloud after the change is extracted.

  Furthermore, according to the present embodiment, since the image processing apparatus 104 includes the texture classification processing unit 144 that performs texture classification processing, it is possible to automatically classify textures. Therefore, it is possible to obtain an effect of eliminating the need for manual work such as human interpretation of the place where the change has occurred after the change extraction.

1 is a block diagram illustrating an example of a configuration of a change extraction system according to Embodiment 1. FIG. 1 is a block diagram illustrating an example of a configuration of an image processing device according to a first embodiment. 2 is a diagram illustrating an example of a hardware configuration of an image processing apparatus according to Embodiment 1. FIG. 3 is a flowchart showing an outline of a registration process according to the first embodiment. 3 is a flowchart showing an outline of change extraction processing according to Embodiment 1; 4 is a flowchart illustrating an outline of cloud extraction processing according to the first embodiment. 5 is a flowchart showing an outline of texture classification processing according to the first embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 Change extraction system, 101 Geostationary optical satellite, 102 Satellite image receiving center, 103 Satellite image database, 104 Image processing apparatus, 105 Terminal apparatus, 106 Network, 141 Registration processing part, 142 Change extraction processing part, 143 Cloud extraction processing part , 144 texture classification processing unit, 201 matching small region selection processing unit, 202 matching processing unit, 203 misregistration amount calculation processing unit, 204 resampling processing unit, 205 feature amount calculation processing unit, 206 movement vector extraction processing unit, 207 movement Quantity processing unit, 208 quality index calculation processing unit, 209 NDVI calculation processing unit, 210 HSI conversion processing unit, 251 processor, 252 memory, 253 input device, 254 output device, 901 display device, 902 keyboard, 903 mouse , 904 FDD, 905 CDD, 906 Printer device, 911 CPU, 912 Bus, 913 ROM, 914 RAM, 915 communication board, 920 a magnetic disk device, 921 operating system, 922 Window system, 923 Program group, 924 File group.

Claims (8)

An image processing apparatus for storing two images taken at different times in a memory in advance,
Each time the two images are read from the memory, a first region of a predetermined number of pixels is moved by one pixel or more in one image, and the first region is moved, the first region And a matching processing unit that calculates a correlation coefficient between the second region in the same position as the first region in the other image by a processor,
A positional deviation amount calculation processing unit that estimates a positional deviation amount between the two images by a processor in units of less than one pixel based on the correlation coefficient calculated by the matching processing unit; apparatus. Each time the matching processing unit moves the first area, the position of the first area and the correlation coefficient between the first area and the second area are associated with each other and stored in the memory. Remember
The positional deviation amount calculation processing unit reads the position of the first region and the correlation coefficient from a memory, and calculates an approximate expression indicating the relationship between the position of the first region and the correlation coefficient by a processor. The image processing apparatus according to claim 1, wherein an amount of positional deviation between the two images is estimated in units of less than one pixel using the approximate expression. The image processing apparatus further includes:
A matching small region selection process in which the one image is divided into a plurality of regions each including a predetermined number of pixels, and the first region is selected based on at least one of pixel value dispersion and edge amount in each region. The image processing apparatus according to claim 1, further comprising a unit. The image processing apparatus further includes:
The processor compares a feature amount between the image obtained by shifting the positional deviation amount estimated by the positional deviation amount calculation processing unit with respect to the one image and the other image, and changes between the two images based on the comparison result. The image processing apparatus according to claim 1, further comprising: a change extraction processing unit that extracts the image. The two images are images of the ground surface taken from above,
The image processing apparatus further includes:
The processor estimates an object appearing at a different position between the image obtained by shifting the displacement amount estimated by the displacement amount calculation processing unit and the other image from the one image, and a distance between the different positions is predetermined. 5. The image processing apparatus according to claim 1, further comprising a cloud extraction processing unit that determines that the object is a cloud when the threshold is equal to or greater than a threshold value. The two images are images of the ground surface taken from above,
The image processing apparatus further includes:
A red wavelength value and an NDVI (Normalized / Difference / Vegetation / Index) value of each pixel of at least one of the image obtained by shifting the position shift amount estimated by the position shift amount calculation processing unit and the other image. And S (Saturation) value and I (Intensity) value are calculated by the processor, and based on the red wavelength value, NDVI value, S value and I value of each pixel, each pixel has a plant, water area, land and mountain surface. The image processing apparatus according to claim 1, further comprising a texture classification processing unit that determines whether the image is a cloud. An image processing method using an image processing apparatus that stores two images taken at different times in a memory in advance,
The matching processing unit of the image processing apparatus reads the two images from the memory, moves a first area including a predetermined number of pixels in one image in units of one pixel or more, and moves the first area to the first area. A first step of calculating by a processor a correlation coefficient between the first region and a second region at the same position as the first region in the other image each time it moves;
A second unit for estimating a positional shift amount between the two images by a processor in units of less than one pixel based on the correlation coefficient calculated in the first step; And an image processing method. An image processing program that is executed by a computer that stores two images taken at different times in a memory in advance,
Each time the two images are read from the memory, a first region of a predetermined number of pixels is moved by one pixel or more in one image, and the first region is moved, the first region And a matching process in which a processor calculates a correlation coefficient between the second image in the same position as the first region in the other image,
An image that causes a computer to execute a positional deviation amount calculation process in which a processor estimates a positional deviation amount between the two images in units of less than one pixel based on the correlation coefficient calculated by the matching process. Processing program.

Images