JP2000076423A - Image analyzer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make fast graspable the change with time of an object by comparing the images photographed on different dates with each other to automatically grasp the change with time. SOLUTION: An image information recording part 101 stores the image information on the satellite images, aerial photographs, etc., and a rotation correction part 102 corrects the angles of rotation of plural images stored in the part 101 to perform the differential analyses among these images. A parallel movement correction part 103 moves the images corrected at the part 102 in parallel to each other to correct the positions of these images on a two- dimentional plate. A differential image extraction part 104 extracts the differences among the images which undergone the position matching by the rotations and movements of them and rocords these images at a differential image recording part 105. A display control part 106 shows one or both of image information which are stored in both parts 101 and 105 on a display device 108 based on the input received from an input device 107. As a result, the change with time of an object can be fast grasped from among a large number of images.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image analyzing apparatus for analyzing a temporal change of a target building, land, road or the like from image information such as a satellite image and an aerial photograph.

[0002]

2. Description of the Related Art In recent years, the use of satellite and aerial photography has become widespread. In the creation and maintenance of maps, satellite and aerial photographs are widely used in order to efficiently explore vast areas. The use of aerial photographs and satellite photographs is used to acquire height information by stereo vision of aerial photographs and to grasp changes in land conditions by using photographs. It is also used for monitoring vegetation and land use over a wider area.

As such a technique, Japanese Patent Laid-Open Publication No.
In the "method of obtaining three-dimensional numerical data from stereo image data" disclosed in Japanese Patent Publication No. 78-78, a method for obtaining three-dimensional numerical data (height information) based on an image obtained from an artificial satellite or an aircraft is used. Techniques for finding corresponding points in both images have been proposed. In this technique, height information is obtained by finding feature points in left and right images and automatically associating them. For those feature points that cannot be automatically associated, interactive association is performed manually.

Further, in the technique disclosed in Japanese Patent Application Laid-Open No. 8-63683, a device for taking a difference between images in order to use an image from a television camera for a monitoring device has been proposed. In this device, since monitoring is performed on an image captured by one television camera, difference information is not simply obtained between time-series images. By performing the differential processing, the effect of the change of the sunshine at the monitoring target place is removed.

[0005]

As an application field of the aerial photograph and the satellite photograph, there is a conceivable use for grasping a land use situation and a change of a block situation. By using aerial photographs and satellite photographs, it is possible to monitor a wide area more efficiently as compared to grasping the land use situation on the ground. However, in the method of using images proposed in JP-A-8-63683 and JP-A-3-167678, images to be compared are shot at almost the same time.

On the other hand, the images to be compared in order to grasp the change in the use status of the land and the rebuilding of the building are images taken in a period of several months to several years. When comparing images having different shooting dates and times as described above, a difference in the shooting conditions of each image poses a problem.

[0007] The differences in the photographing conditions include, for example, a slight difference in the flight path of an airplane taking an aerial photograph, and a slight difference in attitude of a satellite taking a satellite image. Due to such a difference in photographing conditions, even images photographed in the same area do not completely overlap. If the two images are slightly displaced, a large density difference occurs at almost all points, and significant information cannot be obtained even if the two images are overlapped and the density difference for each pixel is extracted.

[0008] To correct such a shift between images, the ground is divided into several rectangular areas, and the affine transformation is performed in units of the rectangular area, whereby the effects of horizontal shift and rotation can be removed. . However, usually, when affine transformation is used, it is necessary to designate three points where correspondence can be obtained between images to be superimposed. Since these three points need to be manually specified, they are not suitable for mass processing.

The present invention solves such a problem, and provides an image analysis apparatus that can automatically superimpose images and can quickly grasp a temporal change of an object from a large number of images. The purpose is to do so.

[0010]

According to the present invention, in order to solve the above-mentioned problems, a contour line is extracted by differentiating each of a plurality of pieces of image information photographed at different times to generate a contour line image. Contour extraction processing means, frequency conversion processing means for two-dimensionally Fourier transforming the plurality of generated contour images to generate a Fourier transformed image, and a difference between respective pixel values of the generated plurality of Fourier transformed images. Rotation angle calculating means for obtaining a rotation angle that minimizes the sum of the above, and one original image is rotated on the two-dimensional plane by the calculated rotation angle, and the sum of the differences between the pixel values in the vertical and horizontal directions is minimized. Moving amount calculating means for obtaining a parallel moving amount, one of a plurality of pieces of image information photographed at different times is rotated on a two-dimensional plane by a rotation angle calculated by the rotation angle calculating means, and the moving amount calculating means so The plurality of pieces of image information are moved on the two-dimensional plane by moving by the given parallel movement amount, and a difference image is generated by calculating a difference between each pixel of the plurality of pieces of image information in a normal state of the positioning. It is characterized by comprising a difference image generation processing means, and an image output means for outputting the generated difference image as a visible image.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a system configuration diagram showing an embodiment of the present invention.
1, a rotation correction unit 102, a parallel movement correction unit 103, a difference image extraction unit 104, a difference image recording unit 105, a display control device 106, an input device 107, and a display device 108.

The image information recording section 101 stores image information such as satellite images and aerial photographs. Rotation correction unit 1
Numeral 02 performs a correction process of the rotation angle of the image in order to perform a difference analysis between a plurality of images stored in the image information recording unit 101. The parallel movement correction unit 103 performs parallel movement on the plurality of images corrected by the rotation correction unit 102 to correct the position on a two-dimensional plane. The difference image extracting unit 104 extracts a difference between the images after the alignment by rotation and translation, and records the difference in the difference image recording unit 205. The display control unit 106 displays one or both of the image information stored in the image information recording unit 101 and the difference information recording unit 105 on the display device 108 according to the input from the input device 107.

First, the principle of correcting the rotational displacement and the displacement in the vertical and horizontal directions of two images according to the present invention will be described.
This will be described with reference to FIG. FIGS. 2A and 2B are aerial photograph images of the same spot. (C) and (d) are the figure which expanded and showed the-part of the image of (a) and (b), respectively. The rectangles in (c) and (d) represent buildings that appear in the images in (a) and (b), respectively. In this example, the images (a) and (b) are obtained by photographing the same point from the sky. However, since the directions at the time of photographing are different, it is not possible to extract the difference by superimposing the two images as they are.

An object of the present invention is to extract changes in an artificial building, for example. Artifacts such as buildings and roads have more geometric regularities than forests and wilderness. Focusing on this regularity, the present invention differentiates the images schematically shown in FIGS. 2 (c) and 2 (d) to extract the contour lines, and shows the images schematically in FIGS. 2 (e) and 2 (f). Is generated.

Next, the objects such as buildings included in the contour images (e) and (f) are often arranged regularly along roads and the like. To extract this feature,
The two-dimensional Fourier transform processing is performed on the images of (e) and (f). FIGS. 2G and 2H are Fourier transform images obtained by performing a two-dimensional Fourier transform on the images of FIGS. 2E and 2F and plotting the absolute values of the data at each pixel point. Since many straight lines such as roads and house frames appear parallel or perpendicular to the original image, dark components appear in the direction inclined by 90 degrees in the Fourier transform images (g) and (h). . Therefore, a rotation angle that best associates the images of (g) and (h) is obtained. Specifically, the rotation angle at which the sum of the differences between the pixel values of the Fourier transform images (g) and (h) is minimized is determined. Then, rotation correction is performed on the image (b) at that angle, so that the rotation directions of the images (a) and (b) can be associated (aligned). Thereafter, one original image is rotated on the two-dimensional plane by the calculated rotation angle, and a translation amount that minimizes the sum of the differences between the pixel values in the vertical and horizontal directions is obtained.

Then, the image (b) is translated in parallel by the calculated amount of translation so that the images (a) and (b) can be associated (positioned) in the vertical and horizontal directions. Note that the translation amount may be calculated immediately after the rotation angle is obtained. When both the rotation angle and the translation amount are obtained, the image in (b) is rotated and further translated. You may.

FIG. 3 is a schematic processing flowchart up to displaying difference information between two images. In the image analysis apparatus according to the present embodiment, first of all, the two images stored in the image information recording unit 101 are corrected by a rotation angle correction process to correct a shift due to rotation between the images (step 301). Next, the displacement between the images in the horizontal and vertical directions is corrected by the parallel movement correction processing (step 302). The horizontal and vertical directions correspond to the horizontal and vertical directions in the drawing of FIG. After these correction processes, a difference information extraction process is performed, and a difference image is generated by calculating a difference between corresponding pixels of the two images (the images of (a) and (b) of FIG. 2). It is stored in the recording unit 105 (step 303). This difference image is displayed as a visible image on the display device 108 by the extraction result display processing (step 304).

FIG. 4 is a flowchart showing details of the rotation angle correction processing (step 201) shown in the processing flow of FIG. In the rotation angle correction process, first, in the sample area extraction process, for example, “64
A certain number of areas of "* 64" pixels are taken out (step 4).
01). Next, in the image edge extraction processing, edge extraction (contour extraction) is performed on the image extracted in the sample area extraction processing (step 401) to generate a contour image (step 402). Next, a two-dimensional Fourier transform process (frequency transform process) is performed on the contour image to acquire a frequency component (step 403). Next, a rotation angle estimating process for detecting the rotation angle of the two images by performing a small angle rotation of the Fourier transformed image after the frequency conversion process to find an angle at which the difference between the two images is minimized ( Step 404). The rotation angle is corrected by rotating one of the original images (the image in FIG. 2B) at the rotation angle obtained by this processing.

FIG. 5 is a flowchart showing details of the sample area extracting process (step 401) in the process flow of FIG. First, the density values of the pixels corresponding to the (X, Y) coordinates of the original image (the image of (a) or (b) in FIG. 2) are stored in arrays A1 [X, Y] and A2 [X, Y]. Suppose The density value of the pixel at the (X, Y) coordinate of the extracted image of the I-th sample area is represented by an array B1 [I, X,
Y], B2 [I, X, Y]. First, 1 is substituted for a variable I (step 501). next,
It is determined whether the value of the variable I does not exceed the number N of sample areas to be extracted (step 502). If the variable I is larger than N, the process ends.

If the variable I is equal to or smaller than N, an integer random number in the range of 0 to "the number of horizontal pixels of the image-64" is generated and assigned to the variable X (step 503). Further, an integer random number ranging from 0 to “the number of vertical pixels of the image−64” is generated and assigned to the variable Y (step 504).

Next, 1 is substituted for the variable J (step 5).
05). Next, it is determined whether the value of the variable J is “64” or less (step 506). If it is less than "64", "1" is substituted for the variable K (step 507). Next, it is determined whether the variable K is “64” or less (step 508). When the value of the variable K is equal to or less than “64”, (X
(+ J, Y + K) pixel values are stored in arrays B1 [I, J, K] and B2 [I, J, K] (steps 509, 51).
0). Next, "1" is added to the value of the variable K (step 5).
11). Thereafter, the process returns to step 508 again to determine the value of K. If K exceeds “64” in this determination, “1” is added to the value of the variable J (step 51).
2). Then, the process returns to step 506 to determine whether J exceeds “64”. If it exceeds "64", "1" is added to the value of the variable I (step 51).
3) Returning to step 502, it is determined whether the value of I exceeds N.

FIG. 6 is a flowchart showing details of the image edge extraction processing (step 402) of FIG. First, "1" is substituted for the variable I (step 601). If the value of the variable I exceeds the number N of the sample areas, the process ends (step 602). If not, "1" is substituted for the variable X (step 603). Next, it is determined whether the value of the variable X has exceeded “64” (step 604). If so, "1" is added to the value of the variable I (step 611), and the process returns to step 602 to determine whether the value of I exceeds N. If not exceeded, "1" is substituted for the variable Y (step 60).
5). Next, it is determined whether or not the value of the variable Y exceeds “64” (step 606). If so, "1" is added to the value of the variable X (step 610), and the process returns to step 604 to determine whether the value of X exceeds "64".

If the value of the variable Y does not exceed "64", the left and right and upper and lower pixels (X-1, Y), (X + 1, Y), centered on the (X, Y) pixel of one image, (X, Y +
1), a value four times (= 4 *) the difference between the density values of (X, Y-1) is assigned to the array C1 [I. X, Y] (step 60).
7). Similarly, left and right and upper and lower pixels (X-1, Y), (X + 1, Y) centered on the (X, Y) pixel of the other image,
A value four times (= 4 *) the difference between the density values of (X, Y + 1) and (X, Y-1) is stored in the array C1 [I, X, Y] (step 608). Then, "1" is added to the variable Y (step 609), and the process returns to step 606 to return to Y
It is determined whether or not the value of “?” Does not exceed “64”. Thus, a contour image of the two images is generated.

FIG. 7 is a flowchart showing details of the rotation angle estimation processing (step 404) of FIG. By the image edge extraction processing shown in FIG. 6, the contour images of a plurality of sample areas extracted from the two images are arranged in an array C1 [I, X,
Y] and C2 [I, X, Y]. These two images are subjected to two-dimensional Fourier transform processing in FIG. 4 so that their power spectra are arranged in arrays P1 [I, X, Y] and P2.
[I, X, Y] is stored.

In the rotation angle estimation processing, an optimum rotation angle is calculated for each sample area, and the average value is used as the rotation angle of the entire image. This processing will be described with reference to the flowcharts of FIGS. First, "1" is substituted for the variable I (step 701). Next, it is determined whether or not the variable I exceeds the number N of sample areas (step 702). If the variable I exceeds N, the value of the variable S is divided by N, the average rotation angle is determined, and the process is terminated (step 72 in FIG. 8).
1). However, when the variable I is N or less, first, the variable S
The total difference value of the images of the two sample areas to be compared is substituted into [I]. "-10" is substituted for the variable L [I] (step 703).

Next, the variable J "-9" is substituted (step 704). Next, it is determined whether this variable J is less than “10” (step 705). When the variable J is less than “10”, the image stored in P [I, X, Y] is converted to the minute angle θ.
Is stored in Q1 [X, Y] (step 706). Next, the sum of the differences for each pixel between the rotated image Q1 [X, Y] and the non-rotated image P [I, X, Y] is calculated as a variable D in the following procedure.

First, a value "0" is substituted for a variable D (step 707). Next, "1" is substituted for the variable X (step 708). It is determined whether this variable X does not exceed “64” (step 709). If the variable X is equal to or less than "64", "1" is substituted for the variable Y (step 71).
0). Next, it is determined whether or not this variable Y exceeds “64” (step 711). If it does not exceed “64”, (X−32) {2+ (Y−32)} 2 is 32}
It is determined whether the number does not exceed 2 (step 712). “$ 2” means “square”. If not, the variables D are Ql [X, Y] and P2 [I,
[X, Y] is added (step 713).

Next, "1" is added to the variable Y (step 7).
14) The value is determined again in step 711. If Y exceeds “64” in this determination, the value of the variable X is “1”.
Is added (step 714), and the value of X is determined again in step 709. If the value of X exceeds “64” in this determination, it is determined whether or not the total difference value when rotating by J * θ (= J times θ) is the smallest among the calculated values so far. Therefore, it is determined whether the variable D is equal to or less than S [I] (step 71).
6). If D is small, the value of D is substituted for the variable S [I], and the value of J is substituted for the variable L [I] (step 71).
7).

Next, in order to perform the same processing on the image further rotated by θ, “1” is added to the value of the variable J (step 718), and the value of the variable J is determined again in step 705. If the value of the variable J is “10” or more, the rotation angle of the I-th image is added to the variable S (step 719), and then “1” is added to the variable I (step 720). If the value of the variable I exceeds the number of sample areas, the value of the variable S is divided by the number N of sample areas to determine the rotation angle of the entire image (step 721), and the process ends. .

As described above, in the present embodiment, a plurality of pieces of image information photographed at different times are differentiated (edge-processed) to extract a contour, and a contour image is generated. Each of multiple contour images
A four-dimensional Fourier transform is performed to generate a Fourier-transformed image. Further, a rotation angle at which the sum of the differences between the pixel values of the plurality of generated Fourier-transformed images is minimized is determined. Is rotated on a two-dimensional plane, the amount of translation that minimizes the sum of the differences between the pixel values in the vertical and horizontal directions (horizontal and vertical directions) is obtained, and one of a plurality of pieces of image information captured at different times is calculated. The image information is rotated on the two-dimensional plane by the rotation angle calculated in the processing, and is moved by the parallel movement amount calculated in the movement amount calculation processing, so that a plurality of pieces of image information are aligned on the two-dimensional plane. Conventionally, a difference image is generated by calculating a difference between each pixel of a plurality of pieces of image information, and the generated difference image is output as a visible image to a display device or a printer. The superposition of which was the image can be automated, the temporal change of the object it is possible to grasp at high speed from a large number of images.

Therefore, it is extremely important to quickly grasp the temporal change of the photographed object (road, building, land, forest, river, etc.) from images such as satellite images and aerial photographs taken at different times. It will be effective.

The present invention is not limited to a satellite image or an aerial photograph. For example, a camera installed at a ground monitoring point of a large-scale parking lot can use the parking lot from the sky at appropriate time intervals. The present invention can also be applied to a case where a photograph is taken and a temporal change such as a use state of a parking lot is grasped from the photographed image.

[0033]

As described above, according to the present invention,
Compare images such as aerial photos and satellite photos taken at different dates and times to understand the temporal use of land, such as roads and buildings, and changes in building and road conditions over time. Can be automated, erroneous difference extraction due to a shift in the photographing position can be prevented, and the temporal change of the object can be quickly grasped from a large number of images. Therefore, it is extremely effective in quickly grasping temporal changes of objects (roads, buildings, land, forests, rivers, lakes, etc.) taken from images such as satellite images and aerial photographs taken at different times. It becomes something.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory view including a photograph showing a change of an image in an image analysis process of the present invention.

FIG. 3 is a flowchart showing an outline of an image analysis process of the present invention.

FIG. 4 is a detailed flowchart of a rotation angle correction process in the flowchart of FIG. 3;

FIG. 5 is a flowchart showing details of a sample area extraction process in the flowchart of FIG. 4;

FIG. 6 is a flowchart showing details of edge extraction processing in the flowchart of FIG. 4;

FIG. 7 is a flowchart showing details of a rotation angle correction process in the flowchart of FIG. 4;

FIG. 8 is a flowchart showing a continuation of FIG. 7;

[Description of sign]

101: Image information recording unit, 102: Rotation correction unit, 103
... a parallel movement correction unit, 104 ... a difference image extraction unit, 105 ...
Difference image recording unit, 106: display control device, 107: input device, 108: display device.

 ────────────────────────────────────────────────── ─── Continued on the front page F term (reference) 5B057 AA14 CA08 CA12 CA16 CB08 CB12 CB16 CC03 CE03 DA07 DB02 DB09 DC16 DC32 5L096 AA06 BA18 DA01 FA06 FA23 GA08 GA24 HA01 JA11

Claims (1)

[Claims] An image for analyzing a temporal change of an object photographed in a plurality of images using a plurality of pieces of image information of a range including a target object photographed from above at different times. An analysis apparatus, wherein a plurality of pieces of image information photographed at different times are subjected to differential processing to extract a contour line, and a contour line extraction processing means for generating a contour image, and the generated plurality of contour line images are provided. Frequency conversion processing means for performing a two-dimensional Fourier transform to generate a Fourier transform image, respectively; rotation angle calculating means for determining a rotation angle that minimizes the sum of the differences between the pixel values of the plurality of generated Fourier transform images; Moving amount calculating means for rotating one original image on the two-dimensional plane by the calculated rotation angle and obtaining a parallel moving amount that minimizes the sum of the differences between the pixel values in the vertical and horizontal directions; One of the plurality of shaded image information is rotated on the two-dimensional plane by the rotation angle calculated by the rotation angle calculation means, and is moved by the parallel movement amount calculated by the movement amount calculation means to convert the plurality of image information. A difference image generation processing means for obtaining a difference between each pixel of a plurality of pieces of image information and generating a difference image in a state where the registration is performed on a two-dimensional plane, and the generated difference image is visible An image analysis device comprising: an image output unit that outputs an image.