JP5919963B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  Conventionally, a technique for comparing and collating a plurality of images obtained by photographing the same point at different times has been used. For example, as processing using an aerial photograph image, the old photographed aerial photograph image and the new photographed aerial photograph image are collated, and feature points are extracted from each image. Among the feature points of each image, corresponding feature points are extracted as feature point pairs. Then, the two feature points constituting the feature point pair are associated with each other, and the two images are aligned. By comparing both images after alignment, a change occurring at the imaging target point from the previous imaging time point to the subsequent imaging time point can be extracted as a difference.

  In the feature point pair extraction, for example, an image feature and a tilt direction are calculated for each feature point extracted from two images. The similarity between feature points is calculated based on the calculated image feature and tilt direction, and feature points with high similarity are extracted as a pair.

  In addition, an image analysis apparatus has also been proposed for quickly grasping temporal changes in land use status from a plurality of aerial photographs having different shooting dates and times. In this image analysis apparatus, a plurality of pieces of image information are differentiated to obtain a contour image. Then, a Fourier transform image is generated from the acquired image, and the rotation angle at which the total difference of the pixel values is minimized is obtained. After rotating one image at the obtained rotation angle, a translation amount that minimizes the sum of differences between the vertical and horizontal pixel values is obtained. The two images are aligned by moving one image by the obtained movement amount. A difference image is generated by obtaining a difference between the two images after alignment.

  Furthermore, a change extraction apparatus and method for extracting changes from a plurality of images taken at different shooting positions, illumination conditions, angles, and scales have been proposed. In this method, changes are extracted by extracting and comparing footprint information from each image. Footprint information is any one of geometric features (such as contours and internal area information), image features (color or edge features, etc.), and spatial features (spatial location information of features, positional relationships between adjacent features, etc.) Or a combination thereof.

JP-A-2005-173128 JP 2000-76423 A JP 2006-65429 A

  However, when two images are compared and collated using the conventional method, a difference may be erroneously detected from the two images due to a shift or the like in the two images. For example, when two images are taken from different shooting angles or shooting directions, even the same object may be seen differently. In addition, when each of the two images has a shift in the image and the shift is generated in a different part, the two images are compared and collated, based on differences in pattern, brightness, color difference, etc. If the difference is extracted, erroneous detection may occur. As described above, when the difference is detected based on the difference in the pattern, the brightness, the color difference, or the like using the conventional technique, the difference is erroneously detected.

  The disclosed technology has been made in view of the above, and an object thereof is to provide an image processing device, an image processing method, and an image processing program capable of suppressing erroneous detection of a difference between two images.

  In order to solve the above-described problems and achieve the object, the disclosed technique extracts feature points from each of the first image and the second image. Further, for each feature point of the first and second images, a first direction is determined based on the feature point. Further, the second direction is determined based on the first direction. Then, an elliptical matrix having the first direction and the second direction as axes and centering on the feature points is created. A feature amount matrix is created by associating each area of the elliptical matrix with information regarding the difference between the luminance value of the pixel in the area and the luminance value of the feature point. Then, the feature quantity matrix of the feature points of the first image and the feature quantity matrix of the feature points of the second image are compared to determine similar feature point pairs. Based on the shape of the feature quantity matrix of the feature points of the second image used for determining the feature point pair, the second image including the feature points is converted, and the first image converted to the first image is converted. The difference is detected based on the second image.

  The disclosed image processing apparatus, image processing method, and image processing program have an effect of suppressing erroneous detection of a difference between two images.

FIG. 1 is a diagram illustrating an example of the configuration of the image processing server according to the first embodiment. FIG. 2 is a diagram illustrating an example of image information stored in the image storage unit. FIG. 3 is a diagram illustrating an example of feature point information stored in the feature point storage unit. FIG. 4 is a diagram illustrating an example of the configuration of the feature amount matrix information stored in the first matrix storage unit. FIG. 5 is a diagram illustrating an example of a configuration of feature amount matrix information stored in the second matrix storage unit. FIG. 6 is a diagram for explaining an example of the configuration of the feature amount matrix. FIG. 7 is a diagram illustrating an example of feature point pair information stored in the pair storage unit. FIG. 8 is a diagram illustrating an example of a configuration of information stored in the rotation angle storage unit. FIG. 9 is a diagram illustrating an example of a configuration of information stored in the enlargement / reduction ratio storage unit. FIG. 10 is a diagram illustrating an example of a configuration of information stored in the difference storage unit. FIG. 11 is a flowchart illustrating a processing procedure of image processing performed by the image processing server according to the first embodiment. FIG. 12 is a diagram for explaining an example of a feature amount matrix creation process. FIG. 13A is a diagram for explaining that the feature point closest to the attention point is used in the feature amount matrix creation process. FIG. 13B is another diagram for explaining the use of the feature point closest to the attention point in the feature quantity matrix creation process. FIG. 13C is still another diagram for explaining the use of the feature point closest to the attention point in the feature amount matrix creation process. FIG. 14 is a diagram for explaining the feature quantity matrix enlargement / reduction processing. FIG. 15 is a diagram for further explaining the reduction ratio in the enlargement / reduction ratio. FIG. 16A is a diagram for explaining an enlargement rate in enlargement / reduction processing. FIG. 16B is another diagram for explaining the enlargement ratio in the enlargement / reduction process. FIG. 17A is a diagram for explaining the rotation processing of the feature amount matrix. FIG. 17B is another diagram for explaining the rotation process of the feature amount matrix. FIG. 17C is still another diagram for explaining the rotation processing of the feature amount matrix. FIG. 18A is a diagram for explaining a method of determining an angle change range. FIG. 18-2 is another diagram for explaining the method of determining the angle change range. FIG. 18C is still another diagram for explaining a method of determining the angle change range. FIG. 18D is still another diagram for explaining a method of determining the angle change range. FIG. 18-5 is still another diagram for explaining the method of determining the angle change range. FIG. 18-6 is still another diagram for explaining a method of determining the angle change range. FIG. 18-7 is a diagram illustrating an example of the angle change range. FIG. 19A is a diagram for explaining a feature amount matrix creation process that has undergone a rotation process. FIG. 19B is another diagram for explaining the feature matrix creation processing that has undergone the rotation processing. FIG. 19C is still another diagram for explaining a feature amount matrix creation process that has undergone a rotation process. FIG. 19-4 is still another diagram for explaining the feature matrix creation processing that has undergone the rotation processing. FIG. 19-5 is still another diagram for explaining the feature amount matrix creation processing that has undergone the rotation processing. FIG. 19-6 is still another diagram for explaining the feature matrix creation processing that has undergone the rotation processing. FIG. 20 is a diagram illustrating an example of the configuration of the detection unit. FIG. 21A is a diagram for explaining an example of a homography matrix calculation method. FIG. 21B is another diagram for explaining an example of a homography matrix calculation method. FIG. 22-1 is a diagram for explaining the detection process. FIG. 22-2 is another diagram for explaining the detection process. FIG. 23A is a diagram for explaining an image shift that is not eliminated by alignment. FIG. 23B is another diagram for explaining the image shift that is not eliminated by the alignment. FIG. 24 is a flowchart illustrating an example of a processing procedure of detection processing according to the first embodiment. FIG. 25 is a schematic diagram illustrating an example of a computer that executes an image processing program according to the first and second embodiments.

  Embodiments of the disclosed image processing apparatus, image processing method, and image processing program will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  An image processing server 100 will be described as a first embodiment of the image processing apparatus disclosed in the present application (see FIG. 1). For example, the image processing server 100 receives input of two images obtained by capturing the same point from the client 200 at different times. Alternatively, the image processing server 100 receives an instruction from the client 200 to specify two images taken at the same point at different times. Then, the image processing server 100 extracts feature points from each of the two images. Further, the image processing server 100 uses an elliptical matrix centered on the feature point as information on the difference between the brightness value of each feature point and the brightness value of a pixel located within a predetermined distance range from the feature point. A feature amount matrix of each feature point is created in association with each of the plurality of regions. The image processing server 100 compares the feature quantity matrix of the feature points of one image with the feature quantity matrix of the feature points of another image to obtain the similarity. The image processing server 100 determines a feature point pair having a high similarity in the feature quantity matrix as a feature point pair.

  Further, the image processing server 100 according to the first embodiment performs alignment of the two images based on the determined feature point pair, and then extracts a difference area between the two images. And in order to exclude the difference area | region which does not actually correspond to a difference, a detection process is performed.

[Image Processing Server of First Embodiment]
First, an example of the configuration of the image processing server 100 will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the configuration of the image processing server 100 according to the first embodiment. The image processing server 100 includes a storage unit 110, a control unit 120, and an input / output unit 130. The storage unit 110 stores data used for processing in the image processing server 100 and data generated by processing in the image processing server 100. The control unit 120 controls image processing in the image processing server 100. The input / output unit 130 receives data input to the image processing server 100 from the outside, and transmits data generated by the image processing server 100 to the outside.

  The image processing server 100 is connected to the client 200 via the network 300. The client 200 includes a monitor 201, for example. An image processed in the image processing server 100 can be sent to the client 200 via the input / output unit 130 and displayed on the monitor 201. In FIG. 1, one client 200 and the monitor 201 are provided outside the image processing server 100, but the image processing server 100 may be connected to a plurality of clients 200 via the network 300. Then, the image processing server 100 may be configured to receive images transmitted from a plurality of clients 200. Further, a monitor may be provided in the image processing server 100 so that an image can be confirmed. Further, the image processing server 100 itself may be provided with an imaging function, and the image processing server 100 may be connected to an imaging device different from the client 200. The client 200 is, for example, a computer such as a personal computer, a mobile phone, a smartphone, or an Internet TV.

[Configuration of storage unit]
The storage unit 110 includes an image storage unit 111, a feature point storage unit 112, a matrix storage unit 113, a pair storage unit 114, a setting storage unit 115, and a difference storage unit 116. The storage unit 110 is a semiconductor memory element, a storage device, or the like. For example, examples of the semiconductor memory element include a video random access memory (VRAM), a random access memory (RAM), a read only memory (ROM), and a flash memory. Examples of the storage device include storage devices such as a hard disk and an optical disk. Hereinafter, information stored in the storage unit 110 will be described.

[Image information]
The image storage unit 111 stores image information received by the image processing server 100. FIG. 2 is a diagram illustrating an example of a configuration of image information stored in the image storage unit 111. For example, as shown in FIG. 2, the image information includes an image ID (Identifier) that uniquely identifies the image. Then, the image storage unit 111 stores the location where the image was captured, the shooting date and time, the image itself, and the like in association with the image ID. In the example illustrated in FIG. 2, the latitude and longitude of the shooting point of the image with the image ID “001” are stored as “35.6687, 139.74” in association with the image ID “001”. Furthermore, the shooting date “2012/03/30” is stored in association with the image ID “001”. Further, the image itself is stored in association with the image ID “001”.

[Feature information]
The feature point storage unit 112 stores information regarding the feature points of the image received by the image processing server 100. FIG. 3 is a diagram illustrating an example of a configuration of feature point information stored in the feature point storage unit 112.

  Here, the feature point refers to a point showing a remarkable feature in the image. For example, a part having a large luminance gradient with surrounding pixels such as a part corresponding to a corner of an object in the image and a part corresponding to a boundary line is extracted as a feature point. Although feature point extraction will be described later, the extraction method is not particularly limited.

As illustrated in FIG. 3, the feature point storage unit 112 stores information on feature points extracted from the image in association with the image ID “001”, for example. In the example of FIG. 3, a feature point ID “FP001” that uniquely identifies a feature point is stored in association with the image ID “001”. Further, it is stored that the position of the feature point specified by the feature point ID “FP001” is the coordinates “x ₁ , y ₁ ” on the image. A plurality of feature points are extracted for each image, and information about each is stored in the feature point storage unit 112.

[Feature matrix information]
The matrix storage unit 113 stores information on the feature amount matrix of each feature point. FIG. 4 is a diagram illustrating an example of the configuration of the feature amount matrix information stored in the first matrix storage unit 113a. FIG. 5 is a diagram illustrating an example of the configuration of the feature amount matrix information stored in the second matrix storage unit 113b. FIG. 6 is a diagram for explaining an example of the configuration of the feature amount matrix of the feature points of the comparison image. The feature quantity matrix information will be described with reference to FIGS.

  The matrix storage unit 113 includes a first matrix storage unit 113a and a second matrix storage unit 113b (see FIG. 1). The first matrix storage unit 113a stores a feature quantity matrix of feature points of an image to be compared (hereinafter also referred to as “comparison image” as appropriate) out of two images to be collated. The second matrix storage unit 113b stores a feature quantity matrix of feature points of an image to be processed (hereinafter also referred to as “processed image” as appropriate) that is not a comparison image among the two images to be collated. The first matrix storage unit 113a stores one feature amount matrix for each feature point of the comparison image. On the other hand, the second matrix storage unit 113b stores a plurality of feature amount matrices for each feature point of the processed image.

  As shown in FIG. 4, the feature amount matrix information about the feature points of the comparison image includes “image ID”, “feature point ID”, “matrix ID”, “matrix segment ID”, “luminance value”, “luminance difference”. And “luminance difference level”. The feature amount matrix information is information about the feature amount matrix of each feature point. For example, in the example of FIG. 4, information about the feature amount matrix of “matrix ID: M00101” is stored for the feature point of “feature point ID: FP001” of the image of “image ID: 001”.

  On the other hand, as shown in FIG. 5, the feature amount matrix information about the feature points of the processed image includes the following information in addition to the information stored about the feature points of the comparison image. That is, “rotation angle” and “enlargement / reduction ratio”. In addition, a plurality of feature quantity matrices for feature points of the processed image are created for one feature point. In the example of FIG. 5, a plurality of feature quantity matrices such as matrix IDs “M00101”, “M00102”, “M00103”, etc. are stored for the feature point with the feature point ID “FP001”. The matrix ID “M00101” is associated with “rotation angle: 0 °” and “enlargement / reduction ratio 1.0”.

  Details of the feature amount matrix will be further described. The feature amount matrix is generated by the creation unit 124 described later. As shown in FIG. 6, the feature amount matrix is obtained by dividing an ellipse having an axis having a predetermined length with a feature point as the center into a predetermined number of regions in the circumferential direction and the radial direction. For example, the creation unit 124 determines the first direction based on the feature points. Then, the creation unit 124 determines the second direction based on the first direction. The creation unit 124 creates an elliptical matrix with the feature point as the center, with the determined first direction and second direction as axes. For example, the creation unit 124 sets the direction of the feature point closest to the feature point as the first direction. Then, the creation unit 124 sets the direction at an angle of 90 degrees with respect to the line segment connecting the feature point and the closest feature point as the second direction. The creation unit 124 associates the difference between the brightness value of the feature point and the brightness value of the area of the matrix with the feature amount matrix created in this way. The difference between the luminance value of the pixel included in each region and the luminance value of the feature point located at the center of the concentric circle is the “luminance difference” value of each region. The example of FIG. 6 shows a total of 49 pixels. For convenience of explanation, the 49 pixels will be described below by specifying “row numbers and column numbers”. Row numbers are indicated by “1” to “7” from bottom to top, and column numbers are indicated by “1” to “7” from left to right.

  In the example of FIG. 6, the creation unit 124 sets a feature amount matrix that covers a region of three pixels in the major axis direction from the pixel position where the feature point exists with the feature point FP as the center. In the example of FIG. 6, the ellipse constituting the feature amount matrix is divided into two regions in the radial direction and four regions in the circumferential direction, and a total of eight regions are formed. Each area is given a matrix segment ID for uniquely specifying the area. In the example of FIG. 6, matrix segment IDs are assigned in order from the inner side to the outer side in the clockwise direction from the upper center of the ellipse. For example, matrix segment IDs “1” to “4” are assigned clockwise to the four regions included in the center-side ellipse. Matrix segment IDs “5” to “8” are assigned to the four regions included in the outer ellipse clockwise from the upper center.

  When the region of the feature amount matrix is set as described above, one pixel may exist over two or more regions. Here, the creation unit 124 calculates the feature amount by placing the pixel extending over the two regions in the right and lower regions. For example, the pixel “5, 4” in FIG. 6 exists over four regions of matrix segment IDs “1”, “4”, “5”, and “8”. The right area is an area with matrix segment IDs “1” and “5”, and the lower area is an area with matrix segment ID “1”. Therefore, the creation unit 124 calculates the feature amount by putting the pixels “5, 4” in the region of the matrix segment ID “1”.

  In this way, the creation unit 124 calculates the feature amount for each region illustrated in FIG. For example, pixels “5, 4”, “5, 5” are placed in the area of the matrix segment ID “1”. Therefore, the luminance value “(100 + 100) / 2 = 100” is the luminance value of the area of the matrix segment ID “1”. The luminance value of the pixel in the region where the feature point exists is “255”. Therefore, “255-100 = 155” is the value of the luminance difference of the area of the matrix segment ID “1”.

  The area of matrix segment ID “2” includes pixels “4, 4” and pixels “4, 5”. Therefore, the luminance value “(255 + 255) / 2 = 255” is the luminance value of the area of the matrix segment ID “2”. Therefore, “255−255 = 0” is the value of the luminance difference of the area of the matrix segment ID “2”.

  Similarly, when the luminance difference is obtained for the matrix segment IDs “3” to “8” shown in FIG. 6, finally obtained numerical values are as shown in FIG. Thus, the creation unit 124 obtains the luminance difference of each matrix segment for each feature point, and the matrix storage unit 113 stores it as feature amount matrix information.

  In addition, the matrix storage unit 113 divides the value of −255 to 255, which is a possible range of the brightness difference value, into, for example, 13 levels, and stores which level the brightness difference corresponds to. For example, P1 when the luminance difference is “−255 to −215”, P2 when “−215 to −175”, P3 when “−175 to −135”, and “−135 to −95”. Is P4. Further, the brightness difference is “−95 to −55”, P5, “−55 to −15” is P6, “−15 to 15” is P7, “15 to 55” is P8, The case of “55 to 95” is P9, and the case of “95 to 135” is P10. Furthermore, P11 is set when the luminance difference is “135 to 175”, P12 is set when “175 to 215” is set, and P13 is set when “215 to 255” is set. In this case, as shown in FIG. 4, the luminance difference level is “P11” for the area of the matrix segment ID “1” because the luminance difference is “155”. Note that the present invention is not limited to this, and only the luminance difference value may be stored.

[Feature Point Pair Information]
The pair storage unit 114 stores information on feature point pairs determined by the processing by the image processing server 100. FIG. 7 is a diagram illustrating an example of a configuration of feature point pair information stored in the pair storage unit 114.

  As illustrated in FIG. 7, the feature point pair information includes a “processed image ID” that uniquely identifies a processed image and a “comparison image ID” that uniquely identifies a comparative image. Further, a “pair ID” that uniquely identifies each feature point pair is stored in association with the “processed image ID” and the “comparison image ID”. Then, in association with the “pair ID”, a “processed image feature point ID” that uniquely identifies the feature point of the processed image and a “comparison image feature point ID” that uniquely identifies the feature point of the comparison image are stored. To do. For example, in the example of FIG. 7, pair IDs “001”, “002”, “003”, and the like are stored in association with “processed image ID: 001” and “comparison image ID: 201”. Then, “processed image feature point ID: FP001” and “comparison image feature point ID: FP201” are stored in association with the pair ID “001”. Thus, the feature point pair information is information indicating which feature points of the two images correspond to each other.

[Information stored in the setting storage unit]
The image processing server 100 according to the first embodiment generates a plurality of feature amount matrices by scaling up and down a matrix to be applied at a predetermined ratio with respect to a feature point of one of two images (processed image). Further, the image processing server 100 rotates one of the axes of the plurality of generated elliptical feature amount matrices at a predetermined rotation angle around the feature point, and further generates a feature amount matrix. In order to realize such processing, a setting storage unit 115 is provided in the storage unit 110 of the image processing server 100 according to the first embodiment. For example, the setting storage unit 115 includes a rotation angle storage unit 115a and an enlargement / reduction rate storage unit 115b. Note that the information stored in the setting storage unit 115 may be stored based on a user input, or may be stored based on the results of processing by the rotation unit 124a and the enlargement / reduction unit 124b. Further, a predetermined value may be stored in advance in the setting storage unit 115 as an initial value.

[Rotation angle information]
The rotation angle storage unit 115a stores information used for the rotation process. FIG. 8 is a diagram illustrating an example of a configuration of information stored in the rotation angle storage unit 115a. For example, the rotation angle storage unit 115a stores the number of rotations of the axis of the feature amount matrix, an angle change range for changing the rotation angle, and a flag indicating the rotation number and the angle change range applied at that time. In the example of FIG. 8, “rotation speed”, “angle change range”, and “flag” are stored. For example, an angle change range “−89 ° to + 89 °” is stored in association with the rotational speed “178”. In addition, a flag indicating that the rotational speed “178” and the angle change range “−89 ° to + 89 °” are selected at this time is stored. In the example of FIG. 8, a box of “rotation speed: arbitrary (manual setting)” is provided at the bottom so that the user can manually set the rotation speed and the angle change range. When the user manually sets the rotation speed and the angle change range, the number of times set in the bottom blank box and the set angle change range are stored.

  Here, an example in which the angle change range can be changed based on the information stored in the rotation angle storage unit 115a is shown. However, the rotation angle and the rotation number applicable in the image processing server 100 are fixed, and the rotation angle is changed. The storage unit 115a may not be provided. Further, as a result of processing in the rotation unit 124a, an angle change range or the like may be set in the rotation storage unit 115a.

  The enlargement / reduction rate storage unit 115b stores the enlargement / reduction rate when the feature amount matrix is enlarged / reduced by the matrix applied to the feature points for generating the feature amount matrix. FIG. 9 is a diagram illustrating an example of a configuration of information stored in the enlargement / reduction ratio storage unit 115b.

  The enlargement / reduction rate storage unit 115b stores an enlargement / reduction rate, the number of times of enlargement / reduction, and a flag indicating the enlargement / reduction rate applied at that time in association with the type of image. For example, as shown in FIG. 9, the enlargement / reduction ratio “0.5 to 2.0” is stored in association with the image type “satellite image”. Furthermore, “16” which is the number of times of enlargement / reduction in this case is stored. This is within the range of 0.5 to 2.0 times, in increments of 0.1, from "0.5 times", "0.6 times", "0.7 times" to "2.0 times" Means to apply a scaling factor of. Furthermore, a flag indicating that the enlargement / reduction ratio of the “satellite image” is applied at this time is stored.

  For example, the creation unit 124 increases or decreases the length of the major axis or minor axis of the feature amount matrix. Here, the enlargement / reduction process is performed by changing the length of the other axis on the basis of the short axis or the long axis. For example, the length of the major axis is changed to “1”, and the length of the minor axis is changed at an enlargement / reduction ratio of “0.5 to 2.0”. Note that the increase / decrease width of the enlargement / reduction ratio may be defined separately.

  The enlargement / reduction rate storage unit 115b stores the enlargement / reduction rate in advance in association with the type of image, but the user can arbitrarily set the enlargement / reduction rate. As shown at the bottom of FIG. 9, when the user arbitrarily sets the enlargement / reduction ratio, the enlargement / reduction ratio is stored in a blank box and a flag is set.

  Here, an example in which the enlargement / reduction ratio can be changed based on the information stored in the enlargement / reduction ratio storage unit 115b is shown, but the enlargement / reduction ratio that can be applied in the image processing server 100 is fixed, The storage unit 115b can be omitted. Further, as a result of processing in the enlargement / reduction unit 124b, an enlargement / reduction rate or the like may be set in the enlargement / reduction rate storage unit 115b.

  In the first embodiment, in the enlargement / reduction processing, the length of the major axis or the minor axis of the feature amount matrix is increased or decreased. However, when the axis direction changes as a result of rotating the axis of the feature amount matrix in the rotation processing described later, the feature amount matrix may be enlarged or reduced in the axis direction after the rotation.

[Example of difference information]
The difference information will be described with reference to FIG. FIG. 10 is a diagram illustrating an example of a configuration of information stored in the difference storage unit 116. As shown in FIG. 10, the difference information includes a “processed image ID” that uniquely identifies the processed image that caused the difference, and a “comparison image ID” that uniquely identifies the comparative image compared to the processed image. Including. Furthermore, the image extracted as the difference is divided into a plurality of areas, and an “area ID” that uniquely identifies each area is included. Then, “coordinate” information for specifying the position of each region is included. Here, assuming that each area is a rectangular area, the coordinates of the four corners of the rectangle are used as position information. The difference information further includes a “difference area image” extracted as a difference in the area.

In the example of FIG. 10, a plurality of region IDs “R001”, “R002”, and the like are stored in association with “processed image ID: 001” and “comparison image ID: 101”. Further, “coordinates” are stored in association with each area ID. For example, “coordinates: (x ₁ , y ₁ ), (x ₂ , y ₂ ), (x ₃ , y ₃ ), (x ₄ , y ₄ )” are stored in association with “area ID: R001”. The Further, information of the difference image itself is stored in association with “area ID: R001”. Although the difference image is divided into rectangular areas here, the manner of dividing the difference image is not particularly limited.

[Configuration of Control Unit 120]
Returning to FIG. 1, the configuration of the control unit 120 will be described. The control unit 120 includes a setting unit 121, a reception unit 122, an extraction unit 123, a creation unit 124, a determination unit 125, an alignment unit 126, and a detection unit 127. The control unit 120 executes processing for analyzing the image received via the input / output unit 130 and determining a feature point pair. Further, the control unit 120 performs image alignment and extraction of differences between images based on the determined feature point pair. Then, in order to suppress erroneous detection, the control unit 120 converts the extracted difference region using a homography matrix, and determines that the difference region that has become a predetermined size or less after conversion is not a difference. And the control part 120 excludes the difference area | region determined not to be a difference, synthesize | combines a difference image, and outputs it. Data generated by the processing in the control unit 120 is stored in the storage unit 110 as appropriate.

  The control unit 120 is, for example, various integrated circuits or electronic circuits. For example, an ASIC (Application Specific Integrated Circuit) is an example of the integrated circuit. Examples of the electronic circuit include a central processing unit (CPU) and a micro processing unit (MPU).

  The setting unit 121 sets a feature amount matrix in accordance with an input from the client 200 via the input / output unit. That is, the angle change range and the enlargement / reduction ratio of the feature amount matrix are set. For example, by inputting a predetermined command from the monitor 201 of the client 200, an input screen is displayed on the monitor 201, and the user inputs settings from the monitor 201. When there is a setting input, the setting unit 121 sets the input content in the rotation angle storage unit 115a and the enlargement / reduction rate storage unit 115b. For example, a flag is set for the corresponding angle change range and image type. Alternatively, when the user inputs an arbitrary numerical value, the numerical value is input into a blank box, and a flag is set in the box. Note that the enlargement / reduction ratio and rotation angle may be set by default without waiting for input of settings. Further, the enlargement / reduction ratio and the rotation angle may be set as a result of processing in the image processing server 100.

  As described above, in the case where the fixed angle change range and the enlargement / reduction ratio are applied to all images, the setting unit 121 may not be provided.

  The accepting unit 122 accepts at least two images obtained by photographing the same spot at different times via the input / output unit 130. The received image is sent to the extraction unit 123 based on the instruction input. The image received by the receiving unit 122 is stored in the image storage unit 111. The accepting unit 122 normally stores the accepted image only in the image storage unit 111, and reads an image stored in the image storage unit 111 when an instruction is input by an operator or the like, and sends it to the extraction unit 123. It may be configured to send.

  The extraction unit 123 extracts feature points from the image sent from the reception unit 122. For example, contour points and boundary points of buildings, roads, forests, sites, etc. in the image are extracted as feature points. The technique used for extracting the feature points is not particularly limited. For example, a method using a Harris operator or a Moravec operator, a Small Univalue Segment Assimilating Nucleus (SUSAN), a Minimum Intensity Change (MIC), or the like can be used. Information about the feature points extracted by the extraction unit 123 is stored in the feature point storage unit 112.

  The creation unit 124 creates the above-described feature amount matrix for each feature point extracted by the extraction unit 123. The creation unit 124 includes a rotation unit 124a and an enlargement / reduction unit 124b. The creation unit 124 performs the rotation process and the enlargement / reduction process of the feature amount matrix of the feature point of one of the two images (processed image) based on the information stored in the rotation angle storage unit 115a and the enlargement / reduction ratio storage unit 115b. To do. That is, the rotation unit 124a rotates the axis of the feature amount matrix based on the angle change range stored in the rotation angle storage unit 115a. The enlargement / reduction unit 124b enlarges / reduces the matrix applied to the feature points based on the enlargement / reduction rate stored in the enlargement / reduction rate storage unit 115b. Information of the feature amount matrix generated by the creation unit 124 through the rotation process and the enlargement / reduction process by the rotation unit 124a and the enlargement / reduction unit 124b is stored in the matrix storage unit 113. The rotation process and the enlargement / reduction process will be described in more detail later.

  The determination unit 125 compares the feature amount matrices created for the feature points of the two images, and calculates the similarity. For example, the determination unit 125 selects one feature point of one image. Then, the determination unit 125 compares the feature amount matrix of the feature point with the feature amount matrix of all the feature points of the other image. For example, the determination unit 125 compares the luminance difference levels (see FIGS. 4 and 5) of the respective regions of the two feature amount matrices. Then, the determination unit 125 calculates the similarity between the feature amount matrices based on the comparison result. The determination unit 125 determines a feature point pair having the highest similarity as a feature point pair. The method for calculating the similarity is not particularly limited, and, for example, normalized cross-correlation (ZNCC) can be used. Further, a method using a sum of absolute differences (SAD), a sum of squares of differences (SSD), or the like may be applied.

  The alignment unit 126 performs alignment of the two images based on the feature point pair determined by the determination unit 125. For example, the alignment unit 125 calculates a homography matrix based on the coordinates on the feature point pair image, and performs image alignment by converting one image using the homography matrix. However, the method used for alignment is not particularly limited.

  The detection unit 127 performs detection processing using the feature quantity matrix created by the creation unit 124 and stored in the matrix storage unit 113. First, the detection unit 127 extracts a difference area between two images after alignment. For example, the detection unit 127 compares the pixel values of the image after alignment, for example, the luminance component, and extracts the difference. However, the method used for difference extraction is not particularly limited.

  Then, the detection unit 127 extracts feature points that constitute a feature point pair in the difference area from each of the two images. Then, the detection unit 127 compares the extracted feature amount matrices of the feature points to calculate a homography matrix. The detection unit 127 converts the difference region using the calculated homography matrix. When the size of the converted difference area is smaller than a predetermined size, the detection unit 127 excludes the difference area from the difference image.

  And the detection part 127 synthesize | combines a difference image except for the excluded difference area among the extracted difference areas. The difference image synthesized by the detection unit 127 is output to the outside via the input / output unit 130. The configuration of the detection unit 127 and details of the detection process will be described later.

  In the above, the setting unit 121 and the alignment unit 126 are not necessarily provided. Further, the rotation unit 124a and the enlargement / reduction unit 124b are not necessarily provided. The processing may be performed using the feature amount matrix created for each feature point without performing the feature amount matrix rotation processing and the enlargement / reduction processing.

[Image Processing Method According to First Embodiment]
Next, the flow of image processing by the image processing server 100 according to the first embodiment will be described with reference to FIG. FIG. 11 is a flowchart illustrating a processing procedure of image processing performed by the image processing server 100 according to the first embodiment.

  First, the setting unit 121 receives a setting (step S201). Next, the accepting unit 122 accepts input of two images A (comparison image) and B (processed image) to be compared and collated (step S202). The reception unit 122 sends the received images A and B to the extraction unit 123. The extraction unit 123 extracts feature points from each of the received images A and B (step S203). For example, the extraction unit 123 extracts contour points and boundary points of buildings, roads, forests, sites, and the like in the image based on the luminance value of each pixel of the image and the direction of luminance change.

  Next, the creation unit 124 generates a feature amount matrix for each feature point of the image A extracted by the extraction unit 123 (step S204). That is, the creation unit 124 calculates the luminance value of the pixel in each region of the elliptical matrix centered on the feature point, and calculates the difference from the luminance value of the pixel of the feature point. The creation unit 124 generates a feature amount matrix by associating the calculated difference with each region.

  Next, the determination unit 125 selects one feature point of the image B (processed image) for which a feature amount matrix has not been created (step S205). In response to the selection by the determination unit 125, the creation unit 124 creates a feature amount matrix for the selected feature point of the image B (step S206). Then, the determination unit 125 compares the feature amount matrix of the feature points of the image B created by the creation unit 124 with the feature amount matrix of each feature point of the image A (comparison image) to calculate the similarity (step S207). ).

  Next, the determination unit 125 determines whether or not the rotation unit 124a has performed the rotation process for the predetermined number of rotations stored in the rotation angle storage unit 115a (step S208). If it is determined that the rotation processing at the predetermined number of rotations has not been completed (No at Step S208), the determination unit 125 returns the processing to the creation unit 124. The rotating unit 124a further performs rotation processing by rotating the axis of the feature amount matrix by a predetermined angle (step S209). Then, the creation unit 124 creates a feature matrix subjected to the rotation process (step S206), and the determination unit 125 calculates the similarity and stores it in the storage unit 110 (step S207).

  If it is determined that the rotation process at the predetermined number of rotations has been completed (Yes at Step S208), the determination unit 125 completes the enlargement / reduction process corresponding to the enlargement / reduction ratio stored in the enlargement / reduction ratio storage unit 115b. It is determined whether or not (step S210). If it is determined that the enlargement / reduction processing has not been completed (No at Step S <b> 210), the determination unit 125 returns the processing to the creation unit 124. The enlargement / reduction unit 124b executes the enlargement / reduction process of the feature amount matrix with the enlargement / reduction ratio that has not been completed (step S211). Then, the creating unit 124 creates a feature amount matrix of the feature points of the selected image B based on the feature amount matrix after the enlargement / reduction processing (step S206). The determination unit 125 calculates the similarity based on the feature amount matrix of the feature points of the created image B (step S207). Then, the creating unit 124 again determines whether or not the predetermined number of rotation processes have been completed (step S208).

  On the other hand, when it is determined in step S210 that the enlargement / reduction process has been completed (Yes in step S210), the determination unit 125 selects the feature point of the image A for which the highest similarity is calculated among the calculated similarities ( Step S212). Next, the determination unit 125 determines whether the similarity of the feature points of the selected image A is greater than or equal to a predetermined value (step S213). When it determines with less than a predetermined value (step S213, negative), the determination part 125 excludes the feature point of the image B selected at step S205 (step S214). That is, the feature point information is deleted from the feature point information stored in the feature point storage unit 112 for the image B. On the other hand, when it determines with more than predetermined value (step S213, affirmation), the determination part 125 determines the feature point of the image A selected in step S212, and the feature point of the image B selected in step S206 as a pair ( Step S215).

  Then, the determination unit 125 determines whether or not a pair has been determined for all feature points of the processed image (step S216). When it is determined that no pair has been determined for all feature points (No at Step S216), the determination unit 125 selects the next feature point (Step S217), and returns to the process at Step S206. On the other hand, when it is determined that pairs have been determined for all feature points (Yes in step S216), the alignment unit 126 performs image alignment based on the determined feature point pairs (step S218). Next, based on the two images after the alignment, the detection unit 127 performs a detection process (step S219). That is, the detection unit 127 extracts a difference between two images. And the detection part 127 extracts the feature point pair corresponding to a difference. The detection unit 127 calculates a homography matrix based on the feature amount matrix used for determining the feature point pair. Then, the detection unit 127 converts the difference area using the homography matrix, and determines the difference if the size of the converted area is equal to or greater than a predetermined value. Details of the detection process will be described later. This ends the process.

[Feature matrix creation process]
Next, an example of a feature amount matrix creation process will be described. FIG. 12 is a diagram for explaining an example of a feature amount matrix creation process. First, the determination unit 125 selects one feature point (attention point) from the feature points extracted from the image. For example, the determination unit 125 selects the _{P 1} in FIG. 12 (A). The creation unit 124 selects a feature point closest to the point of interest. For FIG. 12 (A), the order feature points located nearest the point of interest _{P 1} is _{P 2,} creating section 124 selects the _{P 2.} Creating unit 124, and calculates a line segment at an angle of 90 degrees relative to a line segment connecting the _{P 1} and _{P 2.} The creation unit 124 calculates a luminance change on the calculated line segment, and extracts a point where the luminance change is equal to or greater than a threshold value. For FIG. 12 (A), the at _{P 3,} since a large luminance change is equal to or greater than the resulting threshold, the result of the search, creation unit 124 extracts the _{P 3.} Then, the creation unit 124 calculates an ellipse that passes through P ₂ and P ₃ with P ₁ as the center (FIG. 12B). Creating unit 124, the calculated oval divided into a plurality of areas, and the luminance value of the pixels in each region, the difference between the pixel value of the feature point P ₁ in the center determined, associating the difference in the respective regions. As a result, a feature quantity matrix of feature points is created.

[Reason for using the closest feature point]
In the above example, the creation unit 124 uses the feature point closest to the attention point in creating the feature amount matrix. FIGS. 13A to 13C are diagrams for explaining the use of the feature point closest to the point of interest in the feature quantity matrix creation process.

First, in the image A shown in Figure 13-1, the point of interest and _{P 1,} closest feature points becomes _{P 2.} On the other hand, it is assumed that the entire image appears dark because the weather has changed, for example, because it was taken on a cloudy day or in the evening. When the entire image becomes dark like the image B shown in FIG. 13B, a portion having a large luminance difference, that is, an edge becomes weak as the entire image. However, the edge is not completely lost, and the creation unit 124 can extract a portion having a large luminance difference. Thus, creation section 124, a point of interest and P _1, it is possible to extract the feature point P ₂ as the nearest feature point.

Furthermore, a change such as a shadow on a part of the image may occur due to a change in sunlight. Let us consider a case in which only the lower left region of the image is shaded and the brightness of only the lower left region is different from the other portions as in the image C shown in FIG. In this case, in the lower left area because the edge is weak, creation unit 124 may not be able to extract the P ₂ as a feature point. As a result of P ₂ is not extracted as a feature point, creating 124, other feature points, for example, to extract the P ₃ as a feature point closest to the point of interest P _1. In this case, the feature matrix created by the creation unit 124 is shifted from the feature matrix created for the feature points of the comparison image, and there is a possibility that a feature point pair cannot be detected correctly.

  However, normally, after extracting hundreds of pairs as feature point pairs, the correctness is checked and processing for excluding erroneous correspondence pairs is performed. In the case of a satellite image or the like, it is unlikely that erroneous detection will occur due to the influence of sunshine change or the like on all feature points. Therefore, even if the influence of sunshine change is taken into consideration, an appropriate feature point pair can be determined by extracting the closest feature point and creating a feature amount matrix. Further, the creation unit 124 performs rotation processing and enlargement / reduction processing for one feature point of the two images, creates a plurality of feature amount matrices for one feature point, and when the similarity is smaller than the threshold value, Exclude feature points. Accordingly, it is possible to determine an appropriate feature point pair while suppressing erroneous detection.

  As described above, when the direction of the closest feature point is set as one axis direction of the elliptical matrix, the region of the feature amount matrix is set on the basis of the axis direction. For example, in the example shown in FIG. 6, the area of the matrix is set as shown in FIG. 6 with the long axis direction of the ellipse (direction from column 1 to column 7) as the direction of the closest feature point, and the luminance of each area Calculate the difference value. This makes it possible to properly compare the feature quantity matrices of feature points of different images. However, if the feature amount matrix can be properly compared, the region of the feature amount matrix may be set using another criterion.

[Enlargement / reduction processing]
The feature amount matrix enlargement / reduction processing will be described with reference to FIG. FIG. 14 is a diagram for explaining the feature quantity matrix enlargement / reduction processing. First, on the line segment at an angle of 90 degrees to the extracted nearest feature point P ₂ from the target points P _1, as described above, connecting the point of interest P ₁ and the nearest feature point P ₂ segment extracting a point P ₃ in the luminance is equal to or more than a threshold. Then, the length of the line segment forming an angle of 90 degrees is increased or decreased by a predetermined ratio to create a plurality of feature amount matrices.

  The length of the line segment that forms an angle of 90 degrees may be set to 0.5 to 2 times the range of increase / decrease based on the length from the point of interest to the feature point closest to the point of interest. Then, the feature amount matrix may be created by changing the length to 0.5 times, 0.6 times, 0.7 times,..., 2.0 times with respect to the reference length. For example, as shown in FIG. 14, the length of a line segment forming an angle of 90 degrees is changed.

  In this way, by creating a plurality of feature amount matrices by changing the length of the short axis or long axis of the feature amount matrix, even if a part of the image is distorted, the brightness of the image can be accurately obtained. Can be created.

  Here, the other feature point closest to the feature point (attention point) is extracted, a line segment connecting the attention point and the other feature point, and a line segment forming an angle of 90 degrees with the line segment. Are the short axis and long axis of the elliptic feature matrix. However, the present invention is not limited to this, and an arbitrary direction and another direction having a predetermined relationship with the direction may be the short axis direction and the long axis direction of the elliptical matrix. For example, if the image includes orientation information, the north direction or the like may be the first direction, that is, one of the minor axis direction and the major axis direction of the ellipse. Also, a direction obtained by correcting 90 ° in any direction by an arbitrary angle (90 ° ± α) with respect to the first direction is the second direction, ie, the other of the minor axis direction or the major axis direction of the ellipse. It is good. Also in this case, the enlargement / reduction unit 124b may perform the enlargement / reduction process by increasing / decreasing the major axis or the minor axis of the feature amount matrix at a predetermined ratio. When both the rotation process and the enlargement / reduction process are executed, the enlargement / reduction unit 124b may enlarge or reduce the feature amount matrix in the axial direction after the rotation.

  Here, the length of the line segment in the 90-degree direction is a point on the direction in which the luminance change at the point is equal to or greater than a predetermined threshold. However, the length of the line segment in the 90-degree direction may be, for example, the length of a predetermined number of pixels, depending on the ratio to the length between the feature point that is the center of the ellipse and the feature point closest to the feature point. You may decide.

[Setting the reduction ratio and enlargement ratio]
FIG. 15 is a diagram for further explaining the reduction ratio in the enlargement / reduction processing. As shown in FIG. 15, the radius of the feature quantity matrix is R (R is the number of pixels), and the number of regions in the radial direction is N. Here, when the radius R is smaller than the number N of regions, one region in the radial direction is less than one pixel. Therefore, the enlargement / reduction unit 124b sets the reduction rate such that one area in the radial direction is 1 pixel or more. That is, the maximum value of the reduction ratio p is set to p = N / R.

The enlargement / reduction unit 124b calculates the enlargement ratio q based on the size of the image. FIGS. 16A and 16B are diagrams for explaining the enlargement ratio in the enlargement / reduction process. As illustrated in FIG. 16A, the enlargement / reduction unit 124b cannot set the enlargement / reduction ratio to a numerical value such that the feature amount matrix exceeds the vertical and horizontal lengths of the image. For example, as shown in FIG. 16A, the vertical length of the image is W and the horizontal length is H. At this time, when the feature point is at the position represented by the coordinates “(x ₁ , y ₁ )” in FIG. 16A, the radius of the feature amount matrix is set longer than the distance from the feature point to the upper end of the image. I can't. Further, when the feature point is at the position represented by the coordinates “(x ₂ , y ₂ )” in FIG. 16A, it is possible to set the radius of the feature amount matrix longer than the distance from the feature point to the right edge of the image. Can not. Therefore, the enlargement / reduction unit 124b can determine the enlargement ratio of the feature amount matrix that can be set according to the coordinate position of the feature point.

  The enlargement ratio q can be determined based on the following formula depending on which of the positions (1) to (4) shown in FIG.

  When the feature point is at position (1) in FIG. 16B, if x is larger than y, q = x / R. If y is smaller than x, q = y / R.

  When the feature point is at position (2) in FIG. 16-2, if y is larger than (W−x), q = (W−x) / R. If y is smaller than (W−x), q = y / R.

  When the feature point is at position (3) in FIG. 16B, if (H−y) is larger than x, q = x / R. If (H−y) is smaller than x, q = (H−y) / R.

  When the feature point is at the position (4) in FIG. 16B, if (H−y) is larger than (W−x), q = (W−x) / R. If (H−y) is smaller than (W−x), q = (H−y) / R.

  Thus, the enlargement ratio and reduction ratio can be set. However, when the enlargement ratio and the reduction ratio are set so that the range of change due to enlargement / reduction is maximized, the processing amount increases and the burden on the image processing server 100 also increases. Therefore, in the case of a satellite image or the like in which there is no great variation in the enlargement ratio between images, an increase in the processing amount may be suppressed by setting a smaller enlargement / reduction width. Further, an enlargement / reduction ratio may be set in advance according to the type of image to be applied.

[Rotation processing]
Next, with reference to FIG. 17A to FIG. FIGS. 17A to 17C are diagrams for explaining the rotation process of the feature amount matrix.

As shown in FIG. 17A, when the shooting angle changes, the image appearance changes between a plurality of images. In the image A shown in FIG. 17A and the image B shown in FIG. 17-2, the ratio of the vertical and horizontal lengths of the subject slightly changes. In the image A, when the attention point is _P1A , the creation unit 124 extracts _P2A as the closest feature point. The creation unit 124 _extracts the _{P 3A} on the line segment at an angle of 90 degrees relative to a line segment connecting the _{P 1A} and _{P 2A.} Since a large luminance change has occurred in P _3A , the creating unit 124 extracts P _3A as a point where the luminance change is equal to or greater than a predetermined value. On the other hand, the length in the horizontal direction is shorter in the image B than in the image A. Therefore, when the point of interest is P _1B , the creation unit 124 extracts P _2B as the closest feature point, but P _{3B is} placed on a line segment that forms 90 degrees with the line segment connecting P _1B and P _2B. because there is _no, it can not be extracted _{P 3B.}

  Here, in order to make the point extracted as the point where the luminance greatly changes between the image A and the image B, the angle of the line segment used for extracting the luminance change may be changed. Therefore, the creation unit 124 creates a plurality of feature amount matrices by increasing or decreasing the angle between the attention point and the line segment connecting the feature point closest to the attention point within a predetermined range from 90 degrees (image C). FIG. 17-3). For example, the angle may be changed by 1 degree within a range from -89 degrees to +89 degrees with respect to the line segment.

[Angle change range determination processing]
The image processing server 100 may calculate the angle change range based on the received information when the information on the shooting angle is received together with the image even if the user does not set the image processing server 100. When the rotation unit 124a calculates the angle change range, the calculated numerical value or the like is stored in the rotation angle storage unit 115a. Then, based on the calculated angle change range, the rotation unit 124a may execute a rotation process of the feature amount matrix.

  Therefore, next, when the rotation unit 124a increases or decreases the angle formed by the major axis and the minor axis of the elliptic feature quantity matrix within a predetermined range to create a plurality of feature quantity matrices, the range in which the angle is changed. A method for determining the value will be described with reference to FIGS. 18-1 to 18-7. 18A to 18D are diagrams for explaining a method of determining the angle change range in the rotating unit 124a. FIG. 18-7 is a diagram illustrating an example of an angle change range determined by the rotating unit 124a.

  For example, in satellite images and aerial images, the angle from the shooting location to the subject may be acquired at the time of shooting. In such a case, the rotating unit 124a can determine a range in which the angle is changed based on the acquired angle. For example, as shown in FIG. 18A, it is assumed that a perfect circle with a radius r exists on the ground. In this case, when an image is taken in the downward direction from the position X in the sky shown in FIG. 18A, a perfect circle with a radius r looks as shown in FIG. Further, it is assumed that a perfect circle having a radius r obliquely downward from the sky position Y shown in FIG. Assume that the angle between the position X and the position Y is θ. In this case, the perfect circle photographed from the position Y looks as shown in FIG. That is, it looks like an ellipse having a major axis radius r and a minor axis radius rcos θ.

  Thus, in the case of an image taken from an oblique direction, the apparent angle of the 90-degree portion may actually differ greatly from 90 degrees. For example, in the example shown in FIG. 18-4, the angle between the two arrows is 90 degrees in an actual subject, but it appears to be larger or smaller than 90 degrees in appearance. On the other hand, when the image is taken from directly above the subject as in the position X in FIG. 18A, the apparent angle matches the actual angle. In the example shown in FIG. 18-5, the apparent angle is 90 degrees, which is the same as the actual angle.

Therefore, the rotation unit 124a determines a range in which the angle is changed so that the angle is changed with respect to a range including the angle of 90 degrees on the actual subject. For example, when the angle at which the shooting position is directly above the subject is θ, when the coordinates with the center as the origin are drawn, the position of the coordinates (r, r cos θ) and the coordinates ( r, the position of -rcosθ), coordinates (-r, by the angle phi _max and phi _min is determined by the position of rcosθ), it can determine the extent of changing the angle. This is because the apparent angle of 90 degrees is actually φ _{max at} the maximum and φ _min at the minimum. Specifically, the range for changing the angle can be defined by the following equation.

φ _min ≤ change range ≤ φ _max

Here, φ _max and φ _min can be obtained by the following equations.

φ _min = 2 tan ⁻¹ (cos θ)

φ _max = 2 tan ⁻¹ (1 / cos θ)

For example, when θ = 30 degrees, φ _min is 81.79 degrees and φ _max is 98.21 degrees. Therefore, the range in which the angle is changed is 81.79 degrees or more and 98.21 degrees or less.

  FIG. 18-7 shows an example of a range in which the angle calculated for each angle of θ is changed. For example, the rotation unit 124a may create the feature amount matrix by changing the angle by a predetermined angle within the change range shown in FIG. 18-7. In actual processing, the rotating unit 124a converts the angle shown in FIG. 18-7 into the number of pixels and determines the angle to be changed. For example, if one pixel corresponds to 50 centimeters to 1 meter, the rotating unit 124a creates a feature amount matrix by changing the angle by two or three pixels. The rotating unit 124a may determine a unit for changing the angle according to the size of the object to be extracted as a difference.

[Process of creating feature matrix after rotation process]
Next, a feature amount matrix creation process by the creation unit 124 when executing the rotation process will be described. FIG. 19A to FIG. 19D are diagrams for explaining a feature amount matrix creation process that has undergone a rotation process. First, as shown in FIG. 19A, it is assumed that the rotating unit 124a rotates the 90-degree direction by an angle θ. In FIG. 19A, the attention point is denoted by P ₁ and the feature point closest to the attention point is denoted by P ₂ .

With such rotation part 124a rotates the 90 degree direction by an angle theta, it is impossible to draw an ellipse around the attention point P _1. Therefore, the rotation unit 124a creates a feature amount matrix by combining four ellipse portions. First, as shown in FIG. 19-2, let P _x be the tip of a line segment rotated 90 degrees in the direction of θ. The rotating portion 124a from _{P x,} calculates the perpendicular line drawn against a line segment connecting the _{P 1} and _{P 2,} the intersection point _{P 3} and a line segment connecting the _{P 1} and _{P 2.} The rotating portion 124a includes a perpendicular and shorter diameter (or major axis) to calculate the ellipse the distance from P ₃ to P ₂ and the major axis (or minor axis). First, as shown in Figure 19-2, the rotation part 124a, for the upper right portion, it calculates an ellipse centered on _{P 3.}

Next, as shown in Figure 19-3, the rotation part 124a, for the upper left portion, centered on _{P 3,} the distance between _{P 1} and _{P 2,} from _{P 3} to the point where extended leftward An ellipse having a major axis (or minor axis) as the distance and a minor axis (or major axis) as the perpendicular is calculated. Furthermore, as shown in Figure 19-4, the rotation part 124a is normal to a line connecting the P ₁ and P ₂ from the lower side of the end point P _y of the line obtained by rotating the direction of 90 degrees by an angle θ Is calculated. The rotation unit 124a calculates the intersection point _{P 4} and a line segment and a vertical line connecting the _{P 1} and _{P 2.} The rotating portion 124a, only the length from _{P 1} to _{P 2,} and calculates the end point _{P 5} of the line segment extended in opposite directions from the _{P 1} and _{P 2.} Then, as shown in Figure 19-4, the rotation part 124a, for bottom left, centered on _{P 4,} the length of _{P 5} from _{P 4} the length of the _{P y} minor axis (or diameter) from _{P 4} An ellipse having a major axis (or a minor axis) as a diameter is calculated. Furthermore, as shown in Figure 19-5, the rotation part 124a, for the lower right portion, centered on _{P 4,} the _{P 4} of the length of the _{P 2} major axis (or minor axis), from _{P 4} to _{P y} An ellipse whose length is the minor axis (or major axis) is calculated.

  The rotation unit 124a calculates a feature matrix by combining the four parts of the ellipse thus obtained (FIG. 19-6).

  Then, the creation unit 124 associates the difference between the luminance value of the pixel in the region and the luminance value of the pixel of the feature point with each region of the feature amount matrix calculated by the rotation unit 124a, thereby generating the feature amount matrix. create.

[Example of detector configuration]
Next, an example of the configuration of the detection unit 127 will be described with reference to FIG. FIG. 20 is a diagram illustrating an example of the configuration of the detection unit 127. As illustrated in FIG. 20, the detection unit 127 includes a difference extraction unit 127a, a conversion unit 127b, and a difference synthesis unit 127c.

  The difference extraction unit 127a extracts a difference from the two images after alignment. The difference extraction unit 127a calculates, for example, the luminance of each image, and extracts an area where the luminance difference is equal to or greater than a predetermined value. However, the method used for difference extraction is not particularly limited.

  Further, the difference extraction unit 127a extracts a feature point corresponding to the extracted difference from each of the two images. For example, the difference extraction unit 127a divides the extracted difference image into a plurality of rectangular areas, and extracts feature points in each area. The difference located in the rectangular area is hereinafter also referred to as “difference area”. For example, the difference extraction unit 127a collates the coordinates of the rectangular area (see FIG. 10) with the coordinates of the feature points stored in the feature point storage unit 112 (see FIG. 3), and the features in the area. Extract points.

  Here, the feature points extracted by the difference extraction unit 127a are preferably in the difference area. However, if the feature points are in the rectangular area including the difference area, the feature points outside the difference area may be extracted. .

  The difference extraction unit 127a compares, for example, the image A (processed image) and the difference, extracts the feature points of the image A included in each region, and configures the feature point and feature point pairs of the extracted image A. Feature points of image B (comparison image) are extracted. Which image and the difference are collated first may be determined by the difference extraction mode.

  The conversion unit 127b calculates a homography matrix based on the feature amount matrix for each of the feature points of the two images extracted by the difference extraction unit 127a. A method for calculating the homography matrix will be described later. Note that the conversion unit 127b calculates a homography matrix using the feature amount matrix that has not been subjected to the rotation process and the enlargement / reduction process of the two feature points determined to be a feature point pair by the determination unit 125. Then, the conversion unit 127b converts the difference area based on the calculated homography matrix.

  Here, it is assumed that the conversion unit 127b calculates a homography matrix using the feature amount matrix used in the feature point pair determination process. However, a homography matrix may be calculated by creating a feature quantity matrix of each feature point forming a feature point pair again for the two images after alignment.

  The difference synthesis unit 127c compares the size of the difference area after being converted by the conversion unit 127b with a predetermined value, and determines that the difference area is a difference if the size is equal to or greater than the predetermined value. That is, the difference synthesis unit 127 c does not delete the difference information from the difference storage unit 116. On the other hand, when it is smaller than the predetermined value, the difference composition unit 127c determines that the difference area is not a difference. That is, the difference synthesis unit 127c deletes the difference information from the difference storage unit 116. That is, the difference is deleted from the difference area image corresponding to the area ID of the rectangular area including the difference area (see FIG. 10).

[Example of homography matrix calculation method]
Next, an example of a homography matrix calculation method will be described with reference to FIGS. 21-1 and 21-2. FIG. 21A and FIG. 21B are diagrams for explaining an example of a homography matrix calculation method.

  For example, the feature quantity matrix of the feature points in the rectangular area has the shape shown in FIG. 21-1 for the image A (comparison image) and the shape shown in FIG. 21-2 for the image B (processed image). To do. At this time, each feature amount matrix is positioned on the coordinates with the feature point as the origin. And the coordinate of the intersection of each coordinate axis of a coordinate and the outline of a feature-value matrix is calculated | required. At this time, for example, the coordinate axis, the feature point at the center of the feature matrix, and the line segment connecting the feature point and the closest feature point may be arranged to coincide with each other. However, the aspect at the time of arrange | positioning a feature-value matrix on a coordinate is not specifically limited.

  In the example of FIG. 21A, the conversion unit 127b calculates four points p1, p2, p3, and p4. In the example of FIG. 21-2, the conversion unit 127b calculates four points p1 ′, p2 ′, p3 ′, and p4 ′. Then, the conversion unit 127b calculates a matrix that moves p1, p2, p3, and p4 to p1 ′, p2 ′, p3 ′, and p4 ′, respectively. The calculated matrix is a homography matrix for the feature point pair.

[Flow of detection processing]
Next, the detection process will be described with reference to FIGS. 22-1 and 22-2. FIGS. 22-1 and 22-2 are diagrams for explaining the detection processing in the first embodiment. For example, as shown in FIG. 22-1, regarding the same object, it is assumed that there is a deviation due to a difference in shooting angle between the object shown in the processed image and the object shown in the comparison image. . In this case, the creation unit 124 creates an elliptic feature matrix as described above, and the rotation unit 124a and the enlargement / reduction unit 124b perform rotation processing and enlargement / reduction processing to determine an appropriate feature point pair. Can do. However, there is a case where the shift of the image itself is not completely eliminated by the alignment.

  FIG. 23A and FIG. 23B are diagrams for explaining image shifts that are not eliminated by alignment for images obtained by photographing the same subject. For example, in the case of an image C and an image D shown in FIG. In this case, even if the determination unit 125 extracts the feature point pair and the alignment unit 126 executes the alignment, the difference in the appearance of the subject itself as shown in FIG. Misalignment may remain.

  When the image is shifted as described above, the feature amount matrix of each feature point forming the feature point pair is in a state where the aspect ratios are different as shown in the lower part of FIG. However, in the feature point pair determination processing, since the rotation processing and the enlargement / reduction processing are performed on the feature amount matrix, the feature amount matrix shown in the lower part of FIG. 22-1 can be determined as an appropriate pair. Therefore, the conversion unit 127b calculates a homography matrix that reflects a shift between images based on the two feature amount matrices when the rotation process and the enlargement / reduction process are not performed.

  Then, as illustrated in FIG. 22-2, the conversion unit 127b converts the difference area extracted in the vicinity of the feature point using the homography matrix of the feature point. When the same object is photographed from different directions, for example, when the two images are captured differently and extracted as a difference, the converted difference area is a minute area. Therefore, a predetermined value is set as a threshold value in advance, and when the difference composition unit 127c has a smaller difference area after conversion, the difference area is deleted from the difference, thereby detecting an erroneous detection of the difference. Can be suppressed.

[Example of detection process flow]
Next, an example of the flow of detection processing in the detection unit 127 will be described with reference to FIG. FIG. 24 is a flowchart illustrating an example of a processing procedure of detection processing according to the first embodiment. First, the difference extraction unit 127a of the detection unit 127 extracts a difference between two images (step S301). The difference extracted by the difference extraction unit 127a is stored in the difference storage unit 116. At this time, the difference information is stored in the difference storage unit 116 for each of a plurality of areas (see FIG. 10). The difference extraction unit 127a first selects one region and extracts a feature point pair in the region (step S302). When the feature point pair is extracted, the conversion unit 127b reads the feature amount matrix of each feature point forming the feature point pair from the matrix storage unit 113. Then, as described above, the conversion unit 127b positions each feature amount matrix on the coordinates and extracts four corresponding points. Then, the conversion unit 127b calculates a homography matrix based on the four corresponding points (step S303).

  The conversion unit 127b converts the difference region using the calculated homography matrix (step S304). The difference composition unit 127c determines whether or not the converted difference area has a size greater than or equal to a predetermined value (step S305). When determining that the converted difference area is smaller than the predetermined value (No at Step S305), the difference combining unit 127c excludes the difference area from the difference (Step S306). That is, the information in the difference storage unit 116 is updated to information indicating that there is no difference for the difference area. On the other hand, if the difference composition unit 127c determines that the converted difference area is greater than or equal to a predetermined value (Yes in step S305), it determines that the difference area is a difference (step S307). That is, information that there is a difference in the difference area in the difference storage unit 116 is maintained.

  Then, the difference synthesis unit 127c determines whether or not all the difference areas have been processed (step S308). If it is determined that not all the difference areas have been processed (No at Step S308), the next difference area is selected (Step S309), and the process returns to Step S302. On the other hand, if it is determined that all the difference areas have been processed (Yes in step S308), the difference combining unit 127c combines the processed difference areas to generate a difference image. Then, the difference composition unit 127c outputs the generated difference image (Step S310). This completes the detection process.

  In the first embodiment, the homography matrix is calculated based on the feature amount matrix created before the image alignment. However, a more accurate difference detection can be performed by creating a feature matrix again based on the two images after alignment and calculating a homography matrix. Therefore, after the processing by the difference extraction unit 127a, the process may be returned to the creation unit 124 to create a feature amount matrix, and then the conversion unit 127b may calculate the homography matrix.

[Effect of Example 1]
In the first embodiment described above, the image processing server 100 extracts feature points from each of the first image and the second image. Then, the image processing server 100 determines a first direction for each feature point of the first and second images based on the feature point. The image processing server 100 determines the second direction based on the first direction. The image processing server 100 creates an elliptical matrix with the first direction and the second direction as axes and centering on the feature points. The image processing server 100 creates a feature quantity matrix by associating each area of the created matrix with information regarding the difference between the luminance value of the pixel in the area and the luminance value of the feature point. Further, the image processing server 100 compares the feature amount matrix of the feature points of the first image with the feature amount matrix of the feature points of the second image, and determines similar feature point pairs. The image processing server 100 converts the second image including the feature point based on the shape of the feature quantity matrix of the feature point of the second image used for determining the feature point pair. The image processing server 100 detects a difference based on the first image and the converted second image. Thus, since the image is converted based on the shape of the substantially elliptic feature matrix as the feature amount of the feature point, the image can be converted in consideration of the aspect ratio of the image. For this reason, even when there is a shift between images, alignment can be performed accurately.

  Further, in the first embodiment, the image processing server 100 uses the feature amount matrix of the feature points of the first image and the feature amount matrix of the feature points of the second image used for determining the feature point pair by the determination unit 125. Are arranged on the coordinates with the feature point as the center. Then, the image processing server 100 extracts an intersection between the coordinate axis of the coordinate and the outline of the feature amount matrix. Then, the image processing server 100 calculates a homography matrix that associates the intersection extracted for the feature quantity matrix of the feature point of the first image with the intersection extracted for the feature quantity matrix of the feature point of the second image. . The image processing server 100 converts the second image including the feature points using the calculated homography matrix. The image processing server 100 detects a difference based on the second image and the first image converted using the homography matrix. For this reason, even if any uniform image is generated, it is possible to accurately position and extract the difference. Thereby, erroneous detection in difference extraction can be suppressed.

  In addition, according to Example 1, the point which changed has been identified by comparing the image image | photographed with the artificial satellite and the aircraft each image | photographed at two time points apart in time, and extracting a difference. be able to. For example, according to the first embodiment, it is possible to specify a land or a building that has been illegally expanded or reconstructed, a forest that has been illegally cut, or a place where illegal dumping is performed. In addition, it is possible to investigate the actual place after specifying the place on the image.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[Omission of rotation and scaling]
In the above-described first embodiment, it has been described that a plurality of feature amount matrices are created by changing the rotation angle and the enlargement / reduction ratio for all feature points of one image. However, as a result of comparison by varying the rotation angle and the enlargement / reduction ratio for some feature points, a rotation angle and / or enlargement / reduction ratio having a high degree of similarity may be determined. In such a case, only the feature amount matrix having the same angle and / or the same enlargement / reduction ratio may be created for other feature points, and the similarity may be calculated.

  Further, the rotation process and the enlargement / reduction process may be executed for the feature points of both images.

  In the first embodiment, the rotation process and the enlargement / reduction process are performed. However, either the rotation process or the enlargement / reduction process may be executed.

[Image processing server]
The disclosed image processing method can be implemented in an image processing server configuring a cloud system and applied to processing of an image transmitted from a user via the cloud system. For example, photo data provided as needed from a satellite photo system or an aerial photo system is stored in a cloud system server. A user attaches information about a shooting location to an image and transmits the image. In addition, the user specifies an object to be detected for each shooting point, and designates an interval for performing processing for detecting whether there is a change. The image processing server determines feature point pairs by performing the disclosed image processing, and performs image alignment and difference extraction. With this configuration, the user can easily confirm what kind of change has occurred at the same point for each desired period simply by transmitting image data to the cloud system.

[Distribution and integration]
Each component of each illustrated apparatus does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the monitor 201 shown in FIG. 1 may be incorporated into the image processing server 100 so that the image processing server 100 can output an image or the like to the monitor for confirmation. The storage unit 110 may be configured as a single storage unit by integrating the image storage unit 111, the feature point storage unit 112, and the pair storage unit 114, or the image storage unit 111 as an external memory through the network 300. The image processing server 100 may be connected.

[Image processing program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes an image processing program having the same function as that of the above-described embodiment will be described with reference to FIG.

  FIG. 25 is a schematic diagram illustrating an example of a computer that executes an image processing program according to the first and second embodiments. As illustrated in FIG. 25, the computer 1000 includes an operation unit 1100, a display 1200, and a communication unit 1300. The computer 1000 further includes a CPU 1400, a ROM 1500, a RAM 1600, and an HDD (Hard Disk Drive) 1700. These units 1100 to 1700 are connected via a bus 1800.

  As shown in FIG. 25, the HDD 1700 has the same configuration as the setting unit 121, the reception unit 122, the extraction unit 123, the creation unit 124, the determination unit 125, the alignment unit 126, and the detection unit 127 described in the first embodiment. An image processing program 1700a or the like that exhibits the function is stored in advance. The image processing program 1700a may be appropriately integrated or separated as with each component shown in FIG. In other words, all the data stored in the HDD 1700 does not always need to be stored in the HDD 1700, and only the data necessary for processing may be stored in the HDD 1700.

  Then, the CPU 1400 reads the image processing program 1700 a from the HDD 1700 and develops it in the RAM 1600. Thereby, as shown in FIG. 25, the image processing program 1700a functions as an image processing process 1600a. The image processing process 1600a expands various data read from the HDD 1700 in an area allocated to itself on the RAM 1600, and executes various processes based on the expanded data. Note that the image processing process 1600a is performed by the setting unit 121, the reception unit 122, the extraction unit 123, the creation unit 124, the determination unit 125, the alignment unit 126, and the detection unit 127 illustrated in FIG. The processing shown in FIG. 10 is included. In addition, each processing unit virtually realized on the CPU 1400 does not always require that all processing units operate on the CPU 1400, and only a processing unit necessary for processing needs to be virtually realized.

  Note that the image processing program 1700a is not necessarily stored in the HDD 1700 or the ROM 1500 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc) disk, a magneto-optical disk, or an IC card inserted into the computer 1000. Let Then, the computer 1000 may acquire and execute each program from these portable physical media. In addition, each program is stored in another computer or server device connected to the computer 1000 via a public line, the Internet, a LAN, a WAN (Wide Area Network), etc., and the computer 1000 acquires each program therefrom. May be executed.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary Note 1) An extraction unit that extracts feature points from each of the first image and the second image;
For each feature point of the first and second images, the first direction determined based on the feature point and the second direction determined based on the first direction are axes, and A creation unit that creates a feature amount matrix in which each area of an elliptical matrix centered on the feature point is associated with information on a difference between a luminance value of a pixel in the area and a luminance value of the feature point; ,
A determining unit that compares the feature amount matrix of the feature points of the first image with the feature amount matrix of the feature points of the second image and determines a similar feature point pair;
Based on the shape of the feature quantity matrix of the feature points of the second image used for determining the feature point pair by the determination unit, the second image including the feature points is converted, and the first image and A detection unit for detecting a difference based on the converted second image;
An image processing apparatus comprising:

(Additional remark 2) The said detection part calculates the homography matrix for converting the shape of the feature-value matrix of the feature point of the said 2nd image into the shape of the feature-value matrix of the feature point of the said 1st image. The image processing apparatus according to appendix 1, wherein the second image including the feature points is converted using the homography matrix.

(Additional remark 3) The said creation part rotates the said 2nd direction by the predetermined angle centering | focusing on the said feature point, and produces the some feature-value matrix per one feature point,
The determining unit compares a feature amount matrix of a plurality of feature points of the first image with a feature amount matrix of the feature points of the second image, and determines a similar feature point pair. The image processing apparatus according to appendix 1 or 2.

(Supplementary Note 4) The creation unit creates a plurality of feature amount matrices for one feature point by enlarging or reducing the elliptical matrix in the second direction at a predetermined ratio,
The determining unit compares a feature amount matrix of a plurality of feature points of the first image with a feature amount matrix of the feature points of the second image, and determines a similar feature point pair. The image processing apparatus according to any one of appendices 1 to 3.

(Supplementary note 5)
A difference extraction unit for extracting a difference between the first image and the second image;
A conversion unit that converts the difference image extracted by the difference extraction unit based on the shape of the feature amount matrix of the feature points of the second image;
A difference synthesis unit that synthesizes the images of the differences converted by the conversion unit;
The image processing apparatus according to any one of supplementary notes 1 to 4, further comprising:

(Appendix 6) An image processing method executed by an image processing server,
Extracting feature points from each of the first image and the second image;
For each feature point of the first and second images, the first direction determined based on the feature point and the second direction determined based on the first direction are axes, and , Creating a feature amount matrix in which each area of the elliptical matrix centered on the feature point is associated with information on the difference between the luminance value of the pixel in the area and the luminance value of the feature point;
Comparing the feature quantity matrix of the feature points of the first image with the feature quantity matrix of the feature points of the second image, and determining similar feature point pairs;
Based on the shape of the feature amount matrix of the feature points of the second image used for determining the feature point pair, the second image including the feature points is converted,
Detecting a difference based on the first image and the converted second image;
And an image processing method.

(Supplementary Note 7) A homography matrix for converting the shape of the feature amount matrix of the feature point of the second image into the shape of the feature amount matrix of the feature point of the first image is calculated, and the homography matrix The image processing method according to appendix 6, wherein the second image including the feature points is converted by using.

(Supplementary Note 8) Rotate the second direction by a predetermined angle around the feature point to create a plurality of feature amount matrices per feature point,
Appendix 6 or 6, wherein a plurality of feature quantity matrices of feature points of the first image and a feature quantity matrix of feature points of the second image are compared to determine similar feature point pairs 8. The image processing method according to 7.

(Supplementary note 9) The elliptical matrix is enlarged or reduced in the second direction at a predetermined ratio to create a plurality of feature amount matrices for one feature point,
The feature amount matrix of the plurality of feature points of the first image and the feature amount matrix of the feature points of the second image are compared to determine a similar feature point pair. The image processing method according to claim 1.

(Appendix 10) Detecting a difference between the first image and the second image,
The detected difference image is converted based on the shape of the feature amount matrix of the feature points of the second image,
Synthesize the converted images of each difference,
The image processing method according to any one of appendices 6 to 9, further comprising:

(Appendix 11) Extracting feature points from each of the first image and the second image,
For each feature point of the first and second images, the first direction determined based on the feature point and the second direction determined based on the first direction are axes, and , Creating a feature amount matrix in which each area of the elliptical matrix centered on the feature point is associated with information on the difference between the luminance value of the pixel in the area and the luminance value of the feature point;
Comparing the feature quantity matrix of the feature points of the first image with the feature quantity matrix of the feature points of the second image, and determining similar feature point pairs;
Based on the shape of the feature quantity matrix of the feature points of the second image used for determining the feature point pair, the second image including the feature points is converted, and the first image converted to the first image is converted. Detecting a difference based on the two images;
An image processing program that causes a computer to execute each process.

(Additional remark 12) The homography matrix for converting the shape of the feature amount matrix of the feature point of the second image into the shape of the feature amount matrix of the feature point of the first image is calculated, and the homography matrix The image processing program according to appendix 11, wherein the second image including the feature points is converted by using.

(Additional remark 13) The said 2nd direction is rotated by the predetermined angle centering | focusing on the said feature point, A some feature-value matrix is produced per one feature point,
The feature amount matrix of the first image and the feature amount matrix of the feature point of the second image are compared with each other, and a similar feature point pair is determined. 12. An image processing program according to 12.

(Supplementary Note 14) The elliptical matrix is enlarged or reduced in the second direction at a predetermined ratio to create a plurality of feature amount matrices for one feature point,
Supplementary notes 11 to 11, wherein a plurality of feature amount matrices of feature points of the first image and a feature amount matrix of feature points of the second image are compared to determine similar feature point pairs. The image processing program according to any one of 13.

(Supplementary Note 15) Detecting a difference between the first image and the second image,
The detected difference image is converted based on the shape of the feature amount matrix of the feature points of the second image,
Synthesize the converted images of each difference,
The image processing program according to any one of appendices 11 to 14, characterized by comprising:

100 image processing server 110 storage unit 111 image storage unit 112 feature point storage unit 113 matrix storage unit 113a first matrix storage unit 113b second matrix storage unit 114 pair storage unit 115 setting storage unit 115a rotation angle storage unit 115b enlargement / reduction ratio storage Unit 116 difference storage unit 120 control unit 121 setting unit 122 reception unit 123 extraction unit 124 creation unit 124a rotation unit 124b enlargement / reduction unit 125 determination unit 126 alignment unit 127 detection unit 127a difference extraction unit 127b conversion unit 127c difference synthesis unit 130 Output unit 200 Client 201 Monitor 300 Network

Claims (9)

An extraction unit that extracts feature points from each of the first image and the second image;
For each feature point of the first and second images, a first direction toward the other feature point closest to the feature point and a second direction determined based on the first direction are respectively axes. And creating a feature quantity matrix in which each area of the elliptical matrix centered on the feature point is associated with information on the difference between the luminance value of the pixel in the area and the luminance value of the feature point. The creation department;
A determining unit that compares the feature amount matrix of the feature points of the first image with the feature amount matrix of the feature points of the second image and determines a similar feature point pair;
Based on the shape of the feature quantity matrix of the feature points of the second image used for determining the feature point pair by the determination unit, the second image including the feature points is converted, and the first image and A detection unit for detecting a difference based on the converted second image;
An image processing apparatus comprising:   The detection unit calculates a homography matrix for converting the shape of the feature amount matrix of the feature points of the second image into the shape of the feature amount matrix of the feature points of the first image, and the homography The image processing apparatus according to claim 1, wherein the second image including the feature points is converted using a matrix. The creation unit rotates the second direction by a predetermined angle around the feature point to create a plurality of feature amount matrices per feature point,
The determining unit compares a plurality of feature amount matrices of feature points of the first image with a feature amount matrix of feature points of the second image, and determines similar feature point pairs. The image processing apparatus according to claim 1 or 2. The creating unit creates a plurality of feature amount matrices per feature point by enlarging or reducing the elliptical matrix at a predetermined ratio in the second direction,
The determining unit compares a plurality of feature amount matrices of feature points of the first image with a feature amount matrix of feature points of the second image, and determines similar feature point pairs. The image processing apparatus according to any one of claims 1 to 3. The detector is
A difference extraction unit for extracting a difference between the first image and the second image;
A conversion unit that converts the difference image extracted by the difference extraction unit based on the shape of the feature amount matrix of the feature points of the second image;
A difference synthesis unit that synthesizes the images of the differences converted by the conversion unit;
The image processing apparatus according to claim 1, further comprising: An image processing method executed by an image processing server,
Extracting feature points from each of the first image and the second image;
For each feature point of the first and second images, a first direction toward the other feature point closest to the feature point and a second direction determined based on the first direction are respectively axes. And a feature quantity matrix in which information on the difference between the luminance value of the pixel in the area and the luminance value of the feature point is associated with each area of the elliptical matrix centered on the feature point. ,
Comparing the feature quantity matrix of the feature points of the first image with the feature quantity matrix of the feature points of the second image, and determining similar feature point pairs;
Based on the shape of the feature quantity matrix of the feature points of the second image used for determining the feature point pair, the second image including the feature points is converted, and the first image converted to the first image is converted. Detecting a difference based on the two images;
And an image processing method.   Calculating a homography matrix for converting the shape of the feature amount matrix of the feature points of the second image into the shape of the feature amount matrix of the feature points of the first image, and using the homography matrix, The image processing method according to claim 6, wherein the second image including the feature point is converted. Extracting feature points from each of the first image and the second image;
For each feature point of the first and second images, a first direction toward the other feature point closest to the feature point and a second direction determined based on the first direction are respectively axes. And a feature quantity matrix in which information on the difference between the luminance value of the pixel in the area and the luminance value of the feature point is associated with each area of the elliptical matrix centered on the feature point. ,
Comparing the feature quantity matrix of the feature points of the first image with the feature quantity matrix of the feature points of the second image, and determining similar feature point pairs;
Based on the shape of the feature quantity matrix of the feature points of the second image used for determining the feature point pair, the second image including the feature points is converted, and the first image converted to the first image is converted. Detecting a difference based on the two images;
An image processing program that causes a computer to execute each process.   Calculating a homography matrix for converting the shape of the feature amount matrix of the feature points of the second image into the shape of the feature amount matrix of the feature points of the first image, and using the homography matrix, The image processing program according to claim 8, wherein the second image including the feature point is converted.

Images