JP5971112B2 - Image processing method, image processing apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing method, an image processing apparatus, and an image processing program.

  In order to extract a difference between a plurality of images, there is a technique for aligning images using feature points of the plurality of images. A feature point is a point indicating a representative image position extracted from an image. For the extraction of feature points, for example, a corner of an object such as a building or an intersection included in an image is used. By aligning the feature points extracted from the images with the feature points of other images and matching the positions of the associated feature points, alignment between the plurality of images is performed.

  In addition, as a technique using the feature points of the image, for example, the feature points are extracted from the old aerial photograph image and the new aerial photograph image, and the correspondence between the feature points of the old and new aerial photograph images is specified, for example, There is a technique for calculating parameters for the alignment of a new aerial photograph image.

JP-A-2005-173128

  When associating feature points between images, it is unlikely that all feature points in one image are correctly associated with feature points in the other image, and there are pairs of feature points that are not correctly associated. . The greater the number of feature point pairs with incorrect association, the lower the alignment accuracy. However, the process for specifying and removing feature point pairs with incorrect associations is complicated, and there is a problem that the processing load is high and the process takes a long time.

  In one aspect, an object of the present invention is to provide an image processing method, an image processing apparatus, and an image processing program capable of determining whether or not feature points are correctly associated between images by simple processing.

  In one proposal, an image processing method is provided in which the following processing is performed by an image processing apparatus. In this image processing method, the image processing apparatus generates a plurality of feature points extracted from the first image and a plurality of feature points extracted from the second image in a one-to-one correspondence. In the state where the plurality of feature point pairs obtained are obtained and the first image and the second image are arranged in parallel, the inclination of the straight line connecting the feature points included in the feature point pair is determined by the plurality of feature point pairs. It calculates about each and it determines whether the matching of the feature points contained about each of several feature point pairs is correct based on the frequency of the calculated inclination.

Further, in one proposal, an image processing apparatus that performs the same processing as the above-described image processing method is provided.
Furthermore, in one proposal, an image processing program for causing a computer to execute the same processing as the above-described image processing method is provided.

  In one aspect, it is possible to determine whether or not the feature point association between images is correct with a simple process.

1 is a diagram illustrating an example of a functional configuration and an example of an operation of an image processing apparatus according to a first embodiment. It is a block diagram which shows the hardware structural example of the image processing apparatus of 2nd Embodiment. It is a block diagram which shows the example of a function structure of an image processing apparatus. It is a figure which shows the example of a feature point pair table. It is a figure which shows the example of the histogram based on the frequency information of the inclination about a feature point pair. It is a figure which shows the example of a histogram when a to-be-photographed object has a relationship of parallel movement. It is a figure which shows the example of a histogram when an image has a relationship of expansion / contraction. It is a figure which shows the example of a histogram when a to-be-photographed object has a relationship of rotation. It is a flowchart which shows the example of the image processing of 2nd Embodiment. It is a figure which shows the process example of 3rd Embodiment. It is a figure which shows the example of a histogram when a to-be-photographed object has a relationship of parallel movement. It is a figure which shows the example of a histogram when an image has a relationship of expansion / contraction. It is a figure which shows the example of a histogram when a to-be-photographed object has a relationship of rotation. 10 is a flowchart illustrating an example of image processing according to a third embodiment.

Hereinafter, the present embodiment will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating a functional configuration example and an operation example of the image processing apparatus according to the first embodiment. The image processing apparatus 10 is a computer that performs image processing, for example. The image processing apparatus 10 includes a storage unit 11 and a control unit 12.

  The storage unit 11 is realized by, for example, a volatile memory such as a RAM (Random Access Memory) or a nonvolatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The storage unit 11 includes a plurality of feature points 1a, 1b, 1c, and 1d extracted from the image 1 (first image) and a plurality of feature points 2a and 2b extracted from the image 2 (second image). Information on a plurality of feature point pairs generated by associating each of 2b, 2c, and 2d on a one-to-one basis is stored.

  The control unit 12 is realized using, for example, a processor such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor), and other electronic circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). Is done.

  The control unit 12 acquires information on a plurality of feature point pairs from the storage unit 11. The control unit 12 calculates, for each of the plurality of feature point pairs, the slope of a straight line connecting the feature points included in the feature point pair in a state where the image 1 and the image 2 are arranged in parallel. Then, the control unit 12 determines whether or not the feature points are correctly associated with each other for each of the plurality of feature point pairs based on the calculated inclination frequency.

  In the distribution of the inclination frequency of the straight line connecting the feature points included in each feature point pair, a specific property appears according to the relationship between the subjects included in each image. In view of such properties, the control unit 12 accurately determines whether the feature points are correctly associated with each other for each feature point pair based on the inclination frequency of the straight line connecting the feature points included in each feature point pair. Can be determined.

In addition, as a method for eliminating a feature point pair in which feature points are erroneously associated with each other, a method using a RANSAC (Random Sample Consensus) method or a LMedS (Least Median Square) method is known. In these methods, since the calculation is repeatedly performed for almost all combinations of the feature points of one image and the feature points of the other image, the processing load is large. For example, when the number of feature points is 100, and four pairs of feature points with the correct association are extracted from them, assuming that all combinations of feature points are calculated, ₁₀₀ C ₄ = 3921225 times Will be executed.

  On the other hand, since the control unit 12 repeats the calculation of the slope of the straight line by the number of feature points of each image, the processing load is greatly reduced. When the number of feature points is 100, the number of slope calculations is reduced to 100 times. That is, according to the image processing apparatus 10 of the present embodiment, it is possible to determine whether or not the feature points are correctly associated between the images with a simple process.

  For example, when the inclination frequency is concentrated and distributed over a certain ratio or more in a specific angle range, the control unit 12 determines that the calculated straight line inclination of the plurality of feature point pairs is within the specific range. The included feature point pair is determined as a correct feature point pair. For example, when the control unit 12 is concentrated and distributed at a certain ratio or more at one specific angle with an inclination frequency, the control unit 12 determines a feature point pair whose calculated straight line inclination is the specific angle as a feature It is determined that the point-to-point correspondence is a correct feature point pair. In addition, the range of the specific angle may be determined according to a direction in which the first image and the second image are arranged in parallel.

  Here, FIG. 1 shows an example in which a subject obtained by parallel translation of a subject included in image 1 is included in image 2. It is assumed that feature points 1a, 1b, 1c, and 1d are detected from the image 1, and feature points 2a, 2b, 2c, and 2d are detected from the image 2. Also, the feature point 1a and the feature point 2a, the feature point 1b and the feature point 2b, the feature point 1c and the feature point 2c, and the feature point 1d and the feature point 2d are features in which the feature points of each image are associated one-to-one. It is a point pair. Furthermore, it is assumed that the feature point 1a and the feature point 2a, the feature point 1b and the feature point 2b, the feature point 1c and the feature point 2c are correctly associated, and the feature point 1d and the feature point 2d are invalidly associated.

  The control unit 12 performs processing by arranging the images 1 and 2 in parallel in the horizontal direction, for example. Here, the histogram 3 shows the frequency distribution of the slope of the straight line connecting the feature points included in the feature point pair, calculated by the control unit 12. In the example of FIG. 1, the slopes of the straight lines connecting the feature point 1a and the feature point 2a, the feature point 1b and the feature point 2b, and the feature point 1c and the feature point 2c are both around 15 °, and the slope frequency around 15 ° is 3 On the other hand, the slope of the straight line connecting the feature point 1d and the feature point 2d is around 359 °, and the slope frequency near 359 ° is 1. In this case, it is determined that the inclination frequency distribution is concentrated at an inclination of 15 °.

  At this time, a feature point pair whose calculated slope is 15 ° where the frequency distribution of the slope is concentrated, or a feature point pair included in a predetermined range centered on 15 °, It is determined that the correspondence is correct. That is, it is determined that the correspondence between the feature point 1a and the feature point 2a, the feature point 1b and the feature point 2b, and the feature point 1c and the feature point 2c is correct. On the other hand, the association between the feature point 1d and the feature point 2d included in the feature point pair whose calculated slope is around 359 ° where the slope frequency distribution is not concentrated is determined to be invalid.

  According to the image processing apparatus 10 of the first embodiment described above, the inclination of the straight line connecting the feature points included in the feature point pair is calculated and calculated in a state where the image 1 and the image 2 are arranged in parallel. It is determined whether or not the association is correct for each of the feature point pairs based on the inclination frequency. Thereby, it is possible to determine whether or not the correspondence between the feature point pairs between the images is correct by a simple process.

[Second Embodiment]
FIG. 2 is a block diagram illustrating a hardware configuration example of the image processing apparatus according to the second embodiment. The image processing apparatus 100 is a computer having an image processing function.

  For example, the image processing apparatus 100 receives data of two images to be aligned from the client 40. The image processing apparatus 100 generates a plurality of feature point pairs by associating feature points having similar feature quantities between two images on a one-to-one basis, and based on the positional relationship between the associated feature points, Perform image alignment. The image processing apparatus 100 has a function of determining whether or not the feature points are correctly associated with each other for each of the plurality of feature point pairs. The image processing apparatus 100 performs alignment using only the feature points that are determined to have the correct association between the feature points.

  The image processing apparatus 100 includes a processor 101, a RAM 102, an HDD 103, an image signal processing unit 104, an input signal processing unit 105, a disk drive 106, and a communication interface 107.

  The processor 101 includes a computing unit that executes program instructions. The processor 101 loads at least a part of the program and data stored in the HDD 103 into the RAM 102 and executes the program. The processor 101 may include a plurality of processor cores. In addition, the image processing apparatus 100 may include a plurality of processors. The image processing apparatus 100 may perform parallel processing using a plurality of processors or a plurality of processor cores. A set of two or more processors, a dedicated circuit such as an FPGA or an ASIC, a set of two or more dedicated circuits, a combination of a processor and a dedicated circuit, and the like may be referred to as a “processor”.

  The RAM 102 is a volatile memory that temporarily stores programs executed by the processor 101 and data referred to by the programs. Note that the image processing apparatus 100 may include a plurality of volatile memories.

  The HDD 103 is a non-volatile storage device that stores software programs and data such as an OS (Operating System), firmware, and application software. Note that the image processing apparatus 100 may include another type of storage device such as an SSD (Solid State Drive), or may include a plurality of nonvolatile storage devices.

  The image signal processing unit 104 outputs an image to the display 21 connected to the image processing apparatus 100 in accordance with an instruction from the processor 101. As the display 21, a CRT (Cathode Ray Tube) display, a liquid crystal display, or the like can be used.

  The input signal processing unit 105 acquires an input signal from the input device 22 connected to the image processing apparatus 100 and notifies the processor 101 of the input signal. As the input device 22, a pointing device such as a mouse or a touch panel, a keyboard, or the like can be used.

The display 21 and the input device 22 may be formed integrally with the housing of the image processing apparatus 100.
The disk drive 106 is a drive device that reads programs and data recorded on the recording medium 23. Examples of the recording medium 23 include a magnetic disk such as a flexible disk (FD) and an HDD, an optical disk such as a CD (Compact Disc) and a DVD (Digital Versatile Disc), and a magneto-optical disk (MO). Can be used. The disk drive 106 stores the program and data read from the recording medium 23 in the RAM 102 or the HDD 103 in accordance with an instruction from the processor 101. Note that the image processing apparatus 100 may not include the disk drive 106.

The communication interface 107 communicates with other information processing apparatuses such as the client 40 via the network 30.
FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing apparatus. The image processing apparatus 100 includes an image information storage unit 110, a feature point pair calculation unit 120, a feature point information storage unit 130, a feature point pair determination unit 140, a frequency information storage unit 150, and an alignment processing unit 160.

The image information storage unit 110 stores information indicating two images to be aligned. In the present embodiment, it is assumed that the two images to be aligned are photographs.
The feature point pair calculation unit 120 extracts feature points of each image based on the image information stored in the image information storage unit 110. Next, the feature point pair calculation unit 120 generates a plurality of feature point pairs in which the feature points of one image and the feature points of the other image are associated, and stores information about the generated feature point pairs as feature point information storage Stored in the unit 130. Feature points having similar feature quantities are associated with the feature point pairs.

The feature point information storage unit 130 stores information regarding each feature point pair generated by the feature point pair calculation unit 120.
The feature point pair determination unit 140 is a straight line connecting feature points included in each feature point pair for each of the plurality of generated feature point pairs in a state where images to be aligned are arranged in parallel in a predetermined direction. Calculate the slope. The calculated inclination is an angle between a straight line connecting feature points included in the feature point pair and a predetermined reference line. In this embodiment, the reference line is a horizontal line, the right direction from the point of interest along the horizontal line is 0 °, the counterclockwise direction is the positive rotation direction, and the clockwise direction is the negative rotation direction. The feature point pair determination unit 140 generates the frequency information of the slope calculated for each feature point pair and stores it in the frequency information storage unit 150.

  Based on the generated frequency information, the feature point pair determination unit 140 determines whether or not the association between the feature points included in each of the feature point pairs is correct, and determines that the association is incorrect. The feature point pair information is deleted from the feature point pair table.

The frequency information storage unit 150 temporarily stores the frequency information of the tilt calculated for the feature point pair generated by the feature point pair determination unit 140.
The alignment processing unit 160 performs alignment between images based on the positional relationship between feature points included in each feature point pair registered in the feature point pair table.

  FIG. 4 is a diagram illustrating an example of a feature point pair table. The feature point pair table 131 is a table that stores information on each feature point pair for each feature point pair. The feature point pair table 131 is stored in the feature point information storage unit 130.

  The feature point pair table 131 includes items of ID, feature point # 1, feature point # 2, and inclination. In the ID item, a number for identifying a feature point pair is set. In the feature point # 1, the coordinates of the feature point in one of the two images are set. In the feature point # 2, the coordinates of the feature point in the other of the two images are set.

  One feature point pair is configured by the two feature points set in the feature points # 1 and # 2 associated with the same ID. The same number of feature points are extracted from each image to be aligned, and each feature point of one image and each feature point of the other image are associated one-to-one. Thereby, the same number of feature point pairs as the feature points extracted from one image are generated.

In the item of inclination, an inclination calculated for a feature point pair in a state where two images are arranged is set.
FIG. 5 is a diagram illustrating an example of a histogram based on inclination frequency information for feature point pairs. The histogram 151 is a histogram based on inclination frequency information for each feature point pair. Frequency information corresponding to the histogram 151 is stored in the frequency information storage unit 150. In the histogram 151, the x-axis indicates the inclination calculated for the feature point pair, and the y-axis indicates the frequency of the inclination.

  Next, when images are arranged in the horizontal direction, the relationship between the subject state of two images and the frequency information of inclination will be described. In addition, in the following FIGS. 6-9, the broken-line arrow shows the straight line which connected the feature points contained in the feature point pair.

FIG. 6 is a diagram illustrating an example of a histogram when the subject has a translational relationship. The image 113 is an image in which the subject 111 included in the image 112 is translated in the lower right direction.
It is assumed that feature points 112a, 112b, 112c, and 112d are extracted from the image 112, and feature points 113a, 113b, 113c, and 113d are extracted from the image 113. It is assumed that the feature point 112a and the feature point 113a, the feature point 112b and the feature point 113b, the feature point 112c and the feature point 113c are associated with each other as a feature point pair, and the association is correct. On the other hand, it is assumed that the feature point 112d and the feature point 113d are associated as a feature point pair, but the association is incorrect.

  The feature point pair determination unit 140 connects the feature points for each feature point pair with a straight line, and calculates the inclination (angle) of those straight lines. As shown in FIG. 6, in a state where the images 112 and 113 are arranged in the horizontal direction, the slopes calculated for each feature point pair with the correct association are the same. For this reason, the inclination about the feature point pair with which matching is correct concentrates on one value like the histogram 152. On the other hand, for feature point pairs such as feature point pairs of the feature points 112d and 113d that are not correctly associated, the slope of the calculated straight line is indefinite, and thus is dispersed without concentrating on one value. As a result, the frequency of the inclination of the feature point pair that is correctly associated is more prominent than the frequency of the other inclinations, and a significant peak appears in the frequency distribution.

  Therefore, when a significant peak appears in the slope frequency distribution when images are arranged in the horizontal direction, a feature point pair whose slope corresponding to the peak is calculated is a feature point pair whose feature points are correctly associated with each other. Can be judged. Note that, for example, in consideration of image distortion, a feature point pair in which the calculated inclination is included in a certain width centered on a peak may be determined as a feature point pair in which the feature points are correctly associated with each other. Good.

  FIG. 7 is a diagram illustrating an example of a histogram when an image is in a scaling relationship. The images 114 and 115 include the same subject 111. The image 115 is an enlarged image of the image 114.

  It is assumed that feature points 114a, 114b, 114c, and 114d are extracted from the image 114, and feature points 115a, 115b, 115c, and 115d are extracted from the image 115. It is assumed that the feature point 114a and the feature point 115a, the feature point 114b and the feature point 115b, the feature point 114c and the feature point 115c are associated with each other as a feature point pair, and the association is correct. On the other hand, it is assumed that the feature point 114d and the feature point 115d are associated as a feature point pair, but the association is incorrect.

  As shown in FIG. 7, in a state where the upper ends of the images 114 and 115 are aligned and aligned in the horizontal direction, the inclination calculated for each feature point pair with the correct association is in the range of 270 ° to 0 ° (360 °). Inside. On the other hand, since the inclination of the feature point pair with the incorrect association such as the feature point pair of the feature points 114d and 115d is indefinite, the inclination calculated for each feature point pair varies. Therefore, the inclination frequency calculated for the feature point pair in the above case is concentrated in the range of 270 ° to 0 ° as in the histogram 153. Therefore, when the images 114 and 115 are arranged in the horizontal direction with their upper ends aligned, if the distribution of inclination frequencies is concentrated in the range of 270 ° to 0 ° (360 °), the calculated inclination is within the range. It is presumed that the feature point pairs not included in are not correctly associated with each other.

  Note that, for example, in a state where the lower ends of the images 114 and 115 are aligned and aligned in the horizontal direction, the inclination calculated for each feature point pair with the correct association is in the range of 0 ° to 90 °. Therefore, when the images 114 and 115 are arranged in the horizontal direction with the bottom ends aligned, if the distribution of the inclination frequency is concentrated in the range of 0 ° to 90 °, the calculated inclination is not included in the range. It is presumed that the feature points are not correctly associated with each other for the feature point pair.

  The above description is a case where the other image 115 is an enlargement of one image 114. For example, when the other image is a reduction of one image, the following determination is made. Is done. When the images are arranged in the horizontal direction with their upper ends aligned, if the distribution of the inclination frequency is concentrated in the range of 0 ° to 90 °, the feature point pair whose calculated inclination is not included in the range is described. Is estimated that the feature points are not correctly associated with each other. On the other hand, when the images are arranged in the horizontal direction with the bottom ends aligned, if the distribution of inclination frequencies is concentrated in the range of 270 ° to 0 ° (360 °), the calculated inclination is included in the range. It is presumed that the feature point pair is not correctly associated with each other.

  Although not shown, for example, a subject in an image arranged on the right side rotates a subject in an image of the same size arranged on the left side clockwise with respect to a point on the left outer side from the left end of the image. If it is, the inclination calculated for each feature point pair with the correct association is in the range of 270 ° to 0 °. On the other hand, when the subject in the image arranged on the right side is a subject in the same-size image arranged on the left side rotated counterclockwise with respect to a point outside the left end of the image, the association is performed. The slope calculated for each correct feature point pair is in the range of 0 ° to 90 °.

  In addition, when the subject in the image arranged on the right side is a subject in the image of the same size arranged on the left side rotated counterclockwise with respect to a point outside the right end of the image, the association is performed. The slope calculated for each correct feature point pair is in the range of 270 ° to 0 °. On the other hand, if the subject in the image arranged on the right side is a subject in the image of the same size arranged on the left side rotated clockwise with respect to a point outside the right end of the image, the association is performed. The slope calculated for each correct feature point pair is in the range of 0 ° to 90 °.

  Therefore, when the images of the same size are arranged in the horizontal direction, if the distribution of the inclination frequency is concentrated in one of the ranges of 270 ° to 0 ° or 0 ° to 90 °, the calculated inclination is within the range. It is presumed that the feature point pairs not included in are not correctly associated with each other.

  FIG. 8 is a diagram illustrating an example of a histogram when the subject is in a rotational relationship. The image 117 is an image obtained by rotating the subject 111 included in the image 116 around a point inside the image 116.

  It is assumed that feature points 116a, 116b, 116c, and 116d are extracted from the image 116, and feature points 117a, 117b, 117c, and 117d are extracted from the image 117. It is assumed that the feature point 116a and the feature point 117a, the feature point 116b and the feature point 117b, the feature point 116c and the feature point 117c are associated with each other as a feature point pair, and the association is correct. On the other hand, the feature point 116d and the feature point 117d are associated as a feature point pair, but the association is incorrect.

  As shown in FIG. 8, in the state where the images 116 and 117 are arranged in the horizontal direction, the inclination calculated for each feature point pair with the correct association is 0 ° (360 °) −n ° to 0 ° + n ° ( However, it falls within the range of 0 <n <90). On the other hand, since the inclination of the feature point pair that is not correctly associated, such as the feature point pair of the feature points 116d and 117d, is indefinite, the inclination calculated for each feature point pair varies. Therefore, the slope frequency calculated for the feature point pair in the above case is concentrated in the range of 0 ° (360 °) −n ° to 0 ° + n ° as shown in the histogram 154. Note that the range that n can take is theoretically 0 <n <90, but it is rare that the slope calculated for the feature point pair with the correct association approaches 90 °. Therefore, practically, the maximum value of n may be set to about 45 °.

  From the above, when the images are arranged in the horizontal direction, when the distribution of inclination frequencies is concentrated in the range of 0 ° (360 °) −n ° to 0 ° + n °, the calculated inclination is within the range. For feature point pairs not included, it is presumed that the feature points are not correctly associated with each other.

FIG. 9 is a flowchart illustrating an example of image processing according to the second embodiment. In the following, the process illustrated in FIG. 9 will be described in order of step number.
(Step S11) The feature point pair calculation unit 120 extracts feature points from each image based on the information of each image stored in the image information storage unit 110. The feature points may be extracted using points such as corners of buildings and intersections, for example. The feature points may be extracted using SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Feature), or the like.

  (Step S12) The feature point pair calculation unit 120 generates a feature point pair in which each feature point of one image is associated with each feature point of the other image on a one-to-one basis. Information about the generated feature point pair is stored in the feature point information storage unit 130.

  As a procedure for associating feature points, for example, the feature point pair calculation unit 120 first calculates a feature amount indicating feature point information for each extracted feature point. The feature quantity includes, for example, a SIFT feature quantity or a SURF feature quantity obtained by extracting a feature vector at the extracted feature point as a feature quantity. The feature amount may be calculated based on an average value or a variance value of the luminance of each pixel in a predetermined area centered on the feature point.

  Next, the feature point pair calculation unit 120 identifies the feature point having the feature amount having the highest similarity with the feature amount of the feature point of one image from the other image. Then, a feature point pair is generated by associating the identified feature point with the feature point of one image.

  (Step S13) The feature point pair determination unit 140 is a straight line that connects the feature points included in the feature point pair for each feature point pair in a state where the images are arranged in a predetermined direction (here, the horizontal direction). Is calculated. At this time, the feature point pair determination unit 140 juxtaposes the images so that the upper end or lower end of each image matches in the horizontal direction. The inclination is an angle formed by a direction from a feature point of one image to a feature point of the other image and a right direction along the horizontal line.

  The feature point pair determination unit 140 registers the calculated inclination as the inclination of the corresponding feature point pair in the feature point pair table 131. The feature point pair determination unit 140 generates frequency information based on information indicating the inclination frequency of each feature point pair in the feature point pair table 131 and registers the frequency information in the frequency information storage unit 150.

  (Step S14) The feature point pair determination unit 140 determines whether the calculated inclination is concentrated at one angle based on the frequency information (that is, whether there is a significant peak in the frequency distribution). If the inclination is concentrated at one angle, the process proceeds to step S15. If the inclination is not concentrated on one angle, the process proceeds to step S16.

  Whether or not the inclination is concentrated at one angle can be determined by whether or not there is a significant peak in the histogram. For example, the determination can be made based on whether the highest frequency is several times (for example, three times) or more than the second highest frequency. Further, the determination may be made based on whether or not the highest frequency relative to the total of all frequencies is equal to or higher than a predetermined ratio (for example, 40%).

  (Step S15) The feature point pair determination unit 140 calculates, for each of the feature point pairs included in a certain range centered on the slope with the highest frequency (the slope corresponding to the prominent peak). It is determined that the correspondence between the included feature points is correct. The range centered on the inclination having the highest frequency is, for example, a range from the inclination −3 ° to the inclination + 3 °. The feature point pair determination unit 140 deletes information about feature point pairs that have not been determined to be correct from the feature point pair table 131.

  (Step S16) The feature point pair determination unit 140 determines whether the frequency distribution is concentrated in the range of 0 ° to 90 ° based on the frequency information. When the frequency distribution is concentrated in the range, the feature point pair determination unit 140 sets the angle range to 0 ° to 90 °. Then, the process proceeds to step S19. On the other hand, if the frequency distribution is not concentrated in the range, the process proceeds to step S17.

  Whether or not the frequency distribution is concentrated is, for example, whether the sum of the frequencies in the range of 0 ° to 90 ° is several times (for example, twice) the sum of the frequencies in the range other than 0 ° to 90 °. It can be judged by whether or not.

  (Step S17) The feature point pair determination unit 140 determines whether the frequency distribution is concentrated in the range of 270 ° to 360 ° based on the frequency information. When the frequency distribution is concentrated in the range, the feature point pair determination unit 140 sets the angle range to 270 ° to 360 °. Then, the process proceeds to step S19. On the other hand, if the frequency distribution is not concentrated in the range, the process proceeds to step S18.

  Whether or not the frequency distribution is concentrated is, for example, whether the total frequency in the range of 270 ° to 360 ° is several times (for example, twice) or more of the total frequency in the range other than 270 ° to 360 °. It can be judged by whether or not.

  (Step S18) Based on the frequency information, the feature point pair determination unit 140 calculates a range in which the distribution of the inclination frequency centering on 0 ° is concentrated. The calculation of the range is performed by calculating a predetermined ratio (for example, 50) of the total frequency in the range of 0 ° (360 °) −n ° to 0 ° + n ° (n is an integer of 1 or more). %) Can be realized by calculating n. As described above, the range of n is theoretically 0 <n <90, but it can be practically 0 <n <45. The feature point pair determination unit 140 sets the angle range from 0 ° (360 °) −n ° to 0 ° + n ° using the calculated value of n.

  (Step S19) The feature point pair determination unit 140 associates the feature points included in each of the feature point pairs for which the inclination included in the angle range set in any of steps S16 to S18 is calculated. Judge that it is correct. The feature point pair determination unit 140 deletes information about feature point pairs that have not been determined to be correct from the feature point pair table 131.

  (Step S <b> 20) The alignment processing unit 160 performs alignment of each image based on the positional relationship of the feature point pairs currently registered in the feature point pair table 131. In addition, the alignment processing unit 160 may detect an area having a large difference between the images in the state where the alignment is performed, and output information indicating the area.

  In the processing of FIG. 9 described above, the processing order of steps S34 and S35 can be changed. However, the determination process in step S32 is preferably prioritized over the determination processes in steps S34 and S35. In addition, it is desirable that the determination process in steps S34 and S35 has priority over the determination process in step S36.

  According to the image processing apparatus of the second embodiment, with respect to two images arranged in the horizontal direction, the distribution of inclination frequencies calculated for feature point pairs has a significant peak or is concentrated in a specific range. If it is determined that the feature point pair for which the slope corresponding to the peak is calculated or the feature point pair included in the feature point pair for which the slope included in the specific range is calculated is correct. Thereby, compared with the method using the RANSAC method or the LmedS method, it is possible to realize whether the association is correct by a simple process. Therefore, the load of the determination process is reduced and the processing speed is improved.

  This also makes it possible to accurately determine a feature point pair including feature points that are correctly associated even when the ratio of feature point pairs that include feature points that are correctly associated with each other is small. For example, in the method using the feature point pair combination with the smallest coordinate deviation among the feature point pair combinations by the RANSAC method or the LmedS method, the feature point pair including the feature points with the correct correspondence is more than half of the whole. If it does not occupy, the accuracy of identifying a feature point pair whose association is incorrect may be reduced. On the other hand, when the image processing apparatus 100 is used, for example, when specifying a feature point pair for translated images, the highest inclination frequency is several times (for example, three times) the second highest inclination frequency or Even about 40% of the total, it is possible to identify feature point pairs whose association is incorrect. Therefore, even when the frequency of the highest gradient is lower than the frequency of the other gradients, it is possible to accurately remove the feature point pairs including the feature points that are associated incorrectly.

  In addition, when the image processing apparatus 100 arranges the images in the horizontal direction, if a significant peak appears in the frequency distribution in one inclination, the inclination included in a certain range centered on the angle corresponding to the significant peak. The calculated feature point pair is determined as a feature point pair including feature points that are correctly associated with each other. This makes it possible to determine whether or not the feature point pair is correctly associated with the feature point pair extracted from the two images in which the subject has a parallel movement relationship.

  In addition, when the distribution of the inclination frequency is concentrated only in one of the ranges of 270 ° to 0 ° or 0 ° to 90 °, the image processing apparatus 100 calculates the feature point pair in which the inclination in the range is calculated. Are determined to be feature point pairs including feature points that are correctly associated with each other. Thereby, it is possible to determine whether or not the feature point pair is correctly associated with each other for the feature point pairs extracted from the two images having the enlargement / reduction relationship.

  Further, when the distribution of the frequency of the inclination is concentrated in the range of (360 °) −n ° to 0 ° + n °, the image processing apparatus 100 associates the feature point pairs in which the inclination in the range is calculated. Is a feature point pair including correct feature points. This makes it possible to determine whether or not the feature point pairs are correctly associated with each other for feature point pairs extracted from two images in which the subject is in a rotational relationship around one point in the image.

  In the above second embodiment, the case where the images are arranged in the horizontal direction has been described. However, the direction in which the images are arranged in parallel is not limited to the horizontal direction, and for example, the images can be arranged in the vertical direction. . The determination conditions when images are arranged in parallel in the vertical direction are the same as the determination conditions described in the third embodiment.

[Third Embodiment]
Next, a third embodiment will be described. The image processing apparatus according to the third embodiment is obtained by modifying a part of the processing by the image processing apparatus 100 according to the second embodiment.

FIG. 10 is a diagram illustrating an example of processing according to the third embodiment. The image 212 is an image obtained by translating the subject 111 included in the image 211 in the right direction.
It is assumed that feature points 211a, 211b, and 211c are extracted from the image 211, and feature points 212a, 212b, and 212c are extracted from the image 212. It is assumed that the feature point 211a and the feature point 212a, the feature point 211b, and the feature point 212b are associated with each other as a feature point pair, and the association is correct. On the other hand, the feature point 211c and the feature point 212c are associated as a feature point pair, but the association is incorrect. The feature points 211c and 212c exist at different coordinates only in the x coordinate.

  In the state where the images 211 and 212 are arranged in parallel in the horizontal direction as shown in the upper part of FIG. 10, the inclination calculated for the feature point pair (feature point 211c and feature point 212c) with incorrect association is a feature with which the association is correct. This is the same as the slope calculated for the feature point pair including the points. For this reason, the feature point pair including the feature point 211c and the feature point 212c is not determined as a feature point pair with an incorrect association even though the association is incorrect.

  On the other hand, in the state where the images 211 and 212 are arranged in parallel in the vertical direction as shown in the lower part of FIG. 10, the slope calculated for the feature point pair (feature point 211 c and feature point 212 c) whose association is incorrect is correlated It is different from the slope calculated for a feature point pair including correct feature points. For this reason, the feature point pair including the feature point 211c and the feature point 212c is not accurately determined as a feature point pair whose association is incorrect.

As described above, there may be a case where a feature point pair with an incorrect association cannot be accurately determined depending on the direction in which the images are arranged in parallel.
Therefore, in the third embodiment, the determination accuracy is improved by arranging the images in a plurality of directions and determining whether or not the feature points included in the feature point pairs are correctly associated in each parallel state. In this embodiment, it is assumed that each image is arranged in parallel in the vertical direction in addition to the horizontal direction.

  In the following description, components of the image processing apparatus according to the third embodiment are indicated using the same reference numerals as those of the image processing apparatus 100 according to the second embodiment. Descriptions of functions and configurations similar to those of the second embodiment are omitted.

First, in the case where images are arranged in the vertical direction, the relationship between the subject state of two images and the frequency information of inclination will be described.
FIG. 11 is a diagram illustrating an example of a histogram when the subject has a translational relationship. The image 214 is an image obtained by translating the subject 111 included in the image 213 in the lower right direction.

  It is assumed that feature points 213a, 213b, 213c, and 213d are extracted from the image 213, and feature points 214a, 214b, 214c, and 214d are extracted from the image 214. The feature points 213a and 214a, the feature points 213b and 214b, and the feature points 213c and 214c are associated with each other as feature point pairs, and the association is correct. On the other hand, the feature points 213d and 214d are associated as feature point pairs, but the association is incorrect.

  As shown in FIG. 11, in a state where the images 213 and 214 are arranged in the vertical direction, the slopes calculated for each feature point pair with the correct association are the same. On the other hand, since the slopes of feature point pairs such as feature point pairs of the feature points 213d and 214d that are not correctly associated are indefinite, the slopes calculated for the respective feature point pairs vary. Therefore, the inclination frequency calculated for the feature point pair in the above case is concentrated at one angle as in the histogram 251, and a remarkable peak appears in the frequency distribution.

  Therefore, when images are arranged in the vertical direction, if a noticeable peak appears in the distribution of the frequency of inclination, the feature point for which the inclination corresponding to the peak or the inclination included in a certain width around the peak is calculated The pair can be determined as a feature point pair including feature points that are correctly associated.

  Note that when images are arranged in the horizontal direction, the probability that the frequency distribution concentrates is near 0 °. On the other hand, when images are arranged in the vertical direction, the slope at which the frequency distribution is concentrated has a high probability of being around 270 °. As described above, when images are arranged in the vertical direction, the inclination at which the frequency distribution is concentrated is different from the case where the images are arranged in the horizontal direction.

  FIG. 12 is a diagram illustrating an example of a histogram when images are in a relationship of enlargement / reduction. The images 215 and 216 include the same subject 111. The image 216 is an enlarged image of the image 215.

  It is assumed that feature points 215a, 215b, 215c, and 215d are extracted from the image 215, and feature points 216a, 216b, 216c, and 216d are extracted from the image 216. It is assumed that the feature point 215a and the feature point 216a, the feature point 215b and the feature point 216b, the feature point 215c and the feature point 216c are associated with each other as a feature point pair, and the association is correct. On the other hand, the feature point 215d and the feature point 216d are associated as a feature point pair, but the association is incorrect.

  As shown in FIG. 12, in the state where the left ends of the images 215 and 216 are aligned and arranged in the vertical direction, the inclination calculated for each feature point pair with the correct association is in the range of 270 ° to 0 ° (360 °). Inside. On the other hand, since the inclination of the feature point pair in which the association such as the feature point pair of the feature points 215d and 216d is incorrect is indefinite, the inclination varies. Therefore, the inclination frequency calculated for the feature point pair in the above case is concentrated in the range of 270 ° to 0 ° as in the histogram 252. Therefore, when the images 215 and 216 are arranged in the vertical direction with the left ends aligned, if the distribution of inclination frequencies is concentrated in the range of 270 ° to 0 ° (360 °), the calculated inclination is within the range. It is presumed that the feature point pairs not included in are not correctly associated with each other.

  Note that in a state where the right ends of the images 215 and 216 are aligned and aligned in the vertical direction, the inclination calculated for each feature point pair with the correct association is in the range of 180 ° to 270 °. Therefore, when the images 215 and 216 are arranged in the vertical direction with their right ends aligned, if the distribution of inclination frequencies is concentrated in the range of 180 ° to 270 °, the calculated inclination is not included in the range. It is presumed that the feature points are not correctly associated with each other for the feature point pair.

  The above description is a case where the other image 216 is an enlargement of one image 215. For example, when the other image is a reduction of one image, the following determination is made. Is done. When each image is arranged in the vertical direction with the left edges aligned, if the distribution of the inclination frequency is concentrated in the range of 180 ° to 270 °, the feature point pair whose calculated inclination is not included in the range Is estimated that the feature points are not correctly associated with each other. On the other hand, when the images are arranged in the vertical direction with the right ends aligned, if the distribution of the inclination frequency is concentrated in the range of 270 ° to 0 ° (360 °), the calculated inclination is included in the range. It is presumed that the feature point pair is not correctly associated with each other.

  Although not shown, for example, a subject in an image arranged on the lower side rotates a subject in an image of the same size arranged on the upper side clockwise with respect to an outer point below the lower end of the image. In the case of the above, the inclination calculated for each feature point pair with the correct association is in the range of 270 ° to 0 ° (360 °). On the other hand, if the subject in the image placed on the lower side is obtained by rotating the subject in the same-size image placed on the upper side counterclockwise with respect to a point outside the lower end of the image, The slope calculated for each feature point pair with correct is in the range of 180 ° to 270 °.

  Also, if the subject in the image placed on the lower side is a subject in the same-size image placed on the upper side rotated counterclockwise with respect to the outer point above the upper end of the image, The inclination calculated for each feature point pair with correct attachment is in the range of 180 ° to 270 °. On the other hand, if the subject in the image placed on the lower side is a subject in the image of the same size placed on the upper side rotated clockwise with respect to the outer point above the upper end of the image The slope calculated for each correctly paired feature point is in the range of 270 ° to 0 ° (360 °).

  Therefore, when the images of the same size are arranged in the vertical direction, if the distribution of the inclination frequency is concentrated in one of the ranges of 270 ° to 0 ° or 0 ° to 90 °, the calculated inclination is within the range. It is presumed that the feature point pairs not included in are not correctly associated with each other. In this case, the inclination range for determining a feature point pair including feature points that are correctly associated with each other is different from that in the case of parallel arrangement in the horizontal direction.

  FIG. 13 is a diagram illustrating an example of a histogram when the subject is in a rotational relationship. The image 218 is an image obtained by rotating the subject 111 included in the image 217 around a point inside the image 217.

  It is assumed that feature points 217a, 217b, 217c, and 217d are extracted from the image 217, and feature points 218a, 218b, 218c, and 218d are extracted from the image 218. It is assumed that the feature point 217a and the feature point 218a, the feature point 217b and the feature point 218b, the feature point 217c and the feature point 218c are associated with each other as a feature point pair, and the association is correct. On the other hand, the feature point 217d and the feature point 218d are associated as a feature point pair, but the association is incorrect.

  As shown in FIG. 13, in a state where the images 217 and 218 are arranged in the vertical direction, the inclination calculated for each feature point pair with the correct association is 270 ° -n ° to 270 ° + n ° (where 0 < n <90). On the other hand, since the inclination of the feature point pair whose association is incorrect, such as the feature point pair of the feature points 217d and 218d, is indefinite, the inclination varies. Therefore, the inclination frequency calculated for the feature point pair in the above case is concentrated in the range of 270 ° -n ° to 270 ° + n ° as in the histogram 253. As in FIG. 8, the range that n can take is theoretically 0 <n <90, but in practice, the maximum value of n may be set to about 45 °.

  As described above, when images are arranged in the vertical direction, if the distribution of inclination frequencies is concentrated in the range of 270 ° -n ° to 270 ° + n °, the calculated inclination is not included in the range. A pair is estimated to be a feature point pair in which feature points are not correctly associated with each other. In this case, it differs from the case where it is arranged in parallel in the horizontal direction in that the range in which the inclination can be taken is 270 ° -n ° to 270 ° + n °, not 0 ° -n ° to 0 ° + n °.

  As shown in FIG. 11 to FIG. 13, in the state where images are arranged in the horizontal direction and the state where images are arranged in the vertical direction, in the determination of each feature point pair, it is determined as a feature point pair including feature points that are correctly associated with each other. The range of tilt to be different is different.

  FIG. 14 is a flowchart illustrating an example of image processing according to the third embodiment. The process described in FIG. 14 is executed between step S15 or step S19 described in FIG. 9 and step S20. In the following, the process illustrated in FIG. 14 will be described in order of step number.

  (Step S31) The feature point pair determination unit 140 selects the feature points included in the feature point pair for each feature point pair in a state where the images are arranged in a different direction (in this case, the vertical direction) from step S13. Calculate the slope of the connected straight line. At this time, the feature point pair determination unit 140 aligns the images so that the left end or the right end of each image matches in the vertical direction.

  The feature point pair determination unit 140 updates the tilt of the feature point pair in the feature point pair table 131 with the calculated tilt. Then, the feature point pair determination unit 140 generates frequency information based on information indicating the inclination frequency of each feature point pair in the feature point pair table 131.

  Note that, at the time of starting execution of step S31, the information on the feature point pair that has been determined that the association between the feature points is incorrect by step S15 or step S19 of FIG. 9 is deleted from the feature point pair table 131. Therefore, in step S31, since the inclination is calculated only for the remaining feature point pairs registered in the feature point pair table 131, the processing load is reduced.

  (Step S32) Similar to step S14 in FIG. 9, the feature point pair determination unit 140 determines whether the calculated inclination is concentrated on one angle based on the frequency information (that is, there is a significant peak in the frequency distribution). Judgment is made. If the inclination is concentrated at one angle, the process proceeds to step S33. If the inclination is not concentrated on one angle, the process proceeds to step S34.

  (Step S33) Similar to step S15 in FIG. 9, the feature point pair determination unit 140 sets the calculated slope within a certain range centered on the slope having the highest frequency (the slope corresponding to the prominent peak). For each of the included feature point pairs, it is determined that the association between the included feature points is correct. The feature point pair determination unit 140 deletes information about feature point pairs that have not been determined to be correct from the feature point pair table 131.

  (Step S34) The feature point pair determination unit 140 determines whether the frequency distribution is concentrated in the range of 180 ° to 270 ° based on the frequency information. When the frequency distribution is concentrated in the range, the feature point pair determination unit 140 sets the angle range to 180 ° to 270 °. Then, the process proceeds to step S37. On the other hand, if the frequency distribution is not concentrated in the range, the process proceeds to step S35.

Whether or not the frequency distribution is concentrated is determined in the same manner as in step S16 in FIG.
(Step S35) The feature point pair determination unit 140 determines whether the frequency distribution is concentrated in the range of 270 ° to 360 ° based on the frequency information. When the frequency distribution is concentrated in the range, the feature point pair determination unit 140 sets the angle range to 270 ° to 360 °. Then, the process proceeds to step S37. On the other hand, if the frequency distribution is not concentrated in the range, the process proceeds to step S36.

Whether or not the frequency distribution is concentrated is determined in the same manner as in step S17 in FIG.
(Step S36) The feature point pair determination unit 140 calculates a range in which the distribution of the inclination frequency is concentrated around 270 ° based on the frequency information. The calculation of the range is performed by the same procedure as step S18 in FIG. 9, where the sum of the frequencies in the range of 270 ° -n ° to 270 ° + n ° (n is an integer of 1 or more) is the sum of all frequencies. This can be realized by calculating n so as to be a predetermined ratio (for example, 50%). The range of n is theoretically 0 <n <90, but in practice it can be about 0 <n <45. The feature point pair determination unit 140 sets the angle range from 270 ° -n ° to 270 ° + n ° using the calculated value of n.

  (Step S37) The feature point pair determination unit 140 includes feature point pairs whose inclinations are included in the angle range set in any one of the steps S34 to S36 and that are correctly associated with each other. It is determined as a feature point pair. The feature point pair determination unit 140 deletes information about feature point pairs that have not been determined to be correct from the feature point pair table 131. Thereafter, the alignment process in step S20 of FIG. 9 is executed.

  In the above third embodiment, the calculation of the inclination for the feature point pair and the association between the feature points based on the calculation result are determined in a state where the images are arranged in parallel in a predetermined direction. Next, as a result of the determination, calculation of the slope of the straight line is performed in a state where each image is arranged in parallel in a direction different from the predetermined direction, with a feature point pair determined to have a correct correlation between the feature points as a processing target. And the said determination based on the calculation result is performed. Thereby, a feature point pair that cannot be determined to be invalid in a state where the images are arranged in a predetermined direction can be determined to be illegal. Therefore, as a whole, the accuracy of associating feature points can be improved.

  In the third embodiment described above, only the feature point pairs that are determined to have the correct correspondence between the feature points based on the determination in a state where the images are aligned in the horizontal direction cause the images to be aligned in the vertical direction. It was used for the judgment in the state. However, as another processing example, for all feature point pairs generated by the feature point pair calculation unit 120, determination in a state where images are arranged in parallel in the horizontal direction, and state in which images are arranged in parallel in the vertical direction Both of the determinations may be performed, and the feature point pairs that are determined to be correctly associated with each other by both determinations may be used for alignment.

  Note that at least a part of the processing executed by the image processing apparatus according to each of the above embodiments can be realized by causing a computer to execute a program. Such a program can be recorded on a computer-readable recording medium (for example, the recording medium 23). As the recording medium, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like can be used. Magnetic disks include FD and HDD. Optical discs include CD, CD-R (Recordable) / RW (Rewritable), DVD, and DVD-R / RW.

  When distributing the program, for example, a portable recording medium in which the program is recorded is provided. It is also possible to store the program in a storage device of another computer and distribute the program via the network 30. The computer stores, for example, a program recorded on a portable recording medium or a program received from another computer in a storage device (for example, HDD 103), and reads and executes the program from the storage device. However, a program read from a portable recording medium may be directly executed, or a program received from another computer via the network 30 may be directly executed. In addition, at least a part of the information processing described above can be realized by an electronic circuit such as a DSP, ASIC, or PLD (Programmable Logic Device).

1, 2 Images 1a, 1b, 1c, 1d, 2a, 2b, 2c, 2d Feature points 3 Histogram 10 Image processing device 11 Storage unit 12 Control unit

Claims (13)

The image processing device
In a state where the first image and the second image are arranged in parallel, each of the plurality of feature points extracted from the first image and each of the plurality of feature points extracted from the second image For each of a plurality of feature point pairs generated in a one-to-one correspondence, a slope of a straight line connecting the feature points included in each feature point pair is calculated,
Based on the calculated slope frequency, it is determined whether the correspondence between the feature points is correct for each of the plurality of feature point pairs.
An image processing method. In the determination, when the slope frequency is concentrated and distributed in a specific range at a certain ratio or more, a feature point in which the slope of the calculated straight line is included in the specific range among the plurality of feature point pairs. The pair is determined as a feature point pair in which the feature points are correctly associated with each other.
The image processing method according to claim 1. In the determination, when the inclination frequency is concentrated and distributed at a certain ratio at a specific angle, the characteristic point pair whose calculated straight line inclination is the specific angle among the plurality of characteristic point pairs. Is determined to be a feature point pair in which the correspondence between feature points is correct,
The image processing method according to claim 2. The specific range is determined according to a direction in which the first image and the second image are juxtaposed.
The image processing method according to claim 2. The specific range is a range of a constant angle in a positive direction or a negative direction from a reference angle determined according to a direction in which the first image and the second image are arranged in parallel.
The image processing method according to claim 4. When the first image and the second image are aligned in the horizontal direction with their upper ends or lower ends aligned, the reference angle is an angle along the horizontal direction.
The image processing method according to claim 5. When the first image and the second image are aligned in the vertical direction with their right or left ends aligned, the reference angle is an angle along the vertical direction.
The image processing method according to claim 5. The specific range is a range of a predetermined width around a reference angle determined according to a direction in which the first image and the second image are arranged in parallel, and the predetermined width is the inclination Is calculated so that the frequency of
The image processing method according to claim 4. The calculation of the inclination of the straight line and the determination based on the calculation result are executed a plurality of times while changing the direction in which the second image is juxtaposed with respect to the first image.
The image processing method according to claim 1, wherein the image processing method is an image processing method. As a result of performing the calculation of the inclination of the straight line and the determination based on the calculation result in a state where the second image is juxtaposed in the first direction with respect to the first image, the feature points are associated with each other. With the feature point pair determined to be correct as a processing target, the straight line inclination is calculated and the calculation result in a state where the second image is aligned in the second direction with respect to the first image. Performing said determination based on:
The image processing method according to claim 9. Performing the first image second image alignment of the first image and the second image based on the feature point pair determined by the determination that the correspondence between the feature points is correct;
The image processing method according to claim 1, wherein the image processing method is an image processing method. Information on a plurality of feature point pairs generated by associating each of a plurality of feature points extracted from the first image with each of a plurality of feature points extracted from the second image on a one-to-one basis is stored. A storage unit to
In a state where the first image and the second image are arranged in parallel, the inclination of a straight line connecting the feature points included in the feature point pair is calculated for each of the plurality of feature point pairs, and is calculated A control unit that determines whether or not the correspondence between the feature points is correct for each of the plurality of feature point pairs based on the frequency of inclination;
An image processing apparatus comprising: On the computer,
In a state where the first image and the second image are arranged in parallel, each of the plurality of feature points extracted from the first image and each of the plurality of feature points extracted from the second image For each of a plurality of feature point pairs generated in a one-to-one correspondence, a slope of a straight line connecting the feature points included in each feature point pair is calculated,
Based on the calculated slope frequency, it is determined whether the correspondence between the feature points is correct for each of the plurality of feature point pairs.
An image processing program for executing a process.

Images