JP6323009B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program.

  In Patent Document 1, it is an object to correct misalignment, rotation, and scaling of an image with a small amount of calculation without using a specific mark, and a pattern extraction unit generates a pattern in a predetermined size range from an input image. The same pattern extraction unit finds the same pattern as the pattern of the reference image, the corresponding point detection unit detects the centroid of the same pattern corresponding to the input image and the reference image, and the image correction unit detects the centroid of the corresponding pattern A correction formula is determined by calculating a shift between positions, a corrected image of the input image is generated according to the correction formula, and the character image is extracted from the corrected image based on the coordinates of the character image existing area of the reference image by the character image cutout unit. It is disclosed to cut out and recognize a character by a character recognition unit.

  Japanese Patent Laid-Open No. 2004-260260 has an object to perform quick and highly accurate alignment without recording a marker or the like together with a subject for alignment in a radiographic image alignment method. A reference region is set in the X-ray image b, and template matching is performed by matching the template region with the reference region using a correlation method. At this time, the common orthogonal coordinates are set in the sheet, and then the affine transformation is performed. The affine transformation that corrects the rotation / enlargement of the template area is performed, the template matching is performed again on the template area after the conversion, the affine transformation coefficient is obtained, and the rotation / enlargement / reduction and translation are corrected. An affine transformation is disclosed.

JP 09-245173 A Japanese Patent Laid-Open No. 06-165036

  The present invention determines a correction coefficient for alignment between the first image and the second image, and determines a correction coefficient with higher accuracy than when the present configuration is not provided. An object of the present invention is to provide an image processing apparatus and an image processing program.

The gist of the present invention for achieving the object lies in the inventions of the following items.
The first aspect of the present invention is a first extraction means for extracting a plurality of reference points in a first image, a second extraction means for extracting a plurality of reference points in a second image, and the first extraction. Detection means for detecting a reference point extracted by the second extraction means, which is a candidate corresponding to the reference point extracted by the means, and the first extraction based on the number of the detected reference points It is determined whether or not there is a reference point in the second image corresponding to the reference point extracted by the means, and if it is determined that there is a reference point, the reference that is a candidate detected by the detection means Among the points, associating means for associating the reference point in the second image corresponding to the reference point extracted by the first extracting means; the reference point in the associated first image; and In the second image Correction coefficient determining means for determining a correction coefficient for alignment between the first image and the second image based on a reference point at which the second extraction is performed. When there are a plurality of reference points in the first image corresponding to the reference points extracted by the means, one reference point in the second image is selected as a reference point in the second image. Is selected as a reference point corresponding to the reference point, a correction coefficient is determined, and the reference point is selected by referring to the association around the reference point and having the same positional relationship as the reference point in the second image. The image processing apparatus is characterized in that a reference point is selected so as to have the correspondence that it has.
According to a second aspect of the present invention, there is provided a first extraction unit that extracts a plurality of reference points in a first image, a second extraction unit that extracts a plurality of reference points in a second image, and the first extraction. Detection means for detecting a reference point extracted by the second extraction means, which is a candidate corresponding to the reference point extracted by the means, and the first extraction based on the number of the detected reference points It is determined whether or not there is a reference point in the second image corresponding to the reference point extracted by the means, and if it is determined that there is a reference point, the reference that is a candidate detected by the detection means Among the points, associating means for associating the reference point in the second image corresponding to the reference point extracted by the first extracting means; the reference point in the associated first image; and In the second image Correction coefficient determining means for determining a correction coefficient for alignment between the first image and the second image based on a reference point in which the matching means is based on the correction coefficient Performing alignment between the first image and the second image, and associating a reference point in the second image corresponding to the reference point extracted by the first extraction means that was not previously associated; The correction coefficient determining means associates a set of the reference point in the first image and the reference point in the second image associated with the first time by the associating means, and the second association by the associating means. The second correction coefficient is determined using a reference point in the first image and a set of reference points in the second image.

According to a third aspect of the present invention, the association means is extracted by the first extraction means when the number of reference points detected by the detection means is greater than or greater than a predetermined value. The reference point in the second image corresponding to the reference point is not associated, otherwise it is extracted by the first extraction means from the reference points that are candidates detected by the detection means. the image processing apparatus according to claim 1 or 2, characterized in that associating the reference point in the second image corresponding to the reference points.

According to a fourth aspect of the present invention, the association unit performs alignment between the first image and the second image based on the correction coefficient, and is extracted by the first extraction unit that has not been associated previously. associating a reference point in the second image corresponding to the reference points, the correction factor determining means is an image processing apparatus according to claim 1, wherein the determining a second correction coefficient .

According to a fifth aspect of the present invention, when there are a plurality of reference points in the first image corresponding to the reference points extracted by the second extracting means, the correction coefficient determining means (1) Determining a correction coefficient without using (2) selecting one reference point as a reference point corresponding to the reference point in the second image from among a plurality of reference points in the first image, and correcting The image processing apparatus according to claim 2, wherein one of the processes of determining a coefficient is performed.

According to a sixth aspect of the present invention, there is provided a computer comprising: a first extracting unit that extracts a plurality of reference points in a first image; a second extracting unit that extracts a plurality of reference points in a second image; Based on the number of the detected reference points, detection means for detecting the reference points extracted by the second extracting means, which are candidates corresponding to the reference points extracted by one extracting means, It is determined whether or not there is a reference point in the second image corresponding to the reference point extracted by one extraction unit, and if it is determined that there is a reference point, the candidate is detected by the detection unit. An association means for associating a reference point in the second image corresponding to the reference point extracted by the first extraction means, and a reference in the associated first image Point and Based on a reference point in the second image, it functions as a correction coefficient determination means for determining a correction coefficient for alignment between the first image and the second image, and the correction coefficient determination means includes: When there are a plurality of reference points in the first image corresponding to the reference points extracted by the second extracting means, one reference point is selected from the plurality of reference points in the first image. Selecting a reference point corresponding to the reference point in the image of the image, determining a correction coefficient, selecting the reference point with reference to the association around the reference point, and the positional relationship of the reference point in the second image The image processing program is characterized in that the reference point is selected so that the correspondence has the same positional relationship.
According to a seventh aspect of the present invention, there is provided a computer comprising: a first extracting unit that extracts a plurality of reference points in a first image; a second extracting unit that extracts a plurality of reference points in a second image; Based on the number of the detected reference points, detection means for detecting the reference points extracted by the second extracting means, which are candidates corresponding to the reference points extracted by one extracting means, It is determined whether or not there is a reference point in the second image corresponding to the reference point extracted by one extraction unit, and if it is determined that there is a reference point, the candidate is detected by the detection unit. An association means for associating a reference point in the second image corresponding to the reference point extracted by the first extraction means, and a reference in the associated first image Point and Based on a reference point in the second image, it functions as a correction coefficient determination means for determining a correction coefficient for alignment between the first image and the second image, and the association means A reference in the second image corresponding to the reference point extracted by the first extraction means that has not been previously matched is performed by aligning the first image and the second image based on a correction coefficient. Corresponding points, the correction coefficient determining means is a combination of a reference point in the first image and a reference point in the second image associated with the first time by the associating means, and the associating means. An image processing program for determining a correction coefficient for the second time using a reference point in the first image and a set of reference points in the second image associated with the second time.

According to the image processing device of claim 1, when determining a correction coefficient for alignment between the first image and the second image, compared to a case where the present configuration is not provided, A highly accurate correction coefficient can be determined.
According to the image processing device of claim 2, when determining a correction coefficient for alignment between the first image and the second image, compared to a case where the present configuration is not provided, A highly accurate correction coefficient can be determined.

According to the image processing device of claim 3 , if the reference point of the second image, which is a candidate corresponding to the reference point of the first image, is greater than or greater than a predetermined value, There may be no reference point to perform.

According to the image processing apparatus of the fourth aspect , when it is determined that there is no corresponding reference point, it is possible to perform the association after performing the alignment.

According to the image processing device of claim 5 , even when there are a plurality of reference points of the first image corresponding to the reference points of the second image, the first image and the second image are aligned. The correction factor for can be determined.

According to the image processing program of claim 6 , when determining the correction coefficient for the alignment of the first image and the second image, compared to the case where the present configuration is not provided, A highly accurate correction coefficient can be determined.
According to the image processing program of claim 7, when determining the correction coefficient for alignment between the first image and the second image, compared to the case where the present configuration is not provided, A highly accurate correction coefficient can be determined.

It is a conceptual module block diagram about the structural example of this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is explanatory drawing which shows the example of the image of the process target by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

この式の中で、ａ、ｂ、ｃ、ｄ、ｔ_ｘ、ｔ_ｙを、位置合わせのための補正係数という。ａ、ｂ、ｃ、ｄは拡大縮小、回転を示す補正係数であり、ｔ_ｘ、ｔ_ｙは移動を示す補正係数である。 The image processing apparatus 100 according to the present embodiment determines a correction coefficient for alignment between the first image and the second image. As shown in the example of FIG. 1, the reference extraction module 110A, a reference extraction module 110B, a reference candidate extraction module 120, a reference association module 130, and a correction coefficient calculation module 140.
The alignment between the first image and the second image is performed by performing geometric transformation represented by affine transformation on the image A of either the first image or the second image, and the other image. The coincidence overlap with B is maximized (different portions are minimized). Then, as a subsequent process, for example, the comparison between the image A and the image B after the affine transformation is performed, and the difference between the image A and the image B is extracted. For example, when the image B is an original and the image A is an image in which changes such as addition, deletion, and modification have been made to the original, the changed portion can be extracted.
The affine transformation is an expression shown in Expression (1). (X, y) is the coordinates of the image, and (x ′, y ′) is the coordinates of the image after affine transformation. That is, the affine transformation is performed by moving the (x, y) pixel in the image to the (x ′, y ′) position in the other image.

In this _{formula, a, b, c, d} , t x, the _{t y,} that the correction factor for alignment. a, b, c, and d are correction coefficients indicating enlargement / reduction and rotation, and t _x and _ty are correction coefficients indicating movement.

The reference extraction module 110A is connected to the reference candidate extraction module 120. The reference extraction module 110A extracts a plurality of reference points in the reference image 105A.
The reference extraction module 110B is connected to the reference candidate extraction module 120. The reference extraction module 110B extracts a plurality of reference points in the processed image 105B.
Note that the standard extraction module 110A and the standard extraction module 110B may be prepared as two modules that perform processing on the reference image 105A and the processed image 105B, respectively. It is also included in the case of a module that processes In that case, the reference image 105A and the processed image 105B are processed in time series, but either may be processed first.
The reference image 105A and the processed image 105B may be images that are independent of each other (an image that has nothing to do with each other), but in general, one of the images is an original image and the other is a change to the original image. It is the image which went. The original here is an original in a relative sense with the other, and means an image before the change is made. The image may be a binary image, a gray image, or a color image. Further, an image read by a scanner or the like, a document created by a document creation device or the like converted into an image, or the like may be used.

The reference point is a point used for alignment. Examples of reference points include the following.
(1) The center of gravity of each label extracted by the labeling process after the labeling process.
(2) Feature points. Specifically, for example, feature points (patterns) such as cross points, T-shaped points, and L-shaped points (including those rotated in units of 90 degrees) that are features forming a table are extracted as reference points. To do.
There are a plurality of reference points extracted from the reference image 105A and the processed image 105B. In the first place, the association processing by the reference association module 130 is not necessary with only one.

The reference candidate extraction module 120 is connected to the reference extraction module 110A, the reference extraction module 110B, and the reference association module 130. The reference candidate extraction module 120 detects a reference point extracted by the reference extraction module 110B, which is a candidate corresponding to the reference point extracted by the reference extraction module 110A. Note that the reference point extracted by the reference extraction module 110A is also referred to as “reference point of the reference image 105A”, and the reference point extracted by the reference extraction module 110B is also referred to as “reference point of the processed image 105B”.
The reference candidate extraction module 120 projects the corresponding reference point of the processed image 105B onto the reference image 105A, and extracts a reference point within a predetermined distance from the reference point of the reference image 105A as a candidate. There may be a plurality of candidates to be extracted. Of course, the corresponding reference point of the reference image 105A may be projected onto the processed image 105B. Further, as projection, for example, there is a case where the image is superimposed after movement in a predetermined direction and distance (including a case where the movement distance is 0, ie, the reference image 105A and the processed image 105B are directly superimposed).
In order to extract candidates as reference points for alignment, for example, the following method is available.
(1) Pattern matching (2) Feature quantity comparison As the feature quantity, for example, the coordinate position of the reference point, the size of the label, the number of black pixels at the reference point, the number of black pixels after thinning the reference point, the adjacent reference There are a distance to a point, a distance to two adjacent reference points, a point type (a type for distinguishing the above-described cross point, T-shaped point, L-shaped point, and the like). When the distance between two feature amounts in the feature space (the distance between the feature amount of the reference point of the reference image 105A and the feature amount of the reference point of the processed image 105B) is equal to or less than a predetermined distance, the processed image 105B Are extracted as candidates.

  The reference association module 130 is connected to the reference candidate extraction module 120 and the correction coefficient calculation module 140. Based on the number of reference points detected by the reference candidate extraction module 120, the reference association module 130 determines whether or not there is a reference point in the processed image 105B corresponding to the reference point extracted by the reference extraction module 110A. If it is determined that there is a reference point, the reference in the processed image 105B corresponding to the reference point extracted by the reference extraction module 110A from the reference points that are candidates detected by the reference candidate extraction module 120. Association is performed by selecting a point.

Specifically, if the number of reference points detected by the reference candidate extraction module 120 is greater than or equal to a predetermined value, the reference association module 130 extracts the reference extracted by the reference extraction module 110A. Reference points in the processed image 105B corresponding to the points are not associated (resulting in no corresponding reference points), otherwise reference points that are candidates detected by the reference candidate extraction module 120 The association is performed by selecting a reference point in the processed image 105B corresponding to the reference point extracted by the reference extraction module 110A.
In the case where there are a plurality of reference points of the processed image 105B that are candidates for a single reference point of the reference image 105A, if the number of candidates is greater than or equal to a predetermined value, the reference of the reference image 105A A point has no corresponding reference point. This is because it is difficult to determine which of the plurality of reference points of the processed image 105B corresponds, and is not suitable for the calculation of the correction coefficient by the correction coefficient calculation module 140.
If the number of candidates is less than a predetermined value, a corresponding reference point is extracted from the candidates by the following method.
(1) The distance between the coordinates of the two reference points is closest.
(2) The similarity obtained from the features of the two reference points (the reciprocal of the distance between the two reference points in the feature space) is the highest.

The correction coefficient calculation module 140 is connected to the reference association module 130. The correction coefficient calculation module 140 is a correction coefficient for aligning the reference image 105A and the processed image 105B based on the reference point in the reference image 105A and the reference point in the processed image 105B that are associated by the reference association module 130. To decide. An existing technique may be used to determine the correction coefficient.
In addition, when there are a plurality of reference points in the reference image 105A corresponding to the reference point extracted by the reference extraction module 110B, the correction coefficient calculation module 140 determines (1) a correction coefficient without using the association. (2) One of the plurality of reference points in the reference image 105A is selected as a reference point corresponding to the reference point in the processed image 105B, and a correction coefficient is determined. You may make it perform. This process will be described later using the example of FIG.

Further, the processes of the reference association module 130 and the correction coefficient calculation module 140 may be repeated. The second processing is performed as follows.
The reference association module 130 aligns the reference image 105A and the processed image 105B based on the correction coefficient determined by the correction coefficient calculation module 140, and did not associate the previous time (determines that there is no reference point corresponding to the previous time). Correlation may be performed by selecting a reference point in the processed image 105B corresponding to the reference point extracted by the reference extraction module 110A.
The correction coefficient calculation module 140 determines a correction coefficient for the second time. Specifically, the correction coefficient is determined using the reference point set by the first association by the reference association module 130 and the reference point set by the second association.
The second process will be described later using the example of FIG.

FIG. 2 is a flowchart showing an example of processing according to this embodiment.
In step S202, the standard extraction module 110A receives the reference image 105A. An example of the reference image 105A is shown in the example of FIG. For example, it is an image (original image) before correction.
In step S204, the reference extraction module 110A extracts a reference point for alignment from the reference image 105A. Here, a cross point, a T-shaped point, and an L-shaped point are extracted as reference points.
In step S206, the reference extraction module 110B receives the processed image 105B. An example of the processed image 105B is shown in the example of FIG. For example, it is a corrected image (an original image with a change).
In step S208, the reference extraction module 110B extracts a reference point for alignment from the processed image 105B. Here, a cross point, a T-shaped point, and an L-shaped point are extracted as reference points.
Note that either the set of steps S202 and S204 and the set of steps S206 and S208 may be performed first or may be performed in parallel.
FIG. 4 is an explanatory diagram showing a processing example according to the present embodiment. In the example of FIG. 4A, the reference image 105A and the processed image 105B are superimposed. In the example of FIG. 4B, the processing results (reference point extraction results) of step S204 and step S208 are reflected in the example of FIG. In the example of FIG. 4, the □ marks indicate the reference points of the reference image 105A. A circle represents a reference point of the processed image 105B.

In step S210, the reference candidate extraction module 120 selects a target reference point from the reference points of the reference image 105A.
In step S212, the standard candidate extraction module 120 extracts a standard correspondence candidate point corresponding to the target standard point in the reference image 105A from the standard point in the processed image 105B.
In step S214, the reference candidate extraction module 120 determines whether or not the number of extracted reference correspondence candidate points is equal to or less than a predetermined value. If it is equal to or less than a predetermined value, the process proceeds to step S216. In other cases, the process proceeds to step S218.
In step S216, the reference association module 130 performs association by selecting one corresponding reference point from among the reference correspondence candidate points.
In step S218, the reference association module 130 sets the corresponding reference point to “none” for the reference point that is the target of the reference image 105A.
In step S220, the criterion candidate extraction module 120 determines whether or not there remains a reference point in the reference image 105A for which the association processing has not been completed. If it remains, the process returns to step S210, otherwise In this case, the process proceeds to step S222.
In step S222, the correction coefficient calculation module 140 calculates a correction coefficient.

FIG. 5 is an explanatory diagram showing a processing example according to the present embodiment.
In the example shown in FIG. 5A, the processed image 105B is superimposed on the reference image 105A. The reference point 502 in the reference image 105A is set as a reference point.
In the example shown in FIG. 5B, the reference point 502 is projected onto the processed image 105B. Moving in a predetermined direction and distance, the reference point 502 is projected onto the projection reference point 504.
In the example shown in FIG. 5C, the reference points (reference correspondence candidate point 512, reference correspondence candidate point 514, reference correspondence candidate point 516) in the processed image 105B having similar feature quantities of the projection reference point 504 are candidates. As extracted. Here, a reference point that is within a predetermined distance from the projection reference point 504 and has the same point type (T-point) is extracted.
In the example shown in FIG. 5D, since the number of candidates is equal to or less than a predetermined value (three in this case), the reference corresponding candidate point having the closest distance from the candidate to the projection reference point 504. 512 is selected, and the reference point 502 and the reference correspondence candidate point 512 are associated with each other.

FIG. 6 is an explanatory diagram showing a processing example according to the present embodiment. This shows an example in which the number of candidates is greater than a predetermined value, unlike the example of FIG.
In the example shown in FIG. 6A, the reference image 105A is overlaid with the processed image 105B. The reference point 602 in the reference image 105A is set as a reference point.
In the example shown in FIG. 6B, the reference point 602 is projected onto the processed image 105B. Movement is performed in a predetermined direction and distance, and the reference point 602 is projected onto the projection reference point 604.
The example shown in FIG. 6C is a reference point (reference correspondence candidate point 612, reference correspondence candidate point 614, reference correspondence candidate point 616, reference correspondence in the processed image 105B in which the feature amount of the projection reference point 604 is similar. Candidate points 618) are extracted as candidates. Here, a reference point that is within a predetermined distance from the projection reference point 604 and has the same point type (cross point) is extracted.
In the example shown in FIG. 6D, there are no reference points corresponding to the projection reference point 604 (reference point 602) because the number of candidates is larger than a predetermined value (here, three). .

FIG. 7 is an explanatory diagram illustrating a processing example according to the present exemplary embodiment.
In the example shown in FIG. 7A, the processed image 105B is superimposed on the reference image 105A. A reference point 702 in the reference image 105A is set as a reference point.
In the example shown in FIG. 7B, the reference point 702 is projected onto the processed image 105B. Moving in a predetermined direction and distance, the reference point 702 is projected onto the projection reference point 704.
In the example shown in FIG. 7C, reference points (reference correspondence candidate point 712, reference correspondence candidate point 714, reference correspondence candidate point 716) in the processed image 105B having similar feature quantities of the projection reference point 704 are candidates. As extracted. Here, a reference point that is within a predetermined distance from the projection reference point 704 and has the same point type (T-point) is extracted.
In the example shown in FIG. 7D, since the number of candidates is equal to or less than a predetermined value (three in this case), the reference corresponding candidate point having the closest distance from the candidate to the projection reference point 704. 712 is selected, and the reference point 702 and the reference corresponding candidate point 712 are associated with each other.
In the example illustrated in FIG. 7E, the association is performed by repeating the processes from FIGS. 7A to 7D. Here, since the number of candidates is less than or equal to a predetermined value for the reference points for T-shaped points and L-shaped points (reference points on the outer periphery of the table), the association is performed and the reference points for the cross points are determined. Indicates that the number of candidates is larger than a predetermined value, so that the association is not performed. That is, the reference points associated by the first process are shown.

And the correction coefficient calculation module 140 calculates the correction coefficient of Formula (1) shown in FIG.7 (f) using the group of the reference points with which matching was performed. The processing after FIG. 7G is the second processing.
In the example shown in FIG. 7G, a reference point that is a reference point that has not been associated in the first process is set as a target point. In this example, a reference point 722 that is a reference point of the cross point is used.
In the example shown in FIG. 7H, the reference image 105A and the processed image 105B are aligned (projected) using Expression (1) shown in FIG. 7F. The reference point 722 moves to the position of the projection reference point 724.
In the example illustrated in FIG. 7I, a reference correspondence candidate point 732, a reference correspondence candidate point 734, a reference correspondence candidate point 736, and a reference correspondence candidate point 738, which are candidates corresponding to the projection reference point 724, are extracted. Note that what is extracted by the reference candidate extraction module 120 in the first process may be used as it is. The reference point 722 (projection reference point 724) and the reference correspondence candidate point 734 are associated with each other by selecting the reference correspondence candidate point 734 that is closest to the projection reference point 724 as the reference point to be associated. Here, the second association is performed without comparing the number of candidates with a predetermined value. This is because the alignment is performed after the correction coefficient has already been calculated. Therefore, even if the number of candidates is large, the number of associations causing noise is small.
Thereafter, using the first association result and the second association result, the correction coefficient calculation module 140 calculates a correction coefficient. The correction coefficient is calculated using more correlation results than the correction coefficient is calculated using the first correlation result.

FIG. 8 is an explanatory diagram showing a processing example according to the present embodiment. This is a process performed by the correction coefficient calculation module 140. As a result of the association by the reference association module 130, a plurality of reference points in the reference image 105A correspond to one reference point in the processed image 105B. The processing when there are
With reference to the example of FIG. 8A, a case where “a plurality of reference points in the reference image 105A corresponding to the reference points extracted by the reference extraction module 110B are present” will be described. The example of FIG. 8A shows the associated reference points. Here, the reference point 802 and the corresponding reference point 812 are associated with each other, and the reference point 804 is also associated with the corresponding reference point 812. Thus, the fact that a plurality of reference points in the reference image 105A are associated with one reference point in the processed image 105B indicates that any of the associations is incorrect. Therefore, the association is not preferable for calculating the correction coefficient. In order to eliminate this unfavorable association, two solutions are presented.

The first solution will be described. That is, "(1) determining a correction coefficient without using the association" will be described with reference to the example of FIG. The example of FIG. 8B shows that the reference points having the wrong association are not used. In this example, the association between the reference point 802 and the corresponding reference point 812 and the correspondence with the reference point 804 are performed. The correction coefficient calculation module 140 calculates the correction coefficient without using the association with the reference point 812. Therefore, the correction coefficient is calculated using the remaining association result.
The second solution will be described. That is, using the example of FIG. 8C, “(2) Of the plurality of reference points in the reference image 105A, one reference point is selected as the reference point corresponding to the reference point in the processed image 105B, and the correction coefficient is selected. “Determining” will be described. One association is used, but the other association is excluded. That is, the reference points have a one-to-one correspondence.
Which of the plurality of reference points in the reference image 105A is selected may be as follows.
(1) The reference point of the reference image 105A to be selected is selected based on the reference points having a one-to-one correspondence around the target association. For example, the reference point of the reference image 105A to be matched is selected so that the positional relationship between the reference point of the reference image 105A and the reference point of the corresponding processed image 105B is the same. More specifically, referring to the association between the reference point 802 and the corresponding reference point 812 and the association around the association between the reference point 804 and the corresponding reference point 812, the reference point of the reference image 105A is on the upper left. Since the reference point of the processed image 105B has a positional relationship that it is in the lower right, the association between the reference point 802 and the corresponding reference point 812 having the same positional relationship is selected, The correction coefficient calculation module 140 calculates the correction coefficient by using the correlation (excluding the correlation of the reference point 804).
(2) Of the reference points of the reference image 105A, select the one with the closest coordinate distance from the reference point of the corresponding processed image 105B.
(3) Among the reference points of the reference image 105A, the one having the highest similarity with the reference point of the corresponding processed image 105B is selected.

  A hardware configuration example of the image processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 9 is configured by, for example, a personal computer (PC), and illustrates a hardware configuration example including a data reading unit 917 such as a scanner and a data output unit 918 such as a printer.

  A CPU (Central Processing Unit) 901 includes various modules described in the above-described embodiments, that is, the reference extraction module 110A, the reference extraction module 110B, the reference candidate extraction module 120, the reference association module 130, and the correction coefficient calculation module 140. The control unit executes processing according to a computer program describing an execution sequence of each module.

  A ROM (Read Only Memory) 902 stores programs used by the CPU 901, operation parameters, and the like. A RAM (Random Access Memory) 903 stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 904 including a CPU bus.

  The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 905.

  A keyboard 908 and a pointing device 909 such as a mouse are input devices operated by an operator. The display 910 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 911 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 901 and information. The hard disk stores a reference image 105A, a processed image 105B, a correction coefficient, reference point information, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 912 reads out data or a program recorded in a removable recording medium 913 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out from the interface 907 and the external bus 906. , To the RAM 903 connected via the bridge 905 and the host bus 904. The removable recording medium 913 can also be used as a data recording area similar to a hard disk.

  The connection port 914 is a port for connecting the external connection device 915 and has a connection unit such as USB or IEEE1394. The connection port 914 is connected to the CPU 901 and the like via the interface 907, the external bus 906, the bridge 905, the host bus 904, and the like. The communication unit 916 is connected to a communication line and executes data communication processing with the outside. The data reading unit 917 is a scanner, for example, and executes document reading processing. The data output unit 918 is, for example, a printer, and executes document data output processing.

  Note that the hardware configuration of the image processing apparatus shown in FIG. 9 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 9, and the modules described in the present embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 9 may be connected to each other via communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (an image processing apparatus having any two or more functions of a scanner, a printer, a copying machine, a fax machine, etc.).

Further, in the description of the above-described embodiment, “more than”, “less than”, “greater than”, and “less than (less than)” in a comparison with a predetermined value contradicts the combination. As long as the above does not occur, “larger”, “smaller (less than)”, “more than”, and “less than” may be used.
Note that the technology described in the background art may be adopted as the processing content of each module.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 105A ... Reference image 105B ... Processed image 110 ... Reference extraction module 110A ... Reference extraction module 110B ... Reference extraction module 120 ... Reference candidate extraction module 130 ... Reference association module 140 ... Correction coefficient calculation module

Claims (7)

First extraction means for extracting a plurality of reference points in the first image;
Second extraction means for extracting a plurality of reference points in the second image;
Detection means for detecting a reference point extracted by the second extraction means, which is a candidate corresponding to the reference point extracted by the first extraction means;
Based on the number of detected reference points, it is determined whether there is a reference point in the second image corresponding to the reference point extracted by the first extraction means, and it is determined that there is a reference point. In this case, the association means for associating the reference point in the second image corresponding to the reference point extracted by the first extraction means from among the reference points that are candidates detected by the detection means. When,
Correction for determining a correction coefficient for alignment between the first image and the second image based on the reference point in the associated first image and the reference point in the second image Coefficient determination means
When there are a plurality of reference points in the first image corresponding to the reference point extracted by the second extracting unit, the correction coefficient determining unit determines that 1 out of the plurality of reference points in the first image. Selecting one reference point as the reference point corresponding to the reference point in the second image, determining a correction factor,
The reference point is selected by referring to the association around the reference point and selecting the reference point so that the association has the same positional relationship as the positional relationship of the reference point in the second image. An image processing apparatus. First extraction means for extracting a plurality of reference points in the first image;
Second extraction means for extracting a plurality of reference points in the second image;
Detection means for detecting a reference point extracted by the second extraction means, which is a candidate corresponding to the reference point extracted by the first extraction means;
Based on the number of detected reference points, it is determined whether there is a reference point in the second image corresponding to the reference point extracted by the first extraction means, and it is determined that there is a reference point. In this case, the association means for associating the reference point in the second image corresponding to the reference point extracted by the first extraction means from among the reference points that are candidates detected by the detection means. When,
Correction for determining a correction coefficient for alignment between the first image and the second image based on the reference point in the associated first image and the reference point in the second image Coefficient determination means
The association means aligns the first image and the second image based on the correction coefficient, and corresponds to the reference point extracted by the first extraction means that was not previously associated. Associating a reference point in the second image;
The correction coefficient determining means associates a set of the reference point in the first image and the reference point in the second image associated with the first time by the associating means, and the second association by the associating means. A correction coefficient for the second time is determined using a set of reference points in the first image and a set of reference points in the second image. The association means, when the number of reference points detected by the detection means is greater than or greater than a predetermined value, the first corresponding to the reference points extracted by the first extraction means. The reference points in the second image are not associated with each other, and in other cases, the reference points extracted by the first extraction unit are selected from the reference points that are candidates detected by the detection unit. The image processing apparatus according to claim 1, wherein a reference point in the second image is associated. The association means aligns the first image and the second image based on the correction coefficient, and corresponds to the reference point extracted by the first extraction means that was not previously associated. Associating a reference point in the second image;
The image processing apparatus according to claim 1, wherein the correction coefficient determination unit determines a second correction coefficient. When there are a plurality of reference points in the first image corresponding to the reference point extracted by the second extraction unit, the correction coefficient determination unit determines (1) the correction coefficient without using the association. (2) selecting one reference point as a reference point corresponding to the reference point in the second image among a plurality of reference points in the first image, and determining a correction coefficient; The image processing apparatus according to claim 2, wherein one of the processes is performed. Computer
First extraction means for extracting a plurality of reference points in the first image;
Second extraction means for extracting a plurality of reference points in the second image;
Detection means for detecting a reference point extracted by the second extraction means, which is a candidate corresponding to the reference point extracted by the first extraction means;
Based on the number of detected reference points, it is determined whether there is a reference point in the second image corresponding to the reference point extracted by the first extraction means, and it is determined that there is a reference point. In this case, the association means for associating the reference point in the second image corresponding to the reference point extracted by the first extraction means from among the reference points that are candidates detected by the detection means. When,
Correction for determining a correction coefficient for alignment between the first image and the second image based on the reference point in the associated first image and the reference point in the second image Function as a coefficient determination means,
When there are a plurality of reference points in the first image corresponding to the reference point extracted by the second extracting unit, the correction coefficient determining unit determines that 1 out of the plurality of reference points in the first image. Selecting one reference point as the reference point corresponding to the reference point in the second image, determining a correction factor,
The reference point is selected by referring to the association around the reference point and selecting the reference point so that the association has the same positional relationship as the positional relationship of the reference point in the second image. An image processing program characterized by: Computer
First extraction means for extracting a plurality of reference points in the first image;
Second extraction means for extracting a plurality of reference points in the second image;
Detection means for detecting a reference point extracted by the second extraction means, which is a candidate corresponding to the reference point extracted by the first extraction means;
Based on the number of detected reference points, it is determined whether there is a reference point in the second image corresponding to the reference point extracted by the first extraction means, and it is determined that there is a reference point. In this case, the association means for associating the reference point in the second image corresponding to the reference point extracted by the first extraction means from among the reference points that are candidates detected by the detection means. When,
Correction for determining a correction coefficient for alignment between the first image and the second image based on the reference point in the associated first image and the reference point in the second image Function as a coefficient determination means,
The association means aligns the first image and the second image based on the correction coefficient, and corresponds to the reference point extracted by the first extraction means that was not previously associated. Associating a reference point in the second image;
The correction coefficient determining means associates a set of the reference point in the first image and the reference point in the second image associated with the first time by the associating means, and the second association by the associating means. A second correction coefficient is determined using a set of reference points in the first image and a set of reference points in the second image.

Images