JPH10124685A - Image processing method and storage medium and image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing method and device in which robustness against noise is strong, the erroneous detection of a corresponding point is prevented, and image positioning with high precision and practicality can be attained. SOLUTION: At the time of carrying out an image processing for extracting plural corresponding points between two images, and calculating a conversion expression between the two images by using the deviation of the extracted corresponding points, the plural corresponding points are extracted between the two images by a corresponding point extracting means 102, an individual expansion and contraction ratio between the two images is calculated for each group of the plural corresponding points by an individual expansion and contraction ratio calculating means 103, the corresponding points which are likely to be included in the group from which the abnormal expansion and contraction ratio is calculated are eliminated from the plural corresponding points by a first corresponding point removing means 104, and a conversion expression between the two images is calculated by using the corresponding points left as the processed result of the first corresponding point removing means 104 by the deviation of the pertinent corresponding points by a conversion expression calculating means 107.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method such as image recognition and character recognition in a fax
According to an image processing apparatus such as an AXOCR apparatus, a plurality of corresponding points are extracted by using a timing mark or the like between two images such as a form and a document image having a position shift, rotation, and scaling. In the case of calculating the conversion formula between the two images using the shift of the corresponding point and performing alignment between the two images, robustness against noise is strong, and erroneous detection of the corresponding point by a timing mark or the like is prevented. The present invention relates to an image processing method and an image processing apparatus capable of performing high-precision positioning, and performing practical image positioning.

Here, the FAXOCR device cuts out necessary description items from an image sent by facsimile, performs character recognition processing, and outputs it as data. In particular, here, the FAXOCR device is sent by facsimile. After taking in an image such as a form and aligning the image (input image) with the same type of reference image as image recognition processing,
An example in which necessary description items in an input image are cut out, subjected to character recognition processing, and output as character data is handled.

[0003]

2. Description of the Related Art As a known technique using a mark or the like in a method of aligning images in a FAX OCR apparatus or the like, a document scanning apparatus disclosed in Japanese Patent Publication No. 53-48059 is disclosed.
(First conventional example) and "character recognition device" (second conventional example) disclosed in Japanese Patent Publication No. Hei 7-101438. In these conventional examples, a specific mark is printed on a form, and the image is aligned by detecting the specific mark on the image.

However, the first conventional "document scanning apparatus" has the following problems. That is, first, when the reference mark is singular, the reference mark is susceptible to noise and the like, second, the accuracy of tilt detection depends on the size of the reference mark, and third, erroneous detection of the reference mark. It is a problem that there is a high possibility of doing this.

Further, in the embodiment described in the "character recognition apparatus" of the second conventional example, since a plurality of timing marks are used, accuracy and robustness against noise are improved. The third problem of erroneous mark detection remains.

As a method of detecting a positional shift of an image by using a large number of corresponding points, not a document image such as a FAXOCR apparatus but a known technique in the field of medical image processing disclosed in Japanese Patent Application Laid-Open No. Sho 59-218753. There is a disclosed "image registration method" (third conventional example). In this conventional example, the vector of the image displacement is detected for each block,
A method is disclosed in which a singular point having an extremely different vector direction is removed and image alignment is performed.

The method of the third conventional example is called FAXO
When applied to a CR device or the like, it is possible to solve the third problem of erroneous detection of a reference mark, which has been concerned in the "document scanning device" of the first conventional example and the "character recognition device" of the second conventional example. Can be.

[0008] However, in the case of performing image registration using a timing mark in a FAXOCR device or the like, as described in the embodiment described in "Image Registration System" of the third conventional example, It is difficult to use many reference points (marks). That is, if image alignment is performed using many timing marks, the degree of freedom in form design is reduced, which is not practical.

As a conventional technique for correcting not only image displacement and skew but also expansion and contraction by using timing marks, an image processing apparatus disclosed in Japanese Patent Application Laid-Open No. 63-212269 (fourth conventional example) is known. ). In the fourth conventional example, an image expansion / contraction ratio is calculated from a distance between two corresponding marks to correct the image.

[0010]

As described above, in the image processing apparatus of the first conventional example, when the number of reference marks is singular, the size of the reference mark is weak against noise on the reference mark. The accuracy of tilt detection depends on
There is a problem that there is a high possibility that the reference mark is erroneously detected.

In the second conventional image processing apparatus, the accuracy and robustness to noise are improved, but the problem that the reference mark is likely to be erroneously detected is solved. I didn't.

Further, in the image processing apparatus of the third conventional example, many reference points (marks) are required, and image registration using many timing marks is not easy for form design. And it is not practical.

Further, in the image processing apparatus of the fourth prior art, the expansion and contraction ratio of the image is calculated from the distance between the corresponding two marks, and the image is corrected, but the distance between the marks is short. In some cases, there is a problem that the error of the calculated expansion / contraction ratio is large and the mark may be erroneously detected.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned conventional problems, and has been developed in consideration of the use of a timing mark or the like between two images such as a form and a document image having a position shift, rotation, and magnification change. In the case of extracting the corresponding points of, calculating the conversion formula between the two images using the shift of the extracted corresponding points, and performing alignment between the two images, robustness against noise is strong and timing An object of the present invention is to provide an image processing method and an image processing apparatus which can prevent erroneous detection of a corresponding point due to a mark or the like, can perform high-accuracy alignment, and can perform practical image alignment.

[0015]

According to a first aspect of the present invention, there is provided an image processing method comprising: extracting a plurality of corresponding points between two images; A corresponding point extracting step of extracting a plurality of corresponding points between the two images, the method comprising: calculating a conversion equation between the two images by using a plurality of corresponding points; An individual expansion / contraction ratio calculating step of calculating an individual expansion / contraction ratio between the two, and a first corresponding point removing step of deleting from the plurality of corresponding points corresponding points which are often included in a group in which an abnormal individual expansion / contraction ratio is calculated. , A conversion formula calculating step of calculating a conversion formula between the two images by using the corresponding points remaining as a result of the first corresponding point removing step based on a shift of the corresponding points.

[0016] The image processing method according to claim 2 is characterized in that:
In an image processing method for extracting a plurality of corresponding points between two images and calculating a conversion equation between the two images using a shift of the extracted corresponding points, extracting a plurality of corresponding points between the two images A corresponding point extracting step, an expanding / contracting ratio calculating step of calculating an expanding / contracting ratio between the two images by using all the extracted corresponding points,
An individual expansion / contraction ratio calculating step of calculating an individual expansion / contraction ratio between two images, and the plurality of correspondences so that only corresponding points which are often included in a group of corresponding points for which an individual expansion / contraction ratio close to the above expansion / contraction ratio is calculated remain. Using a corresponding point remaining as a result of the second corresponding point removing step to remove a corresponding point from a point, and calculating a conversion equation between the two images based on a shift of the corresponding point using the corresponding point. And steps.

The image processing method according to the third aspect is characterized in that:
In an image processing method for extracting a plurality of corresponding points between two images and calculating a conversion equation between the two images using a shift of the extracted corresponding points, extracting a plurality of corresponding points between the two images Corresponding point extraction step, an individual expansion / contraction ratio calculating step of calculating an individual expansion / contraction ratio between the two images for each group of a plurality of corresponding points, and a case where an abnormal individual expansion / contraction ratio is calculated in many cases. A first corresponding point removing step of deleting a corresponding point from the plurality of corresponding points;
A stretching ratio calculating step of calculating a stretching ratio between the two images using all the corresponding points remaining as a result of the corresponding point removing step, and a corresponding point group for which an individual stretching ratio close to the stretching ratio is calculated. A second corresponding point removing step of removing from the plurality of corresponding points so that only corresponding points that are often included remain, and using the corresponding points remaining as a result of the second corresponding point removing step, the corresponding points are removed. A conversion formula calculating step of calculating a conversion formula between the two images based on the shift.

According to a fourth aspect of the present invention, in the image processing method of the first, second or third aspect, the threshold value for determining an abnormal individual expansion / contraction ratio in the first corresponding point removing step. Alternatively, in the second corresponding point removing step, the threshold value for determining the closeness of the individual expansion / contraction ratio with respect to the expansion / contraction ratio is made variable in accordance with the distance between the corresponding points.

According to a fifth aspect of the present invention, there is provided an image processing method as defined in the first, second, third or fourth aspect.
The distance between the corresponding point remaining at the time of the conversion formula calculation step is between the corrected image obtained by the conversion formula calculated in the conversion formula calculation step and one of the two images. A third corresponding point removing step of calculating and removing a corresponding point having a large distance from the remaining corresponding points, and using the corresponding points remaining as a result of the third corresponding point removing step, using the corresponding point shift and And a conversion formula recalculating step of calculating a conversion formula between the corrected image and the one image.

According to a sixth aspect of the present invention, in the image processing method of the fifth aspect, the step of recalculating the conversion formula and the step of removing the third corresponding point include the third step.
The corresponding point removing step is repeatedly performed while the maximum value or average value of the distance between the corresponding points exceeds a predetermined threshold value or while there is a corresponding point to be removed.

According to a seventh aspect of the present invention, in the image processing method according to the first, second, third, fourth, fifth, or sixth aspect, the corresponding points extracted in the corresponding point extracting step are corrected. Is a timing mark.

A computer-readable storage medium according to claim 8 is a computer-readable storage medium.
The image processing method described in 5, 6, or 7 is stored as a program for causing a computer to execute the method.

The image processing apparatus according to the ninth aspect is characterized in that:
Extracting a plurality of corresponding points between the two images, calculating a conversion formula between the two images by using a shift of the extracted corresponding points,
In an image processing apparatus for performing registration between the two images, a corresponding point extracting means for extracting a plurality of corresponding points between the two images, and an individual expansion and contraction between the two images for each group of a plurality of corresponding points An individual expansion / contraction ratio calculating means for calculating a ratio,
First corresponding point removing means for deleting corresponding points, which are often included in a group in which an abnormal individual expansion / contraction ratio is calculated, from the plurality of corresponding points, and a correspondence remaining as a processing result of the first corresponding point removing means And a conversion formula calculating means for calculating a conversion formula between the two images based on the shift of the corresponding point using the points.

Further, the image processing apparatus according to claim 10 is
An image processing apparatus that extracts a plurality of corresponding points between two images, calculates a conversion formula between the two images using a shift of the extracted corresponding points, and performs registration between the two images. A corresponding point extracting unit for extracting a plurality of corresponding points between the two images, a stretching ratio calculating unit for calculating a stretching ratio between the two images using all the extracted corresponding points,
An individual expansion / contraction ratio calculating means for calculating an individual expansion / contraction ratio between the two images for each group of a plurality of corresponding points; A second corresponding point removing means for removing from the plurality of corresponding points so that only the points remain, and a corresponding point remaining as a processing result of the second corresponding point removing means, the two corresponding points being shifted by the corresponding point. And a conversion formula calculating means for calculating a conversion formula between images.

An image processing apparatus according to claim 11 is
An image processing apparatus that extracts a plurality of corresponding points between two images, calculates a conversion formula between the two images using a shift of the extracted corresponding points, and performs registration between the two images. Corresponding point extracting means for extracting a plurality of corresponding points between the two images;
An individual expansion / contraction ratio calculating means for calculating an individual expansion / contraction ratio between two images, and a first corresponding point removal for deleting a corresponding point which is often included in a group in which an abnormal individual expansion / contraction ratio is calculated from the plurality of corresponding points. Means, an expansion / contraction ratio calculating means for calculating an expansion / contraction ratio between the two images using all the corresponding points remaining as a processing result of the first corresponding point removing means, and an individual expansion / contraction ratio close to the expansion / contraction ratio is calculated. Second corresponding point removing means for removing from the plurality of corresponding points so that only corresponding points which are often included in the group of corresponding points to be removed, and corresponding points remaining as processing results of the second corresponding point removing means And a conversion formula calculating means for calculating a conversion formula between the two images based on the shift of the corresponding point.

An image processing apparatus according to claim 12 is
12. The image processing apparatus according to claim 9, 10 or 11, wherein a threshold value for judging an abnormal individual expansion / contraction rate in said first corresponding point removing means or an individual value for an expansion / contraction rate in said second corresponding point removing means. The threshold value for determining the closeness of the expansion / contraction ratio is made variable according to the distance between corresponding points.

The image processing apparatus according to claim 13 is:
13. The image processing apparatus according to claim 9, 10, 11 or 12, wherein the image processing apparatus includes a correction image obtained by the conversion formula calculated by the conversion formula calculation unit and one of the two images. , The distance between corresponding points remaining at the time of processing by the conversion formula calculating means is calculated, and a corresponding point having a larger distance is removed from the remaining corresponding points.
Using a corresponding point removing unit and a corresponding point remaining as a processing result of the third corresponding point removing unit, a conversion formula recalculation for calculating a conversion formula between the corrected image and the one image based on a shift of the corresponding point. Means.

The image processing apparatus according to claim 14 is:
14. The image processing apparatus according to claim 13, wherein the processing in the conversion formula recalculating unit and the third corresponding point removing unit includes:
In the third corresponding point removing means, while the maximum value or the average value of the distance between the corresponding points exceeds a predetermined threshold value,
Alternatively, it is repeatedly performed while the corresponding point to be removed exists.

An image processing apparatus according to claim 15 is
15. The image processing apparatus according to claim 9, wherein the corresponding points extracted by the corresponding point extracting means are used as correction timing marks.

An image processing apparatus according to claim 16 is
16. The image processing device according to claim 9, 10, 11, 12, 13, 14, or 15, wherein the image processing device outputs a corrected image using a conversion formula calculated by the conversion formula calculation unit or the conversion formula recalculation unit. The image processing apparatus further includes a character cutout unit that generates and cuts out a character image from the corrected image based on character data in a character image existing area given in advance for one of the two images.

The image processing device according to claim 17 is:
17. The image processing device according to claim 16, wherein the image processing device includes a character recognition unit that recognizes a character image obtained by the character cutout unit.

Further, the image processing apparatus according to claim 18 is:
Claims 9, 10, 11, 12, 13, 14, 15, 16
Alternatively, in the image processing apparatus according to Item 17, the other of the two images is an image input via a facsimile.

[0033]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The outline of an image processing method and an image processing apparatus according to the present invention and an embodiment of the image processing method and the image processing apparatus according to the present invention will be described in detail with reference to the drawings. .

[Outline of Image Processing Method and Image Processing Apparatus of the Present Invention] In the image processing method and the image processing apparatus according to the present invention, a plurality of corresponding points are extracted between two images. Then, when performing image processing such as calculating a conversion equation between two images by using the shift of the extracted corresponding points, as shown in FIG. Are extracted (corresponding point extraction step), and then the individual expansion / contraction ratio calculating means 103 calculates an individual expansion / contraction ratio between the two images for each group of a plurality of corresponding points (individual expansion / contraction ratio calculating step). Next, the first corresponding point removing means 104 deletes, from the plurality of corresponding points, corresponding points which are often included in the group in which an abnormal individual expansion / contraction ratio is calculated (first corresponding point removing step), and further performs conversion. By the equation calculation means 107 Using the corresponding points remaining as the processing result of the first corresponding point removing unit 104, to calculate the conversion formula between the two images (conversion equation calculation step) by displacement of the corresponding points.

As described above, since a plurality of corresponding points are used, even when some corresponding points cannot be detected, it is possible to position the two images, and the robustness against noise is high. Further, for example, when the corresponding points of erroneous extraction are used as a reference, the number of combinations of corresponding points for which an abnormally large or abnormally small individual expansion / contraction ratio is calculated increases, and a high-accuracy image between two images can be obtained. Although it is difficult to perform positioning, as in the present invention, the corresponding points that are often included in the group in which the abnormal individual expansion / contraction rate is calculated are deleted from a plurality of corresponding points, so that the possible individual expansion / contraction rate can be roughly calculated. The range can be determined, the influence of the corresponding point of the erroneous extraction can be suppressed, and highly accurate positioning between the two images can be performed. Further, since the number of corresponding points is in a practical range as compared with the third conventional example, the degree of freedom in form design and the like can be increased.

Further, in the image processing method according to the second aspect and the image processing apparatus according to the tenth aspect, a plurality of corresponding points are extracted between two images, and a difference between the extracted corresponding points is used to extract two or more corresponding points.
When performing image processing such as calculating a conversion equation between two images, as shown in FIG. 1, first, a plurality of corresponding points are extracted between two images by a corresponding point extracting means 102 (corresponding point extracting step). Next, by the individual expansion / contraction rate calculation means 103,
The individual scaling ratio between the two images is calculated for each group of a plurality of corresponding points (individual scaling ratio calculating step).
Calculating the expansion and contraction ratio between the two images (expansion and contraction ratio calculation step),
Next, the second corresponding point removing unit 106 removes the plurality of corresponding points from the plurality of corresponding points so that only the corresponding points that are often included in the corresponding point group for which the individual expansion / contraction ratio close to the expansion / contraction ratio is calculated remain (the second corresponding point). (2 corresponding point removing step), and further using the corresponding points remaining as the processing result of the second corresponding point removing means 106 by the conversion formula calculating means 107,
Calculate conversion formula between two images (conversion formula calculation step)
Like that.

As described above, since a plurality of corresponding points are used, even when some corresponding points cannot be detected, it is possible to position the two images, and the robustness against noise is high. Also, for example, even when the corresponding points of the erroneous extraction are used as a reference, only the corresponding points that are often included in the group of corresponding points for which the individual expansion and contraction rates close to the expansion and contraction rate are calculated as in the present invention remain. By removing them, it is possible to determine the approximate range of possible individual expansion and contraction ratios, suppress the effects of erroneous extraction corresponding points, and achieve highly accurate alignment between two images. Become. Further, since the number of corresponding points is in a practical range as compared with the third conventional example, the degree of freedom in form design and the like can be increased.

In the image processing method according to the third aspect and the image processing apparatus according to the eleventh aspect, a plurality of corresponding points are extracted between two images, and two points are extracted by using a shift of the extracted corresponding points.
When performing image processing such as calculating a conversion equation between two images, as shown in FIG. 1, first, a plurality of corresponding points are extracted between two images by a corresponding point extracting means 102 (corresponding point extracting step). Next, by the individual expansion / contraction rate calculation means 103,
An individual expansion / contraction ratio between two images is calculated for each group of a plurality of corresponding points (individual expansion / contraction ratio calculating step), and the first corresponding point removing unit 104 includes the group in which an abnormal individual expansion / contraction ratio is calculated. In many cases, corresponding points are deleted from a plurality of corresponding points (first corresponding point removing step), and then all corresponding points remaining as processing results of the first corresponding point removing means 104 are calculated by the expansion / contraction ratio calculating means 105. The expansion / contraction ratio between the two images is calculated (expansion / contraction ratio calculation step), and the second corresponding point removing unit 106 is used to calculate the individual expansion / contraction ratio close to the above-mentioned expansion / contraction ratio in a group of corresponding points. A plurality of corresponding points are removed from the plurality of corresponding points so that only many corresponding points remain (second corresponding point removing step).
Using a corresponding point remaining as a processing result of the second corresponding point removing means 106, a conversion formula between two images is calculated based on a shift of the corresponding point (conversion formula calculation step).

As described above, since a plurality of corresponding points are used, even when some corresponding points cannot be detected, it is possible to position the two images, and the robustness against noise is high. Also, for example, even when the erroneous extraction corresponding point is used as a reference, as in the present invention, a corresponding point which is often included in a group in which an abnormal individual expansion / contraction rate is calculated is deleted from a plurality of corresponding points. The range of possible individual expansion / contraction ratios can be accurately determined by further removing only the corresponding points that are often included in the group of corresponding points where the individual expansion / contraction ratios close to the expansion / contraction ratio are calculated. This makes it possible to minimize the influence of erroneous extraction due to corresponding points, thereby enabling highly accurate alignment between two images. Further, since the number of corresponding points is in a practical range as compared with the third conventional example, the degree of freedom in form design and the like can be increased.

In the image processing method according to the fourth aspect and the image processing apparatus according to the twelfth aspect, the first corresponding point removing unit 104 determines a threshold value for judging an abnormal individual expansion / contraction ratio, or a second threshold value. The corresponding point removing means 106 removes the corresponding point by changing the threshold value for judging the closeness of the individual expansion rate to the expansion rate depending on the distance between the corresponding points.

For example, the error of the expansion / contraction ratio calculated as the distance between the corresponding points is shorter is larger. Therefore, as the distance between the corresponding points is smaller, it is determined that the individual individual expansion / contraction ratio is abnormal. It is necessary to change the threshold value so that the range in which the closeness of the individual expansion / contraction ratio is far is widened. As in the present invention, by changing the threshold value so as to conform to the features of the system, the error between the corresponding points is small because the distance between the corresponding points is short. On the other hand, it is possible to reduce mistakes in which the individual expansion / contraction ratio is determined to be far, thereby enabling more accurate positioning between two images.

In the image processing method according to the fifth aspect and the image processing apparatus according to the thirteenth aspect, the conversion formula calculating means
7 between the corrected image obtained by the conversion formula calculated in step 7 and one of the two images (reference image).
The corresponding point removing unit 108 calculates the distance between the corresponding points remaining at the time of processing by the conversion formula calculating unit 107, and removes the corresponding point having a large distance from the remaining corresponding points (third corresponding point removing step). ), By the conversion formula recalculating means 107
Using the corresponding points remaining as the processing result of the third corresponding point removing means 108, the conversion formula between the corrected image and one image (reference image) is calculated again by the shift of the corresponding points (conversion formula recalculation step). Like that.

As described above, the search range of the corresponding point between the corrected image and one image (reference image) is limited to a predetermined range by the third corresponding point removing means 108, so that the corresponding point can be associated. Can be reduced, and more accurate alignment between two images can be performed.

In the image processing method according to the sixth aspect and the image processing apparatus according to the fourteenth aspect, the conversion formula recalculating means 1
07 and the processing by the third corresponding point removing means 108 are performed by the third corresponding point removing means 108 while the maximum value or the average value of the distance between the corresponding points exceeds a predetermined threshold value.
Alternatively, the process is repeated while the corresponding point to be removed exists. As a result, it is possible to further reduce the error of the correspondence of the corresponding points, and it is possible to perform more accurate positioning between the two images.

In the image processing method according to the seventh aspect and the image processing apparatus according to the fifteenth aspect, the corresponding point extracting means
The corresponding points extracted by 2 are used as correction timing marks. As described above, the processing is facilitated by uniformly setting the corresponding points as correction timing marks such as a “+” shape.

According to the storage medium readable by a computer according to the eighth aspect,
It is desirable that the image processing method described in 5, 6, or 7 be stored as a program for causing a computer to execute the method. For example, if each step is prepared as a program part, an image processing system suitable for the characteristics of the input image can be easily constructed by combining the program parts at the image processing system design stage.

In the image processing apparatus according to the sixteenth aspect, as shown in FIG.
A corrected image is generated based on the conversion formula calculated by the conversion formula calculation unit or the conversion formula recalculation unit 107, and the character in the character image existence area 114 that is given in advance for one of the two images (reference image) 18. A character image is cut out from the corrected image based on the data.
In the image processing apparatus according to
The character image obtained by the character extracting means 109 is recognized and output as a character code. When the present invention is applied to, for example, a FAXOCR device, an automatic ordering operation and the like can be performed based on the output result, and a highly accurate business operation can be performed.

Further, in the image processing apparatus according to the eighteenth aspect, another image (input image) of the two images is an image input via a facsimile. When the present invention is applied to, for example, a facsimile apparatus or the like, it is possible to perform an automatic order receiving and ordering operation using a facsimile input as an input image, and to perform a business operation with high accuracy.

FIG. 2 shows a typical system configuration example to which the image processing method and the image processing apparatus of the present invention are applied. FIG.
FIG. 1 is a configuration diagram showing a specific example of an actual device configuration of an ordering / ordering system using a FAXOCR device. FIG. 3 is an explanatory diagram schematically showing the flow of processing of this system.

In the present system, the completed form 205 is
For example, the PC 203 sends necessary information about an input image that is sent from the FAX 201 via the public line 202 and includes rotation, expansion, contraction, or noise of the image. Is extracted, characters are cut out, character recognition processing is performed, and the data is provided as character data to a slip automatic processing or the like.

That is, in FIG. 3, an affine transformation equation is calculated for two images, an input image and a stored master image (reference image) (step S301), and the character recognition target area of the master image is calculated based on the affine transformation equation. Is performed (step S302), a character recognition target region is extracted from the input image based on the character recognition target region (step S303), and the character recognition process (step S302) is performed.
304) is performed, and the character data is subjected to slip automatic processing and the like (step S305).

[Embodiment] FIG. 1 is a configuration diagram of a FAX OCR apparatus which is an example of an image processing apparatus according to an embodiment of the present invention.

In the figure, the image processing apparatus of this embodiment includes an image input unit 101, a timing mark detection unit 10
2. Individual expansion / contraction ratio calculation unit 103, first timing mark deletion unit 104, expansion / contraction ratio calculation unit 105, second timing mark deletion unit 106, image conversion unit 107, third timing mark deletion unit 108, character cutout unit 109, text It comprises a recognition unit 110, an image buffer 111, a timing mark buffer 112, a scaling buffer 113, and a character image area memory 114.

Each component of the image processing apparatus according to the present embodiment corresponds to each unit described in the claims, and the timing mark detection unit 102 includes an individual expansion / contraction ratio calculation unit 103 as a corresponding point extraction unit. Is an individual expansion / contraction rate calculation unit, the first timing mark deletion unit 104 is a first corresponding point removal unit, the expansion / contraction ratio calculation unit 105 is an expansion / contraction ratio calculation unit, and the second timing mark deletion unit 106 is a second corresponding point removal unit. The image conversion unit 107 is a conversion formula calculation unit and a conversion formula recalculation unit, the third timing mark deletion unit 108 is a third corresponding point removal unit, the character cutout unit 109 is a character cutout unit, and the character recognition unit 110 is a Each corresponds to character recognition means.

Further, the processing of each component of the image processing apparatus of the present embodiment corresponds to each step described in the claims, and the processing of the timing mark detection unit 102 includes individual expansion / contraction in the corresponding point extraction step. The processing of the rate calculation unit 103 is an individual expansion / contraction rate calculation step, the processing of the first timing mark deletion unit 104 is a first corresponding point removal step, the processing of the expansion / contraction rate calculation unit 105 is an expansion / contraction rate calculation step, and the second timing mark The process of the deletion unit 106 is a second corresponding point removal step, the process of the image conversion unit 107 is a conversion formula calculation step and a conversion formula recalculation step, and the process of the third timing mark deletion unit 108 is a third correspondence point removal step. , Respectively.

Next, FIG. 4 shows an example of an input image (another image of the two images in the claims) used in the present embodiment, and a reference image (one of the two images). Are shown in FIG.

As shown in FIGS. 4 and 5, in the present embodiment, a corresponding point is used as a correction timing mark, and a plurality of timing marks (411 to 418 in the input image, 511 to 518 in the reference image) are used. Align two images.

In the following, the processing performed by each component of the image processing apparatus of this embodiment shown in FIG. 1 will be described with reference to FIGS.
This will be described in detail with reference to FIG.

First, the image input unit 101 converts an image input from a facsimile or the like into a form that can be processed by the image processing apparatus of this embodiment, and takes it into the image buffer 111.

Next, the timing mark detection section 102 detects a timing mark by pattern matching or the like. In this embodiment, as shown in FIGS. 4 and 5, a simple example is shown in which a cross point is used as a timing mark.

FIG. 6 is a flowchart showing the procedure of the timing mark extraction process in the timing mark detection unit 102.

In extracting a timing mark, first, an original rectangular shape is extracted in step S601, and a mask process is performed in step S602. The mask processing is performed by applying a mask as shown in FIG. 7 to the input image.

That is, as shown in FIG. 7, the timing mark extraction mask includes a central pixel 701 and four surrounding pixels 702 surrounding the central pixel 701.
To 705. This timing mark mask is arranged on the input image as shown in FIG. 9 and subjected to mask processing. That is, for each of the masks A and B, the central pixel is arranged so as to be a black pixel and the four surrounding pixels are arranged as white pixels, in which case the central pixel of each mask is made a black pixel.

Next, in step S603, a process of extracting a cross point extraction rectangle (timing mark extraction rectangle) is performed on the input image after the mask processing. The process of extracting a cross point extraction rectangle is a process of extracting a circumscribed rectangle surrounding a connected component of black pixels. FIG. 10 is an input image obtained by performing a masking process on the input image of FIG. 9, in which a rectangle 1001 is extracted as a cross point extraction rectangle.

Next, the processing after step S604 is a loop processing performed for each cross point extraction rectangle.
First, in step S604, it is determined whether or not the cross point extraction rectangle has an appropriate size. Steps S605 to S607 are a loop process performed on the original rectangle including the cross point extraction rectangle. If the original rectangle is small and the aspect ratio is inappropriate, step S6 is executed.
In step S08, the center coordinates of the cross point extraction rectangular shape are extracted, and the cross coordinates are extracted as cross points (timing marks) in step S609.

In other words, in the extracted cross point extraction rectangle, a rectangle having a certain size appears in the cross point (timing mark) portion in accordance with the thickness of the line segment. Therefore, a cross point extraction rectangle that satisfies all of the following conditions is extracted as a timing mark.

CRmn <CRw <CRmx (1) CRmn <CRh <CRmx (2) Here, CRw is the width [pixel] of the timing mark extraction rectangular shape, and CRh is the timing mark extraction rectangular height [ Pixel]. CRmn and CRmx are constants determined by the thickness of the line segment of the cross point used as a timing mark.

As described above, the timing mark is extracted by the timing mark detection unit 102 and stored in the timing mark buffer 112. Here, FIG. 8 shows the timing mark extraction results for the input image of FIG. 4 and the reference image of FIG.

Next, the individual expansion / contraction ratio calculation unit 103 calculates the enlargement ratio or reduction ratio in the X and Y directions of the input image with respect to the reference image from the three timing marks, and calculates the combination of the timing marks and the expansion / contraction ratio. To expand / contract buffer 1
13 is stored.

Using a certain timing mark as a reference point, two other timing marks are referred to as a first reference point and a second reference mark.
If a reference point is set, X, Y of the input image with respect to the reference image
The enlargement ratio or reduction ratio in the direction can be obtained using the following equation.

Sy = {(Li2 × Lxc1) − (Lxc2 × Li1)} / {(Lyc2 × Lxc1) − (Lxc2 × Lyc1)} (3) Sx = {Li1− (Sy × Lyc1)} / Lxc1 (4) Here, Sx is the expansion ratio in the X direction, and Sy is the expansion ratio in the Y direction. Lxc1 is the square of the distance between the first reference point and the reference point on the reference image in the X direction, Lxc2 is the square of the distance between the second reference point and the reference point on the reference image in the X direction, Lyc1
Is the square of the distance in the Y direction between the first reference point and the reference point on the reference image, Lyc2 is the square of the distance in the Y direction between the second reference point and the reference point on the reference image, and Li1 is the square of the distance in the input image. The square of the distance between the first reference point and the reference point, Li2, is the second square on the input image.
This is the square of the distance between the reference point and the reference point.

Next, the first timing mark deletion unit 104
First, a set of reference points satisfying any of the following conditions is determined to be a set of reference points having an abnormal expansion / contraction ratio, and the number of reference points is totaled for each reference point.

Sy <Symn (5) Sy> Symx (6) Sx <Sxmn (7) Sx <Sxmx (8) Here, Sx is the expansion ratio in the X direction, Sy is the expansion ratio in the Y direction, and Sxmn is the X direction. , Symn is the minimum value of the expansion ratio in the Y direction, Sxmx is the maximum value of the expansion ratio in the X direction,
Symx is the maximum expansion / contraction ratio in the Y direction.

When the number of reference points having an abnormal expansion / contraction ratio is counted for each reference point, a timing mark (reference point) satisfying the following condition is deleted.

Nf> Ntf (9) where Nf is the number of sets of reference points having an abnormal expansion / contraction ratio,
Ntf is an allowable number of sets of reference points having abnormal expansion and contraction rates. The allowable number Ntf is determined according to the number of printed timing marks and the like.

FIG. 11 exemplifies a deletion process performed by the first timing mark deletion unit 104 on the timing mark extraction result of FIG.

When an erroneous timing mark is used as a reference point, the number of combinations of reference points for which an abnormally large or small expansion / contraction ratio is calculated increases. In this embodiment,
The first timing mark deletion unit 104 makes it possible to determine a rough range of the expansion / contraction ratio of the input image with respect to the reference image for each applied system.

Further, by changing the allowable number Ntf according to the average value of the distance between the timing marks and the like, the performance is expected to be improved. That is, since the error of the expansion / contraction ratio calculated as the distance between the timing marks becomes smaller, the allowable number Ntf suitable for the system is calculated so that the allowable number Ntf becomes larger as the distance between the timing marks becomes smaller. By using this, it is possible to reduce errors such as an error being increased due to a small distance between the timing marks and being determined to be abnormal.

Next, the expansion ratio calculator 105 calculates the expansion ratio of the input image with respect to the reference image.

First, the first timing mark deletion unit 104
The median value SX (n) of the expansion and contraction ratio in the X direction is calculated for each of the remaining reference points that have not been deleted in. Next, SX (n)
The median value of the input image is calculated, and is set as the X-direction expansion / contraction ratio Sxim with respect to the reference image. Also, for the Y direction, the expansion / contraction ratio Syim in the Y direction is calculated by the same processing. By calculating the median in two stages in this way, the probability of obtaining the correct expansion / contraction ratio can be improved.

Next, the second timing mark deletion unit 106
In, a reference point having many combinations of reference points for which an expansion ratio close to the expansion ratio of the input image is calculated with respect to the reference image is selected, and other unselected timing marks are deleted.

When each of the individual expansion / contraction rates of the timing marks not deleted by the first timing mark deletion unit 104 satisfies the following conditions, it is determined that the individual expansion / contraction rate is close to the input / output expansion rate.

Si−α <S <Si + α (10) where Si is an expansion / contraction ratio of the input image with respect to the reference image, α
Is a constant, and S is an individual expansion / contraction ratio (determined from three timing marks).

Here, α may be a constant, but performance can be expected to be improved by making it variable according to the distance between the three timing marks and the like. That is, the error of the expansion / contraction ratio calculated as the distance between the timing marks is shorter is larger. Therefore, it is necessary to increase α as the distance between the timing marks is smaller. This is because, by using a calculation formula of α that is appropriate for the system, it is possible to reduce errors such that the distance between the timing marks is short, the error increases, and the expansion / contraction ratio is determined to be long.

Next, the number Ns of combinations of reference points for which individual expansion / contraction ratios close to the expansion / contraction ratio of the input image have been calculated is counted for each reference point and each XY direction.

It can be determined that the greater the number Ns of the combinations, the greater the possibility that the correspondence is correct. Therefore, by setting a predetermined threshold value for the number of combinations Ns and selecting a reference point (timing mark) having a larger number of combinations Ns than the threshold value, erroneous detection of the timing mark can be prevented. .

For each reference point, the number Nsx of combinations of reference points for which the expansion / contraction ratio close to the image is calculated in the X direction, and the expansion / contraction ratio close to the image expansion / contraction ratio in the Y direction are calculated. The smaller of the number Nsy of combinations of the reference points is set as Ns of the reference point. Therefore, the timing mark satisfying the following condition is deleted.

Ns <Ntsc (11) Here, Ns is the number of combinations of reference points at which the expansion / contraction ratio close to the image is calculated, and Ntsc is the reference point at which the expansion / contraction ratio close to the image is calculated. Is the minimum number of combinations.

First timing mark deletion unit 10 in FIG.
4, the second timing mark deletion unit 1
FIG. 12 shows an example of the deletion process performed in step S06.

Next, in the image conversion unit 107, the input image,
The shift between the timing marks of the reference image is calculated, for example, “William H. Press,“ Numerical Recipes in C ”, CAM
BRIDGE UNIVERSITY PRESS, pp59-70, pp671-681 ''
A correction function (conversion equation) is calculated by a method shown in (Reference Document 1). That is, it is assumed that an affine transformation is used as the correction function in the present embodiment, and the following correction function (conversion equation) is obtained by the method described in Reference 1.

[0091]

(Equation 1)

Here, (Xt, Yt) is the coordinate system after correction, and (Xi, Yi) is the coordinate system of the input image. For example, when calculating the pixel value of (2981, 3864) of the corrected image, first, using the conversion formula (12), the coordinates in the corresponding input image are calculated as follows.

[0093]

(Equation 2)

In the case of this pixel, (315) of the input image
8,3726) to the (2981, 38) of the corrected image.
64). When the pixel values of all the pixels of the corrected image are calculated with reference to the input image in this manner, the form of the corrected image exists at the same position as the form of the reference image.

Next, the third timing mark deletion unit 108
Calculates the distance between the timing marks of the corrected image and the reference image, and sets the distance between the timing marks to the threshold value Dth.
Remove larger timing marks.

When the timing mark is deleted by the third timing mark deletion unit 108, the image conversion unit 10
7 is performed again. If there is no timing mark to be deleted in the third timing mark deletion unit 108, the process proceeds to the character cutout unit 109.

Next, the character extracting section 109 refers to the data of the area where the character image exists from the character image area memory 114, and extracts the character image portion from the corrected image. FIG. 13 shows a conceptual diagram of the data stored in the character image area memory 114. FIG. 14B shows a character image cutout result obtained by referring to the character image area memory 114 for the corrected image in FIG.

When the robustness against skew of the character recognition unit 110 is high, the amount of calculation to obtain the corrected image from the input image is large, so the coordinates in the character image area memory 114 are corrected and the character image is corrected. It is more preferable to cut out.

Finally, the character recognizing section 110 recognizes the character image sent from the character extracting section 109 and outputs it as a character code. FIG. 15B shows a character recognition result for the character image cutout result of FIG. 15A, and FIG. 1 is a conceptual diagram of output data from the character recognition unit 110 in this case.
5 (c).

F to which the image processing apparatus of the present embodiment is applied
The AXOCR device or the like further performs an automatic ordering and ordering operation based on the output result from the character recognition unit 110. Since this part is not directly related to the present invention, the description is omitted.

According to the configuration of the image processing apparatus or the image processing method of the present embodiment as described above, the following effects can be obtained. First, since a plurality of timing marks are used, even if some of the timing marks cannot be detected, alignment between two images is possible, and robustness against noise is strong. Second, erroneous detection of timing marks can be prevented. Third, by using a plurality of timing marks over a wide range, highly accurate positioning between two images can be performed. Fourth, the degree of freedom regarding the position of the timing mark is large.

[0102]

As described above, according to the first aspect of the present invention,
According to the image processing method and the image processing apparatus of the ninth aspect, a plurality of corresponding points are extracted between two images, and a conversion equation between the two images is calculated by using a shift of the extracted corresponding points. In the case of performing image processing such as performing an image processing, a corresponding point extracting unit extracts a plurality of corresponding points between two images, and an individual expansion / contraction ratio calculating unit calculates an individual expansion ratio between the two images for each group of the plurality of corresponding points. The corresponding points that are often included in the group in which the abnormal individual expansion / contraction ratio is calculated by the first corresponding point removing means are deleted from the plurality of corresponding points, and the first corresponding point removing means is further converted by the conversion formula calculating means. By using the corresponding points remaining as a result of the above processing to calculate the conversion formula between the two images based on the shift of the corresponding points, robustness against noise is strong, and erroneous detection of the corresponding points due to timing marks and the like is prevented. High accuracy Positioning is possible, moreover it is possible to provide an image processing method and image processing apparatus capable of performing alignment of practical image.

According to the image processing method according to the second aspect and the image processing apparatus according to the tenth aspect, a plurality of corresponding points are extracted between two images, and the corresponding points are shifted by using a shift of the extracted corresponding points. When performing image processing such as calculating a conversion formula between two images, first, a plurality of corresponding points are extracted between two images by a corresponding point extracting unit, and then a plurality of groups of corresponding points are extracted by an individual expansion / contraction ratio calculating unit. Each time, the individual expansion / contraction ratio between the two images is calculated, and the expansion / contraction ratio between the two images is calculated using all the corresponding points extracted by the expansion / contraction ratio calculation means. , The corresponding points that are likely to be included in the corresponding point group for which the individual expansion ratios close to the expansion ratio are calculated are removed from the plurality of corresponding points, and the second corresponding point removal is performed by the conversion formula calculating means. Corresponding points left as a result of processing by means Is used to calculate the conversion equation between the two images based on the shift of the corresponding points, so that the robustness against noise is strong, the erroneous detection of the corresponding points due to timing marks and the like can be prevented, and highly accurate positioning can be performed. Thus, it is possible to provide an image processing method and an image processing apparatus capable of performing a practical image alignment.

Further, according to the image processing method of the third aspect and the image processing apparatus of the eleventh aspect, a plurality of corresponding points are extracted between two images, and two points are extracted by using a shift of the extracted corresponding points. When performing image processing such as calculating a conversion formula between two images, first, a plurality of corresponding points are extracted between two images by a corresponding point extracting unit, and then a plurality of groups of corresponding points are extracted by an individual expansion / contraction ratio calculating unit. An individual expansion / contraction ratio between the two images is calculated for each image, and the corresponding point which is often included in the group in which the abnormal individual expansion / contraction ratio is calculated is deleted from the plurality of corresponding points by the first corresponding point removing means. Next, the expansion and contraction ratio calculating means calculates the expansion and contraction ratio between the two images using all the corresponding points remaining as the processing result of the first corresponding point removing means,
The corresponding point removing means removes from the plurality of corresponding points such that only the corresponding points which are often included in the corresponding point group for which the individual expansion / contraction ratio close to the expansion / contraction ratio is calculated remain. Since the conversion equation between the two images is calculated by using the corresponding points remaining as the processing result of the second corresponding point removing means based on the shift of the corresponding points, robustness against noise is strong, It is possible to provide an image processing method and an image processing apparatus capable of preventing erroneous detection of points, performing high-accuracy alignment, and performing practical image alignment.

According to the image processing method of the fourth aspect and the image processing apparatus of the twelfth aspect, the threshold value for determining the abnormal individual expansion / contraction ratio in the first corresponding point removing means, or (2) Since the threshold for judging the closeness of the individual expansion / contraction ratio to the expansion / contraction ratio in the corresponding point removing means is made variable according to the distance between the corresponding points, the corresponding points are removed, so that higher accuracy is achieved. It is possible to provide an image processing method and an image processing apparatus capable of performing alignment between two images.

According to the image processing method according to the fifth aspect and the image processing apparatus according to the thirteenth aspect, the corrected image obtained by the conversion formula calculated by the conversion formula calculating means and one of the two images. With respect to the image, the third corresponding point removing means calculates the distance between the corresponding points remaining at the time of processing by the conversion formula calculating means, and removes the corresponding point having a large distance from the remaining corresponding points, The conversion formula recalculating means uses the corresponding points remaining as the processing result of the third corresponding point removing means to calculate again the conversion formula between the corrected image and the one image due to the shift of the corresponding points. Since the search range of the corresponding point with one image is limited within a predetermined range, the error of the correspondence of the corresponding point can be reduced, and the image can be positioned with higher accuracy between the two images. Provided is a processing method and an image processing apparatus. Door can be.

According to the image processing method of the sixth aspect and the image processing apparatus of the fourteenth aspect, the processing in the conversion formula recalculating means and the third corresponding point removing means is performed by the third corresponding point removing means. Since the repetition is performed while the maximum value or the average value of the distances between the corresponding points exceeds a predetermined threshold value or while there is a corresponding point to be removed, the correspondence of the corresponding points is further determined. It is possible to provide an image processing method and an image processing apparatus that can reduce errors and can perform more accurate alignment between two images.

According to the image processing method according to the seventh aspect and the image processing apparatus according to the fifteenth aspect, since the corresponding points extracted by the corresponding point extracting means are used as correction timing marks, the processing can be easily performed. An image processing method and an image processing apparatus that can be performed can be provided.

According to the storage medium readable by a computer according to the eighth aspect, the first, second, third and third aspects are provided.
Since the image processing method described in 4, 5, 6, or 7 is stored as a program for causing a computer to execute the image processing method, for example, if each step is a program component, an image processing system suitable for image characteristics can be easily constructed. be able to.

According to the image processing apparatus of the sixteenth aspect, the character cutout means generates a corrected image based on the conversion formula calculated by the conversion formula calculation means or the conversion formula recalculation means, and outputs the corrected image from the two images. A character image is cut out from the corrected image based on character data in a character image existing area given in advance for one of the images, and the image processing apparatus according to claim 17, further comprising: Since the character image obtained by the character extracting means is recognized and outputted as a character code, for example, when the present invention is applied to a FAX OCR device or the like, an automatic ordering and ordering operation based on the output result becomes possible. Business operations with high accuracy can be performed.

Further, according to the image processing apparatus of the eighteenth aspect, since the other of the two images is an image input via a facsimile, for example, when applied to a facsimile apparatus or the like, Automatic ordering and the like using facsimile input as an input image can be performed, and highly accurate business operation can be performed.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a FAXOCR apparatus which is an example of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a typical system to which the image processing method and the image processing apparatus of the present invention are applied.

FIG. 3 is an explanatory diagram schematically showing a processing flow of the system of FIG. 2;

FIG. 4 is an explanatory diagram of an input image used in the embodiment.

FIG. 5 is an explanatory diagram of a reference image used in the embodiment.

FIG. 6 is a flowchart illustrating a procedure of a timing mark extraction process in a timing mark detection unit.

FIG. 7 is an explanatory diagram of a mask used in mask processing of timing mark extraction.

FIG. 8 is an explanatory diagram illustrating a timing mark extraction result.

FIG. 9 is an explanatory diagram of a mask process, illustrating an input image before the mask process.

FIG. 10 is an explanatory diagram of a mask process, and is an explanatory diagram showing an input image after the mask process.

FIG. 11 is an explanatory diagram exemplifying a process of a first timing mark deletion unit for the timing mark extraction result of FIG. 8;

FIG. 12 is an explanatory diagram exemplifying a process of a second timing mark deletion unit with respect to a processing result of the first timing mark deletion unit of FIG. 11;

FIG. 13 is a conceptual explanatory diagram of data stored in a character image area memory.

14A is an explanatory diagram illustrating a corrected image, FIG. 14B is an explanatory diagram illustrating a character image cutout result, and FIG. 14C is an explanatory diagram illustrating data in a character image area memory.

15A is an explanatory diagram showing a character image cutout result, FIG. 15B is an explanatory diagram showing a character recognition result, and FIG. 15C is an explanatory diagram showing output data.

[Explanation of symbols]

Reference Signs List 101 Image input unit 102 Timing mark detection unit (corresponding point extraction unit) 103 Individual expansion / contraction ratio calculation unit (Individual expansion / contraction ratio calculation unit) 104 First timing mark deletion unit (first corresponding point removal unit) 105 Expansion / contraction ratio calculation unit (Extension / contraction) Rate calculating means) 106 second timing mark deleting unit (second corresponding point removing means) 107 image converting unit (conversion formula calculating means, conversion formula recalculating means) 108 third timing mark deleting unit (third corresponding point removing means) 109 Character extraction unit (character extraction unit) 110 Character recognition unit (character recognition unit) 111 Image buffer 112 Timing mark buffer 113 Expansion / contraction ratio buffer 114 Character image area memory

Claims (18)

[Claims] 1. A method for extracting a plurality of corresponding points between two images,
An image processing method for calculating a conversion equation between the two images using a shift of the extracted corresponding points, a corresponding point extracting step of extracting a plurality of corresponding points between the two images, An individual expansion / contraction ratio calculating step for calculating an individual expansion / contraction ratio between the two images for each group; and a corresponding point which is often included in a group where an abnormal individual expansion / contraction ratio is calculated is deleted from the plurality of corresponding points. A first corresponding point removing step, and a conversion equation calculating step of calculating a conversion equation between the two images by using a corresponding point remaining as a result of the first corresponding point removing step, based on a shift of the corresponding point. An image processing method comprising: 2. Extracting a plurality of corresponding points between two images,
An image processing method for calculating a conversion equation between the two images by using a shift of the extracted corresponding points; a corresponding point extracting step of extracting a plurality of corresponding points between the two images; An expansion / contraction ratio calculating step of calculating an expansion / contraction ratio between the two images using corresponding points; an individual expansion / contraction ratio calculating step of calculating an individual expansion / contraction ratio between the two images for each group of a plurality of corresponding points; A second corresponding point removing step of removing from the plurality of corresponding points so that only corresponding points that are often included in a group of corresponding points for which an individual expansion ratio close to the expansion ratio is calculated; Using a corresponding point remaining as a result of the step to calculate a conversion equation between the two images based on a shift of the corresponding point. 3. Extracting a plurality of corresponding points between two images,
An image processing method for calculating a conversion equation between the two images using a shift of the extracted corresponding points, a corresponding point extracting step of extracting a plurality of corresponding points between the two images, An individual expansion / contraction ratio calculating step for calculating an individual expansion / contraction ratio between the two images for each group; and a corresponding point which is often included in a group where an abnormal individual expansion / contraction ratio is calculated is deleted from the plurality of corresponding points. A first corresponding point removing step, a stretching ratio calculating step of calculating a stretching ratio between the two images by using all the corresponding points remaining as a result of the first corresponding point removing step, As a result of the second corresponding point removing step of removing from the plurality of corresponding points so that only corresponding points that are often included in the corresponding point group whose expansion ratio is calculated remain, Tsu by using the corresponding points, the image processing method characterized by having a conversion formula calculating a transformation equation between the displacement by the two corresponding points image. 4. A threshold value for determining an abnormal individual expansion / contraction ratio in the first corresponding point removing step, or a threshold value for determining an individual expansion / contraction ratio close to the expansion / contraction ratio in the second corresponding point removing step. 3. The threshold value of (1) is variable according to the distance between corresponding points.
Or the image processing method according to 3. 5. A corresponding point remaining between the corrected image obtained by the conversion formula calculated in the conversion formula calculation step and one of the two images at the time of the conversion formula calculation step. A third corresponding point removing step of calculating a distance between the corresponding points and removing the corresponding point having a larger distance from the remaining corresponding points, and using the corresponding point remaining as a result of the third corresponding point removing step, A conversion formula recalculating step of calculating a conversion formula between the corrected image and the one image based on a point shift.
5. The image processing method according to 3 or 4. 6. The conversion formula recalculating step and the third step.
The corresponding point removing step is performed while the maximum value or the average value of the distances between the corresponding points exceeds a predetermined threshold value in the third corresponding point removing step, or while the corresponding point to be removed exists. 6. An image processing method according to claim 5, wherein the processing is repeatedly performed. 7. The image processing method according to claim 1, wherein the corresponding points extracted in the corresponding point extracting step are timing marks for correction. 8. The method of claim 1, 2, 3, 4, 5, 6, or 7.
A computer-readable storage medium storing a program for causing a computer to execute the image processing method described above. 9. Extracting a plurality of corresponding points between two images,
An image processing apparatus that calculates a conversion equation between the two images by using the shift of the extracted corresponding points and performs positioning between the two images, extracts a plurality of corresponding points between the two images. Corresponding point extraction means, individual expansion rate calculating means for calculating an individual expansion rate between the two images for each group of a plurality of corresponding points, and a group in which an abnormal individual expansion rate is calculated in many cases. Using a first corresponding point removing means for removing a corresponding point from the plurality of corresponding points, and a corresponding point remaining as a processing result of the first corresponding point removing means, a difference between the two images by a shift of the corresponding point. And a conversion formula calculating means for calculating the conversion formula. 10. A method for extracting a plurality of corresponding points between two images, calculating a conversion formula between the two images using a shift of the extracted corresponding points, and performing positioning between the two images. A corresponding point extracting unit for extracting a plurality of corresponding points between the two images, and a scaling ratio calculating unit for calculating a scaling ratio between the two images using all the extracted corresponding points. And an individual expansion / contraction ratio calculating means for calculating an individual expansion / contraction ratio between the two images for each group of a plurality of corresponding points; So that only many corresponding points remain
Second corresponding point removing means for removing from the plurality of corresponding points;
Using a corresponding point remaining as a processing result of the second corresponding point removing means, a conversion formula calculating means for calculating a conversion formula between the two images based on a shift of the corresponding point. Processing equipment. 11. A method for extracting a plurality of corresponding points between two images, calculating a conversion formula between the two images using a shift of the extracted corresponding points, and performing positioning between the two images. A corresponding point extracting means for extracting a plurality of corresponding points between the two images, and an individual stretching rate calculating means for calculating an individual stretching rate between the two images for each group of a plurality of corresponding points. And a first corresponding point removing unit that deletes a corresponding point that is often included in a group in which an abnormal individual expansion / contraction rate is calculated from the plurality of corresponding points, and remains as a processing result of the first corresponding point removing unit. Expansion / compression ratio calculating means for calculating the expansion / contraction ratio between the two images using all the corresponding points, and only corresponding points which are often included in a group of corresponding points for which individual expansion / contraction ratios close to the expansion / contraction ratio are calculated. So that
Second corresponding point removing means for removing from the plurality of corresponding points;
Using a corresponding point remaining as a processing result of the second corresponding point removing means, a conversion formula calculating means for calculating a conversion formula between the two images based on a shift of the corresponding point. Processing equipment. 12. A threshold value for judging an abnormal individual expansion / contraction ratio in the first corresponding point removing means, or a threshold value for judging the proximity of the individual expansion / contraction ratio to the expansion / contraction ratio in the second corresponding point removing means. 12. The image processing apparatus according to claim 9, wherein the threshold value is variable according to a distance between corresponding points. 13. The image processing apparatus according to claim 1, wherein the image processing apparatus performs processing of the conversion formula calculation unit between a corrected image obtained by the conversion formula calculated by the conversion formula calculation unit and one of the two images. A third corresponding point removing means for calculating the distance between the corresponding points remaining at the time and removing the corresponding points with a large distance from the remaining corresponding points, and a third corresponding point removing means remaining as a processing result. 13. A conversion formula recalculating means for calculating a conversion formula between the corrected image and the one image based on a shift of the corresponding point by using a corresponding point. The image processing apparatus according to any one of the preceding claims. 14. The processing in said conversion formula recalculating means and said third corresponding point removing means, wherein the maximum value or average value of the distance between corresponding points is determined by a predetermined threshold value in said third corresponding point removing means. 14. The image processing apparatus according to claim 13, wherein the processing is repeatedly performed while the number of times exceeds a predetermined time period or while a corresponding point to be removed exists. 15. The image processing apparatus according to claim 9, wherein the corresponding points extracted by the corresponding point extracting means are timing marks for correction. 16. The image processing apparatus according to claim 1, wherein the image processing apparatus generates a corrected image based on the conversion formula calculated by the conversion formula calculation unit or the conversion formula recalculation unit, and the correction image is given in advance for one of the two images. And character extracting means for extracting a character image from the corrected image based on character data in the character image existing area.
The image processing apparatus according to 0, 11, 12, 13, 14, or 15. 17. The image processing apparatus according to claim 16, wherein said image processing apparatus has a character recognizing means for recognizing a character image obtained by said character extracting means. 18. The method according to claim 9, wherein the other of the two images is an image input via facsimile.
18. The image processing device according to 16 or 17.