JPH10340348A - Image aligning method, facsimile character recognizing method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To highly precisely exclude erroneous correspondence from a correspondent coordinate. SOLUTION: Based on the matching distance of correspondent coordinate acquisition part 1, a right correspondence probability calculation part 2 applies right correspondence probability to each correspondent coordinate and stores it in a storage part 3. A parameter estimation part 4 models the generation distribution of coordinate correspondence while using right and wrong correspondence mixed models and finds the parameter of model function through tolerance estimation, and a wrong correspondence exclusion part 5 excludes coordinate data having wrong correspondence probability higher than a prescribed threshold value and calculates the parameter again while using the remaining correspondent coordinates.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image registration method, a facsimile character recognition method, and a recording medium for associating a certain image with an affine-transformed image.

[0002]

2. Description of the Related Art In image processing, a technique is used for coordinating coordinates between two images having substantially the same contents, such as different positions and enlargement ratios. For example, in pre-processing of a fax character recognition device (FAXOCR device), it is necessary to specify an area to be recognized by comparing an image transmitted by FAX with an original image. Therefore, an image alignment technique for obtaining a coordinate correspondence between the transmitted form image and a reference image prepared in advance is important. In order to estimate the conversion of the image, the computer first needs to automatically know which point of the transmitted image to be processed is associated with which point of the reference image stored in advance in the computer.

As a conventional alignment method in a FAXOCR apparatus, a method using a timing mark printed in a specific shape at a specific position has been used (for example, see Japanese Patent Application Laid-Open No. 4-261259).

[0004]

However, the above-described method using the timing mark has a problem in that the cost required to create a new form on which the timing mark is printed is high. Therefore, in order to solve this, the present applicant first extracts a plurality of patterns common to the processing target image and the reference image, and, with respect to the extracted pattern, associates the reference image with the processing target image. An image processing method has been proposed in which the same pattern is detected, the mutual positional relationship between the corresponding patterns is determined, and a correction formula for aligning the processing target image with the reference image is determined based on the positional relationship ( Japanese Patent Application No. 8-54308). In this method, an extraction pattern is used instead of a timing mark.

By the way, in a method of associating coordinates by pattern matching between a reference image and an image to be processed,
Incorrect matching point data may be included due to a matching error. Therefore, it is important to remove all erroneous corresponding points while leaving many correct corresponding points, and the applicant has proposed a method for removing errors in pattern correspondence (Japanese Patent Application Hei 8-
240154), the erroneous correspondence may not be completely removed, and a more accurate method has been demanded.

An object of the present invention is to provide an image registration method, a fax character recognition method, and a recording medium for deleting an incorrect correspondence from corresponding coordinates with high accuracy.

[0007]

In order to achieve the above object, according to the first aspect of the present invention, a reference image and a processing target obtained by subjecting the reference image to a predetermined geometric transformation are provided. An image registration method for associating coordinates with an image, wherein a conversion function is based on a correspondence between coordinates of a pattern extracted from the reference image and coordinates of the same pattern in a processing target image extracted by image matching. When calculating the parameters of, the probability representing the correctness of the corresponding is calculated based on the matching distance of the pattern, using the probability value and the degree of deviation from the transformation function calculated from the corresponding coordinates, the wrong correspondence It is characterized in that the correspondence determined to be present is deleted and the parameters of the conversion function are recalculated.

According to the invention described in claim 2, the occurrence distribution of the correspondence of the coordinates is modeled as two mixed models of a distribution that produces a correct correspondence and a distribution that produces an erroneous correspondence, and the probability of occurrence from the correct correspondence is calculated. When expressed using a probability representing the correctness of the correspondence based on the pattern matching, a parameter of a conversion function that maximizes the likelihood is obtained, and a distribution of an erroneous correspondence for each correspondence is determined using the parameter. It is characterized in that a likelihood ratio is obtained, and a response whose ratio is higher than a predetermined threshold value is determined to be a false response.

According to the third aspect of the present invention, when a parameter of the conversion function that maximizes the likelihood is obtained, a variable indicating whether the correspondence is correct or erroneous is used as a hidden variable, and a sequential EM method is used. It is characterized by performing statistical calculations.

According to the fourth aspect of the present invention, the initial parameters for performing the sequential calculation by the EM method are obtained by a robust estimation method that weights the data having a large residual so that the degree of contribution becomes small. It is characterized by:

In the invention according to claim 5, the transform function is divided into an affine transform portion and a non-linear portion, and after the parameter of the transform function is estimated by the affine transform, the magnitude of the residual exceeds a predetermined threshold. In this case, additional estimation of the residual portion using a nonlinear function system is performed.

According to a sixth aspect of the present invention, a multi-layer perceptron model is used as the nonlinear function system.

According to the seventh aspect of the present invention, the first to sixth aspects are provided.
Using the method described, the processing target image received from the facsimile, take the correspondence of the coordinates of the reference image held in advance, the area in the processing target image corresponding to the area specified in the reference image, It is characterized by character recognition.

According to the invention described in claim 8, in order to establish a coordinate correspondence between the reference image and the processing target image obtained by performing a predetermined geometric transformation on the reference image, the reference image is obtained from the reference image. A function of calculating a parameter of a conversion function based on the correspondence between the coordinates of the extracted pattern and the coordinates of the same pattern in the processing target image extracted by the image matching,
Using the function of calculating the probability indicating the correctness of the correspondence based on the matching distance of the pattern and the probability value and the degree of deviation from the conversion function calculated from the corresponding coordinates, it was determined that the correspondence was incorrect. It is a computer-readable recording medium in which a program for causing a computer to realize a function of deleting the correspondence and a function of recalculating the parameters of the conversion function is recorded.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be specifically described below with reference to the drawings. The image registration method of the present invention can be applied to various devices and systems.
An AXOCR device will be described as an example. As described above, the FAXOCR device captures the image of the form sent by facsimile, aligns the input image (image to be processed) with the reference image of the same type of form, and then describes necessary items in the input image. (This corresponds to an area previously specified in the reference image), is subjected to character recognition processing, and is output as data.

This FAXOCR device comprises an image input unit,
It is roughly divided into an image alignment unit and a recognition processing unit. The present invention relates to an image registration unit, but in the present invention, a part of the method of the above-mentioned application (Japanese Patent Application No. 8-54308) is used. That is, the image registration unit includes an input image memory, a pattern extraction unit, an identical pattern extraction unit, a reference image pattern memory, and a corresponding point detection unit, and the pattern extraction unit extracts a pattern of a predetermined size from the input image. The same pattern extraction unit finds the same pattern as the extracted input pattern from the reference image pattern memory, and the corresponding point detection unit detects the coordinates of the same pattern corresponding to the input image and the reference image.

Embodiment 1 FIG. 1 shows the structure of Embodiment 1 of the present invention. The image registration unit of the present invention includes a correspondence coordinate acquisition unit 1, a correct correspondence probability calculation unit 2, a correspondence storage unit 3, a mixed model parameter estimation unit 4, and a wrong correspondence deletion unit 5. FIG. 2 shows a processing flowchart of the present invention.

First, the corresponding coordinate acquisition unit 1 sets, as data for associating coordinates between two images, a set of coordinates of a reference image and an image to be processed, which are assumed to correspond.

[0019]

(Equation 1)

Is obtained. here

[0021]

(Equation 2)

Is a two-dimensional vector representing the coordinates of the reference image,

[0023]

(Equation 3)

Is a two-dimensional vector representing the coordinates of the image to be processed. For obtaining the set of coordinates, the method of the above-mentioned application can be used.

When this correspondence is created, pattern matching is performed, and matching distances d _1, ..., D _N are calculated for each of these correspondences. Based on this matching distance, positive correspondence probability calculation portion 2, to impart a positive correspondence probability p ₁ ,,,, p _N each corresponding. These probabilities are obtained by calculating the probabilities that each correspondence is correct based on the matching distance. A set of coordinate correspondence and correct correspondence probability

[0026]

(Equation 4)

Are stored in the correspondence storage unit 3 and are used for calculating the parameters of the conversion function.

As an example of a method of obtaining the correct correspondence probability, there is a method using a table prepared in advance. This method
Given a set of matching distances and the correctness of the correspondences for many examples of patterns of reference images and processing target images, the probability p that a correspondence having a matching distance d is a correct correspondence is given.
(D) is calculated by p (d) = (the number of correct correspondences with the matching distance d) / (the number of corresponding correspondences with the matching distance d) (1), and d is quantized to an appropriate roughness. p
It has a correct correspondence probability table giving (d).

To find the conversion function, the parameter θ
Is used to calculate θ that applies to data using a family of functions f (x; θ) having For example, when a conversion between an original image of FAX OCR and a received image is obtained, f (x; θ) is converted to an affine transformation f (x; θ) = Ax + b (2) (θ = (A, b)). Where A is 2 ×
Is a two matrix, and b is a two-dimensional vector.

Further, in this embodiment, a distribution that produces a correct correspondence and a distribution that produces an erroneous correspondence are modeled by two distributions having the same average and different variances, and the occurrence frequency of each is based on the matching distance. It is characterized by being determined by the correct correspondence probability. Particularly for the purpose of simplifying the calculation, it is convenient to set these two distributions as normal distributions. Therefore, only the case of the normal distribution will be described below.

When the correspondence (x, y) has the matching distance d, the probability model that generates it is

[0032]

(Equation 5)

Is determined. σ and τ represent the variances of the error distribution of the correct correspondence and the incorrect correspondence, respectively, and it is considered that σ << τ.
These variances are set in advance to certain constants depending on the knowledge about the problem, and the parameter estimating unit 4 estimates only the parameter θ from the data.
Maximum likelihood estimation is used as a criterion for obtaining the parameter θ.

Estimated amount

[0035]

(Equation 6)

After the data is obtained (step 101), a process for removing what is considered to be an erroneous correspondence is performed by the erroneous correspondence deletion unit 5. Each correspondence

[0037]

(Equation 7)

On the other hand, the ratio of the likelihood of the distribution of the incorrect correspondence

[0039]

(Equation 8)

Is calculated (steps 102 and 103).
P _false (ν) corresponds

[0041]

(Equation 9)

Represents the probability of being erroneous.

When P _false (n) is equal to or greater than a certain threshold value D _th (step 104), the data is determined to be incorrect and is deleted (step 105). After the processing is completed for all the responses (steps 106 and 107),
The parameters are recalculated for the remaining correspondences to obtain a final result (step 108). For the calculation of this parameter, an ordinary least-square error estimation amount may be used.

Next, a method of obtaining the maximum likelihood estimator of the mixed model will be described. A set of the correspondence of the coordinates stored in the correspondence storage unit and the correct correspondence probability

[0045]

(Equation 10)

On the other hand, in order to obtain θ by the maximum likelihood estimation, the EM method which is a sequential parameter updating method is used.

The EM algorithm is a sequential method for obtaining a maximum likelihood estimator when there are hidden variables that are not observed. Here, the general method will be briefly described. Consider a family of probability density functions {p (v, u; θ)} on variable (v, u) with parameter θ. The random variable v is an observable variable and its sample can be obtained, but the variable u is called a hidden variable and the parameter θ
Assume that the sample cannot be taken in estimating. Log likelihood if all variables can be observed

[0048]

[Equation 11]

It is sufficient to maximize

[0050]

(Equation 12)

Since it is not possible to use the parameters obtained as a sequential method,

[0052]

(Equation 13)

Then, for the hidden variable u,

[0054]

[Equation 14]

The conditional average when given

[0056]

(Equation 15)

The value of θ that maximizes

[0058]

(Equation 16)

It is assumed that The process of calculating the conditional average is called E step, and the process of calculating log likelihood is called M step. The EM algorithm seeks to maximize the likelihood by repeating these two steps.

When the EM algorithm is applied to the present invention, modeling is performed as follows. That is, for the obtained correspondence (x, y, d), a label u indicating whether this is a correct correspondence or an incorrect correspondence is introduced as a hidden variable. u is a two-dimensional variable, and takes any value of (1, 0) or (0, 1). Statistical models including hidden variables are

[0061]

[Equation 17]

Is obtained. Marginal distribution p for this model
Since (y | x, d; θ) matches the equation (3), if the maximum likelihood estimation for the above equation can be performed, it is also the maximum likelihood estimation for the equation (3). E described above for this model
Applying the general framework of the M algorithm, the expected value for the hidden variables is

[0063]

(Equation 18)

Is calculated. here,

[0065]

[Equation 19]

Is defined as follows.

[0067]

(Equation 20)

Here, in order to simplify the notation,

[0069]

(Equation 21)

[0070] Therefore, this maximization can be solved by minimizing the weighted square error of the following equation (6).

This EM method is executed in a procedure as shown in FIG. First, the parameter θ is initialized, and the initial parameter is set to θ (0) (step 201). Assuming that the estimated amount θ (t−1) is obtained at the t-th step,
Each correspondence

[0072]

(Equation 22)

For the above

[0074]

(Equation 23)

Is calculated (steps 202 and 203).

Estimated amount of the t-th step

[0077]

(Equation 24)

Is

[0079]

(Equation 25)

Is obtained as the solution of Especially f (x; θ)
Is an affine transformation,

[0081]

(Equation 26)

To write

[0083]

[Equation 27]

Is calculated (step 204). These calculations are repeated until the end determination condition is satisfied (steps 205 and 206), and the final estimator

[0085]

[Equation 28]

Is obtained (step 207). As shown in the processing flow, the number of repetitions may be used as the termination determination condition, or the termination may be performed if the amount of change in the estimated amount is within a certain threshold.

Next, a description will be given of a method of determining an initial value when performing the repeated calculation of the EM method. In an iterative operation such as the EM method, the final result is easily affected by the initial value. Therefore, it is better to start the calculation from a good initial value. Therefore, in the present invention, a W estimation amount well known as robust estimation is used. The calculation of the W estimation amount is performed according to a processing flow as shown in FIG.

First, an appropriate weight function ρ (u) is determined.
This is a function to reduce the contribution of data with large residuals, for example

[0089]

(Equation 29)

And the like.

First, the initial value of the weight

[0092]

[Equation 30]

Are initialized with appropriate values, for example, all having the same value (step 301). When the weight of the (t−1) th stage is determined, the estimated value of the tth stage is:
Weighted squared error

[0094]

(Equation 31)

Calculate an estimator that minimizes In the case of affine transformation

[0096]

(Equation 32)

(Steps 302 and 30)
3). Using this estimator, new weights

[0098]

[Equation 33]

Is calculated (step 304). This calculation is repeated (steps 305 and 306). The estimation amount finally obtained in step 307 becomes the initial parameter in step 201 in FIG.

Further, the present embodiment is effective even when there is nonlinearity in image conversion. First, after performing estimation by the affine transformation by the method of the present invention described above, the residual by the affine transformation is obtained as nonlinear data.

[0101]

(Equation 34)

Is calculated. Then the mean square residual

[0103]

(Equation 35)

If the threshold value is equal to or larger than a certain threshold value, residual compensation by nonlinear conversion is performed. Therefore, a nonlinear function system

[0105]

[Equation 36]

Prepare the data

[0107]

(37)

The optimum parameter ω is determined by using the squared error minimum criterion. As the nonlinear function system, a system such as a polynomial system can be used, but in this embodiment, a multilayer perceptron model is particularly used. Here, a three-layer percebutron model having H intermediate elements will be described. This model has parameters

[0109]

(38)

In contrast,

[0111]

[Equation 39]

This is a non-linear model defined by a functional system. Where s (t) is called the sigmoid function

[0113]

(Equation 40)

This is the form. Since the sigmoid function can express local expansion and contraction, nonlinear compensation by the three-layer perceptron is considered to be able to model the local expansion and contraction in a certain direction (especially the scanning direction) often seen in facsimile images. .

To optimize the parameters of the three-layer perceptron, error back propagation, which is a known technique, is used. The parameters obtained in this way are

[0116]

[Equation 41]

Then, when calculating the transformation of the image,

[0118]

(Equation 42)

The destination is calculated by the following.

<Embodiment 2> The present invention is not limited to the above-described embodiment, but can be realized by software. When the present invention is realized by software,
As shown in FIG. 5, a computer system including a CPU, a ROM, a RAM, a display device, a hard disk, a keyboard, a CD-ROM drive, a scanner, and the like is prepared, and a computer-readable recording medium such as a CD-ROM includes A program for realizing the image alignment function and the character recognition function of the invention is recorded. Also, FA
An image such as a form input from X or the like is temporarily stored on a hard disk or the like. When the program is started, the temporarily stored image data is read, the image registration process is executed, a specific area of the form image is recognized by a character, and the recognition result is output to a display or the like.

[0121]

As described above, according to the present invention, it is possible to perform robust (robust) parameter estimation of a conversion function with few errors in image registration such as FAXOCR. As a result, the accuracy of specifying a recognition area in FAXOCR is improved, and the recognition rate of character recognition is improved.

[Brief description of the drawings]

FIG. 1 shows a configuration of a first exemplary embodiment of the present invention.

FIG. 2 shows a processing flowchart of the present invention.

FIG. 3 shows a flowchart of parameter calculation by the EM method.

FIG. 4 shows a flowchart for calculating a W estimation amount.

FIG. 5 shows a configuration example of a second embodiment when the present invention is realized by software.

[Explanation of symbols]

 1 Correspondence coordinate acquisition unit 2 Correct correspondence probability calculation unit 3 Correspondence storage unit 4 Mixed model parameter estimation unit 5 Erroneous correspondence deletion unit

Claims (8)

[Claims] 1. An image registration method for obtaining a coordinate correspondence between a reference image and a processing target image obtained by performing a predetermined geometric transformation on the reference image, the method comprising extracting the reference image from the reference image. When calculating the parameters of the conversion function based on the correspondence between the coordinates of the extracted pattern and the coordinates of the same pattern in the processing target image extracted by the image matching, it indicates the correctness of the correspondence based on the matching distance of the pattern Calculating the probability, using the probability value and the degree of deviation from the conversion function calculated from the corresponding coordinates, deleting the correspondence determined to be incorrect, and recalculating the parameters of the conversion function. An image alignment method characterized by the following. 2. A distribution of occurrence of the correspondence of the coordinates is defined as a distribution that generates a correct correspondence and a distribution that generates an erroneous correspondence.
When the model is modeled as one mixed model, and the probability of occurrence from a correct correspondence is expressed by using a probability indicating the correctness of the correspondence based on the pattern matching, a parameter of a conversion function that maximizes the likelihood is obtained. 2. The image according to claim 1, wherein the ratio of the likelihood of the distribution of the erroneous correspondence is obtained for each correspondence, and the correspondence having the ratio higher than a predetermined threshold value is determined to be the erroneous correspondence. Alignment method. 3. When obtaining a parameter of a conversion function for maximizing the likelihood, performing a sequential calculation by the EM method using a variable indicating whether the correspondence is correct or incorrect as a hidden variable. 3. The method according to claim 2, further comprising the steps of: 4. The method according to claim 1, wherein an initial parameter for performing the sequential calculation by the EM method is obtained by a robust estimation method that weights the data having a large residual so that the contribution is small. Item 3. The image alignment method according to Item 3. 5. After dividing the transformation function into an affine transformation part and a non-linear part and estimating the parameters of the transformation function by the affine transformation, when the magnitude of the residual exceeds a predetermined threshold, a nonlinear function system is formed. 2. The image registration method according to claim 1, wherein additional estimation of the used residual portion is performed. 6. The image registration method according to claim 5, wherein a multilayer perceptron model is used as the nonlinear function system. 7. A method according to claim 1, wherein coordinates of an image to be processed received from a facsimile and a reference image stored in advance are associated with each other to correspond to an area designated by said reference image. A facsimile character recognition method, wherein character recognition is performed on an area in the image to be processed. 8. A coordinate of a pattern extracted from the reference image in order to establish a coordinate correspondence between the reference image and a processing target image obtained by performing a predetermined geometric transformation on the reference image. And a function of calculating a parameter of a conversion function based on the correspondence with the coordinates of the same pattern in the processing target image extracted by the image matching, and calculating a probability indicating the correctness of the correspondence based on the matching distance of the pattern. Using the function, the probability value, and the degree of deviation from the conversion function calculated from the corresponding coordinates, a function of deleting the correspondence determined to be an erroneous correspondence, and a function of recalculating the parameters of the conversion function Computer-readable recording medium on which a program for causing a computer to realize the above is recorded.