JP2007272572A - Business form image distortion detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent distortion of a business form on an image, erroneous reading of a character recognition result, a blank, deviation of an actual entered character from a reading area, or the like by detecting the distortion of the business form generated on the image. <P>SOLUTION: This business form image distortion detection method comprises steps: for detecting angles A, B, C, D and respective sides AB, AD, BD, CD of the image of the business form configured by four sides as shown in Figure 5; for finding a figure formed by one side and angles of both ends thereof about the side; and for detecting the distortion of the image of the business form by size of the found figure. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a form distortion determination apparatus that sets a form on a mat and determines the distortion of a form image using a non-contact scanner device that reads an image from the top with a sensor unit.

Currently, non-contact scanners for acquiring images of various papers such as forms and identification documents brought in by customers are used in the counter business of financial sales offices. The non-contact type scanner is a type of scanner that acquires an image of a form placed on the mat portion by a camera installed above the mat portion on which the form is placed. In the counter business of a financial sales office, the non-contact type scanner is connected to the counter terminal of the financial sales office counter, and image acquisition control is performed. The non-contact type scanner has features such as a simpler operation for image acquisition and a shorter time required than a flat bed type scanner or a sheet feed type scanner. (For example, Non-Patent Document 1)
The form image acquired at the window is subjected to predetermined character recognition processing by an optical character recognition device, and the contents of entry are converted into data and used for financial transactions. The character recognition process prepares a reading format in which a reading position is recorded in advance for each form, specifies the reading position on the image with reference to the corner of the form (for example, the upper left corner), and cuts out and recognizes the character at the corresponding position. Further, the cut-out reading area image is stored together with the character recognition result to be used as a confirmation image of the character recognition result.
Oki Technical Review January 2002 / No. 189 Vol. 69 No. 1 “Color Direct Scanner”

  As described above, the non-contact type scanner has a structure in which an image is acquired by photographing a form placed on the mat portion from above with a camera. Even if it is not distorted, the form is distorted on the image depending on how the form is placed.

  For example, FIG. 3B expresses the form distortion when the image is acquired in a state where the upper left corner of the form is raised. When the lift occurs, the corner of the form moves outside the field of view of the camera as it appears from the camera. For this reason, the corner portion serving as a reference of the character area of the form is shifted as if it extends to the upper left, and when character recognition is performed, the character area is erroneously corrected to the upper left as shown in the figure. Since all character area positions on the form are determined with reference to the corner of one form, the other character areas are also shifted uniformly. For this reason, the entire form has an influence such as a misrecognized character recognition result, a blank space, or a loss of digits due to an actual entry character being removed from the reading area. In such a case, the confirmation image of the character area is also cut out in a shifted manner, so that it is not only difficult to correct, but also difficult to notice digit loss. For example, the character area in the lower right part of FIG. 3B is completely deviated from the actual entry position because the reference corner is shifted. In such a case, it may be regarded as blank even though there is actually an entry. In such cases, financial transaction accidents may occur.

Even when the form does not float as described above, image distortion may occur depending on the performance of the camera itself used as the scanner. For example, a barrel distortion due to the influence of distortion caused by the optical properties of the lens can be mentioned. FIG. 3C shows an example when a barrel distortion occurs. In this case, because the form is distorted into a barrel shape, the entered characters move outward and are out of the reading area. In such a case, an overwritten character is not recognized and a digit loss occurs, which may cause a financial transaction accident.
Therefore, when such a form image distortion occurs, it is necessary to invalidate the image, correct the lift of the form, change the scanner, etc., and let the operator obtain the form image again. .

In order to solve such a problem, the form image distortion detection method of the present invention has the following configuration.
<Configuration 1>
A step of detecting each side and corner of an image of a form consisting of four sides, a step of obtaining a corner and a figure composed of the sides with respect to one side and both corners thereof, and a step And detecting the distortion of the image of the form according to the size of the figure.
<Configuration 2>
The figure is a triangle of one side and its corners, with the corners at the both ends as two vertices and the selected point on the side as another vertex.
<Configuration 3>
It is determined that distortion has occurred when at least one of the graphic sizes exceeds a predetermined value.
<Configuration 4>
When the combination of figures corresponds to a predetermined distortion pattern model, it is determined that distortion has occurred.
<Configuration 5>
The distortion pattern model includes a combination of the size of the figure, the generation position, and the generation direction.
<Configuration 6>
A step of detecting each side and corner of a four-side form image, a step of dividing one side, a step of obtaining a figure composed of a line connecting the divided side and the division point, and each side When the combination of figures obtained for the above corresponds to a predetermined distortion pattern model, it is determined that distortion has occurred.
<Configuration 7>
The figure is a triangle connecting the divided sides and corners or dividing points at both ends, and the distortion pattern model is a combination of the size of the triangle, the generation position, and the generation direction.
<Configuration 8>
The triangles are compared based on normalized size.
<Configuration 9>
Detecting corners at both ends of each side of a four-side form image, obtaining a diagonal line, obtaining a size of a figure connecting the corners of each side and the intersection of the diagonal lines; The method includes a step of detecting distortion of the image of the form based on a difference in size of the graphic.
<Configuration 10>
The figure is either a triangle connecting the corners of each side and the intersection of the diagonal lines, or a figure surrounded by a line connecting each side and the intersection of the corner and the diagonal line.
<Configuration 11>
The difference in the size of the graphic is obtained by comparing the opposing graphics centering on the intersection of the diagonal lines.
<Configuration 12>
The difference in size of the graphic is obtained by comparing adjacent graphics with the diagonal line.
<Configuration 13>
The difference between the sizes of the figures is a size ratio, and it is determined that distortion has occurred when the ratio corresponds to a predetermined distortion pattern model.

  According to the present invention, it is possible to detect a distortion of a form generated on an image.

  Examples of the present invention will be described as follows.

A first embodiment of the present invention will be described as follows.
<Configuration of Example 1>
The configuration of this system is shown in FIG.

This system includes a non-contact scanner device 1, an image processing device 2, and a PC 3.
In FIG. 1, the non-contact scanner device 1 is shown as viewed from above. The non-contact scanner device 1 includes a mat 13 and a camera 13 serving as an image acquisition unit arranged so as to be positioned above the mat 11 by a support member 12.

  Although not shown, a scanner device control unit having a control function of the entire non-contact scanner device 1 and a communication function with the image processing device 2 is provided.

  The mat unit 11 is a table for installing a form for acquiring an image. The mat portion 11 is represented by shading in FIG. 1, but a material of a color such as black that can be easily distinguished from a form on the image is used. As a result, the acquired form image has a mat portion as a background, so that the form appears to rise against the black background color. For this reason, it is possible to easily detect the sides of the form.

  The camera 13 is a camera using a CCD, a C-MOS, or the like that has a function of photographing a form on the mat portion 11 and converting it into image data.

  The image processing apparatus 2 identifies the type of the form from the acquired form image, further identifies the area of the form to be recognized, and performs character recognition. At that time, it is simultaneously determined whether or not the form image is distorted. In the present invention, it is expressed that distortion occurs in the form image when distortion occurs in the form portion of the form image.

  The PC 3 is a terminal for controlling the entire system, and a general PC or the like having a display unit such as a memory and a CRT (not shown) and an operation unit such as a keyboard mouse is used.

  Here, the image processing apparatus 2 is described as being realized by independent hardware, but may be software provided in the PC 3.

Next, the configuration of the image processing apparatus 2 will be described with reference to FIG.
The image processing apparatus includes a control unit 21, a form detection processing unit 22, a distortion detection processing unit 23, a distortion pattern DB 24, a character recognition processing unit 25, a form format DB 26, and a storage unit (not shown). In the present invention, DB is an abbreviation for database.

  The control unit 21 controls the entire image processing apparatus 2 or each processing unit.

  The form detection processing unit 22 is a processing unit that detects the outer shape of the form part from the acquired form image. In the detection of the form part, the outline of the form part is detected by finding a pixel of a part that changes from black in the background part of the form image to white in the form part. Specifically, from the origin of the coordinates of the form image (for example, the upper left), it is confirmed whether the pixel value is black or white sequentially, and if a white pixel is found, the first form edge (the black pixel of the background part and the form part) The white pixel adjacent to the black pixel is sequentially searched as a portion where the white pixel is adjacent. The found pixels at the edge of the form are stored sequentially. In this way, when the pixel at the first form edge is reached, the outline of the form part is detected as the coordinates of the white pixels at the continuous form edge.

The distortion detection processing unit 23 is a processing unit that detects a distortion of a form based on the detected outline of the form.
The distortion detection processing unit 23 further includes a form side detection unit 232, a form angle detection unit 231, a form distortion feature extraction unit 233, and a form distortion determination unit 234.

  The form corner detection unit 231 is a processing unit that detects the corner of the form from the outline of the form. If the amount of change in the coordinates of the form edge is calculated sequentially while moving the detected form edge in a certain direction, the amount of change in each of the x-coordinate (horizontal direction) and y-coordinate (vertical direction) changes uniformly or smoothly in the side portion. To come. For example, in the case of the vertical right side, if it is exactly vertical, the amount of change in the x coordinate is 0 and the amount of change in the y coordinate is 1 when the pixel at the end of the form is moved pixel by pixel from top to bottom. When the corner of the form is reached, this time, the horizontal lower side is moved pixel by pixel from right to left. Here, when the pixel at the end of the form is moved to the pixel at the end of the next form, the change amount of the x coordinate turns to −1 and the change amount of the Y coordinate turns to 0. The amount of change is constant while moving along the horizontal side. If this is expressed as a vector, it will be a downward vector while moving on the right side, but it will change by 90 ° at the corner and become a leftward vector. In this way, by tracking the pixels at the edge of the form and calculating the amount of change in coordinates, a point where the direction of change in coordinates changes can be detected, and this is detected as the corner of the form. The detected corner coordinates are stored in the storage unit.

  Even in the case where the form is distorted as described above, the change of the vector is gentle on the side, and when it reaches the corner, it changes by approximately 90 °, and thus can be detected.

  As the resolution increases, it becomes easier to detect minute disturbances at the edge of the form. However, in order to prevent such disturbances from affecting corner detection, the above corner detection is performed using a plurality of pixels (for example, 8 × 8, 16 × 16, etc.). Are averaged to obtain a luminance average value and tracked as a pseudo pixel.

  The form side detection unit 232 detects the end of the form between the corners as a side as a result of the above corner detection. The sides are detected as a set of pixel coordinates along the form edge and stored in the storage unit. The form side detection unit 232 may be incorporated in the form angle detection unit 231 to perform side detection in parallel with the corner detection.

The form distortion feature extraction unit 233 is a processing unit that extracts a form distortion feature based on the detected form angle and the form side. The form distortion feature extraction method in this embodiment will be described later.
The form distortion determination unit 234 compares the extracted form distortion feature with the distortion pattern acquired from the distortion pattern DB, and detects whether or not the form image is distorted depending on whether or not the corresponding distortion pattern exists. In addition, it is possible to grasp what kind of distortion has occurred.

Even if there is a corresponding distortion pattern, it is determined that there is no distortion if the distortion is not more than a predetermined amount.
Even if there is no distortion corresponding to the distortion pattern DB, if the form distortion feature amount is large, it can be detected as a distortion other than the expected distortion.

  The distortion pattern DB 24 is a DB that stores a distortion characteristic of a form assumed in advance as a distortion pattern.

  The character recognition processing unit 25 is a processing unit for performing character recognition processing of a form when it is determined that there is no distortion, and includes a recognition area correction unit 251 and a character recognition unit 252.

  The recognition area correction unit 251 corrects the position of the reading area where character recognition is performed based on the corner serving as the reference of the form, and the character recognition unit 252 performs character recognition of the reading area.

  The reading area is defined as a form format, and the form format is registered in the form format DB. The form format is extracted from the form format DB by identifying the form and specifying the type of form.

  Note that each processing unit described above is realized by a program that operates in the image processing apparatus 2, but may be realized by hardware as long as it exhibits an equivalent function. Each DB is realized by a file on the storage device, but may be configured in the same DB as long as the logical configuration is independent.

Here, the processing of the form distortion feature extraction unit 233 in the present embodiment will be described in detail.
FIG. 5A is a diagram illustrating the distortion of the form image that occurs because the left end of the form has floated.
Note that, in common with the following description, a form is expressed as a graphic ABCD with four corners A, B, C, and D.

In FIG. 5A, side AB and side AD are curved in the form ABCD due to distortion. Here, as a distortion feature, a figure drawn by a side and corners at both ends thereof is obtained. Here, consider a triangle drawn by one point selected on the side and the corners at both ends. The following can be considered as an example of how to select one point for return.
(A) One point on the side AD is set as L1, and the area of the triangle ADL1 is calculated while moving L1, and the point having the maximum area is obtained.
(B) The side AD is approximated by a curve, and a point where the curvature is maximum is obtained.
(C) A point where the distance in the normal direction from the line segment AD connecting the corners is maximum (that is, a point farthest from the line segment AD) is obtained.
The points selected in this way are called feature points.

  In the case of FIG. 5A, for the sides AD and BC, the feature points L1 and L2 are extracted by the method as described above, and two triangles T101 and T102 are obtained.

  No special vertex is detected for side BC and side CD. That is, for example, in the case of (A), since the points A, D, and L1 are aligned on a straight line, a triangle cannot be drawn and the area is always 0. In the case of (B), the curvature is always 0. In the case of (C), since the line segment AB and the side AB coincide with each other, the distance is always 0. In such cases, feature points are not detected.

  Further, the generation direction of the figure is determined depending on whether the triangle is generated on the inner side or the outer side of the side. In the case of FIG. 5A, since the triangle is generated outside the side, it is expressed that both the triangles T101 and T102 are generated in the outward direction.

These sides of the figure, the area of the figure, and the direction of the figure are combined and extracted as a form distortion feature. In the case of FIG. 5A, it is expressed as follows.
Generation location 1 = side AB, area = (area of T102), direction = outward generation location 2 = side AD, area = (area of T101), direction = outward direction

  In addition, it is good also as a polygon by taking a plurality of feature points instead of a triangle. In this case, the feature point extraction condition may be a predetermined value or more instead of the maximum value for the above (A) to (C), and pixels may be selected every predetermined number.

  Alternatively, the area of the figure surrounded by the side and the line segment may be obtained without taking the feature point and extracted as the area of the above-described form distortion feature. For example, if a figure is obtained like the shaded portion in FIG. 5B, the area does not vary depending on how to take the feature points as in the above (A) to (C), and a stable feature amount can be obtained. Can do. In this case, if the area of the portion surrounded by the dividing points is obtained for a predetermined number of sides, the area of the shaded portion in FIG. 5B can be approximately obtained with a smaller amount of calculation.

  In addition, there is a case where distortion caused by the above-described optical characteristics of the camera occurs instead of lifting the form. FIG. 6A is an explanatory diagram when a barrel distortion occurs due to the influence of distortion of a camera lens.

The distortion characteristics in this case can also be obtained as described above. The distortion feature of FIG. 6A is expressed as follows.
Generation point 1 = side AD, area = (area of T103), direction = inward direction Generation point 2 = side BC, area = (area of T104), direction = inward direction

Similarly, a pincushion distortion as shown in FIG. 6B may occur as distortion generated by the lens. The distortion characteristics in this case can also be obtained as described above. The distortion feature of FIG. 6B is expressed as follows.
Occurrence location 1 = side AD, area = (area of T105), direction = outward occurrence location 2 = side BC, area = (area of T106), direction = outward direction

  As described above, by including the direction in which the graphic is generated in the distortion feature, it is possible to specify the type of distortion even if the area of the graphic is the same. In the case of FIG. 6, it is a barrel type if the generating direction is the inward direction, and is a thread winding method if it is the outward direction.

Here, the distortion pattern registered in the distortion pattern DB 24 includes the following information.
-Occurrence location (edge where the figure is generated)
・ Area (threshold of figure area. If it is larger than this size, it is considered as the corresponding distortion)
-Direction (whether the generated figure is inside or outside the side)
Further, the type of distortion is stored according to the above combination. For example, as shown in FIGS. 5A and 5B, if the pattern is generated on the outside of the upper and left sides of the form, the type of distortion is “lifting the upper left corner” by the “form placement method”. As shown in FIG. 6A, if the pattern is generated on the inside of the left and right sides of the form, the distortion type is “barrel distortion” due to “lens characteristics”. As shown in FIG. 6B, if the pattern is generated on the outside of the left side and the right side of the form, the distortion type is “pincushion type distortion” due to “lens characteristics”.

  In addition, it is assumed that an empty pattern (there is no part where a figure is generated) is also registered as a pattern when there is no distortion in the form image.

<Operation of Example 1>
Next, the operation of this embodiment will be described with reference to FIG.

  S101: First, a non-contact scanner device 1 acquires a form image.

  S102: The form detection processing unit 22 detects a form part from the form image.

  S103: The form angle detection unit 231 detects the form angle.

  S104: The form side is detected by the form side detecting unit 232.

  S105: The form distortion feature extraction unit 233 extracts the form distortion feature.

  S106: The form distortion determination unit 234 searches for a distortion pattern from the distortion pattern DB. At this time, a distortion pattern to be searched for may be selected by calculating a distance from the extracted form distortion feature.

  S107: The form distortion determination unit 234 determines whether there is a pattern corresponding to the extracted form distortion feature in the searched distortion pattern. In this embodiment, the corresponding pattern is a distortion pattern in which the extracted form distortion feature, the generation position, and the generation direction are the same. If there is a corresponding pattern, the process proceeds to S108. If there is no corresponding pattern, the process proceeds to S109.

  S108: It is determined whether or not distortion has occurred by determining whether or not the distortion feature amount of the form is within a threshold value. In the present embodiment, the determination is made based on whether the area of the figure on each side included in the extracted form distortion feature exceeds the area included in the corresponding distortion pattern. At this time, if the threshold is exceeded on any side, it is determined that distortion has occurred. When the corresponding distortion pattern is not generated, that is, when it is an empty distortion pattern, there is no area to be compared, and therefore it is automatically determined that it is within the threshold value.

  S109: When it is determined that distortion has occurred, a form reject is displayed. At this time, the type of distortion may be displayed according to the corresponding distortion pattern. For example, if the form in Fig. 5 is lifted up, the image distortion caused by the "form placement method" has occurred, depending on the type of distortion included in the distortion pattern. Please display the form again. " Further, in the case of FIG. 6A, a display such as “There is“ barrel distortion due to “lens characteristics”. Please check the scanner. ”” Is displayed. As a result, the operator can take measures such as repositioning the form and redoing the process or changing the scanner.

  S110: The character recognition processing unit 25 identifies the form image, identifies the form type, and acquires the corresponding form format from the form format DB. The position of the reading area on the form image is specified based on the position information of the reading area of the form format and the coordinates of the reference point of the form.

  S111: The image of the reading area is cut out and character recognition is performed. The character recognition result is stored together with the cut image.

  If it is sufficient to detect only the occurrence of distortion regardless of the type of distortion, the position and direction where the distortion occurs as described above is not considered, and on either side, the size is equal to or larger than a predetermined size. If a figure having an area is generated, the occurrence of distortion may be detected. In that case, the comparison with the distortion pattern is not performed, and the predetermined value may be compared in consideration of only the area of the generated graphic.

  For example, when the distortion pattern of the form is referred to in S107 but only the occurrence of distortion needs to be detected, only the area of the generated figure, that is, the threshold value of the figure size is held, and the area larger than that is retained. It is only necessary to determine whether or not the figure that has been generated. That is, S107 may be omitted and the determination of S108 may be performed.

  According to the present embodiment, for each form side, the generation of the figure by the corners of the form side and both ends of the side, the area of the figure, and the generation direction of the figure are used as the form distortion feature. The comparison made it possible to specify the presence or absence of distortion and the type of distortion. In addition, it is possible to detect only the presence or absence of distortion by comparing only the area of the figure formed by the edges of the form and the corners of the edges with a predetermined threshold.

Next, a second embodiment of the present invention will be described as follows.
In the present embodiment, the present invention is applied to a case where the form side is in a complicated form on a wave having a plurality of curves.

<Configuration of Example 2>
Since the configuration of the second embodiment is the same as that of the first embodiment, only the differences will be described below.
The second embodiment is different from the first embodiment in three forms: a form distortion feature extraction unit 233, a form distortion determination unit 234, and a distortion pattern DB 24.

Here, the processing of the form distortion feature extraction unit 233 in the present embodiment will be described in detail.
In this embodiment, as in the first embodiment, as a distortion feature, a figure drawn by a side and corners at both ends thereof is obtained, but the intersection of a line segment connecting the side and corners at both ends is also added. Ask for. FIG. 7 is a diagram showing the distortion of the form image generated because the form is placed on the mat portion 11 with the right side of the form being wavy. In FIG. 7, due to the distortion, the side BC of the form ABCD is curved like a wave.

  First, P1 and P2 are obtained as intersections of the side BC and the line segment BC. With this as a dividing point, the side BC is divided into a partial side BP1, a partial side P1P2, and a partial side P2C.

By obtaining a graphic for each of these partial sides in the same manner as in the first embodiment, the following form distortion characteristics can be obtained.
Generation location 1 = side BC, number of divisions = 1/3, area = (area of T111), direction = outward generation location 2 = side BC, number of divisions = 2/3, area = (area of T112), direction = Inner direction Occurrence location 3 = side BC, number of divisions = 3/3, area = (area of T113), direction = outward Here, the number of divisions is the number of sides delimited by side denominator BC, the partial sides of the numerator It indicates which one of them.

  As described in the first embodiment, the figure to be obtained may be either a triangle, a polygon, or a figure itself surrounded by a line segment connecting the partial sides and the ends of the partial sides. .

Here, the distortion pattern registered in the distortion pattern DB 24 includes the following information.
-Occurrence location (edge where the figure is generated)
-Number of divisions (whether the number of divided sides or divided partial sides)
・ Area (threshold of figure area. If it is larger than this size, it is considered as the corresponding distortion)
-Direction (whether the generated figure is inside or outside the side)
Further, the type of distortion is stored according to the above combination. For example, in the case of the pattern shown in FIG. 7, the type of distortion is “Rippling on the right side” by “Form placement method”.

  In addition, it is assumed that an empty pattern (there is no part where a figure is generated) is also registered as a pattern when there is no distortion in the form image.

<Operation of Example 2>
Next, the operation of this embodiment will be described with reference to FIG.

  S101 to S106 are the same as those described in the first embodiment. However, the processing of the form distortion feature extraction unit 233 is as described above.

  S107: The form distortion determination unit 234 determines whether there is a pattern corresponding to the extracted form distortion feature in the searched distortion pattern. In this embodiment, the corresponding pattern is a distortion pattern in which the extracted form distortion feature, the generation position, the number of divisions, and the generation direction are the same. If there is a corresponding pattern, the process proceeds to S108. If there is no corresponding pattern, the process proceeds to S109.

  S108: It is determined whether or not distortion has occurred by determining whether or not the distortion feature amount of the form is within a threshold value. In this embodiment, the determination is made based on whether or not the area of the graphic for each partial side of each side included in the extracted form distortion feature exceeds the threshold value of the area included in the corresponding distortion pattern. At this time, if the threshold is exceeded on any side, it is determined that distortion has occurred. When the corresponding distortion pattern is not generated, that is, when it is an empty distortion pattern, there is no area to be compared, and therefore it is automatically determined that it is within the threshold value.

  S109: When it is determined that distortion has occurred, a form reject is displayed. At this time, the type of distortion may be displayed according to the corresponding distortion pattern. For example, in the case of a wave on the side of the form in FIG. 7, depending on the type of distortion included in the distortion pattern, “image distortion occurred due to the“ placement method of the form. ”“ Waving on the left side of the form ” Please reposition the form. ”Is displayed.

  S110 to S111 are the same as those described in the first embodiment.

If it is sufficient to detect only the occurrence of distortion regardless of the type of distortion, it does not take into account the position where the distortion occurs, the number of divisions, and the direction, and any side or partial side. If a figure having an area of a predetermined size or more is generated, the occurrence of distortion may be detected. In that case, the comparison with the distortion pattern is not performed, and the predetermined value may be compared in consideration of only the area of the generated graphic.
For example, in S107, only the area of the generated graphic, that is, the threshold value of the graphic size is held, and it is only necessary to determine whether a graphic having an area larger than that is generated. That is, S107 may be omitted and the determination of S108 may be performed.

  According to the present embodiment, for each form side, in addition to the form side and the corners at both ends of the side, the generation of a figure by the intersection of the line side connecting the form side and the corners at both ends, the partial side divided by the intersection point, Since the area of the figure and the direction of occurrence of the figure were determined and used as a form distortion feature, even if it was a complicated distortion such as a wavy form by comparison with a pre-registered distortion pattern, It became possible to specify the type of distortion. In addition, it is possible to detect only the presence or absence of distortion by comparing only the area of the figure formed by the edges of the form and the corners of the edges with a predetermined threshold.

Next, a third embodiment of the present invention will be described as follows.
In this embodiment, the form distortion feature is obtained using the diagonal line of the form.

<Configuration of Example 3>
Since the configuration of the third embodiment is the same as that of the first embodiment, only the differences will be described below.
The third embodiment is different from the first embodiment in three forms: a form distortion feature extraction unit 233, a form distortion determination unit 234, and a distortion pattern DB 24.

Here, the processing of the form distortion feature extraction unit 233 in the present embodiment will be described in detail.
In the present embodiment, as a distortion feature, a figure drawn by a side and a diagonal line of a form is obtained.

  FIG. 8A shows a form image in a state where no distortion has occurred. The diagonal lines AC and BD are obtained from the detected angles A, B, C, and D, and the intersection point O is obtained. Thereby, line segments OA, OB, OC, and OD connecting O and the respective corners are obtained. In a state where no distortion occurs, OA, OB, OC, and OD have the same length. For this reason, the areas of the triangles T201, T202, T203, and T204 drawn by these line segments and the respective sides are equal.

On the other hand, when distortion occurs, a difference occurs in the area. By obtaining the area ratio of these triangles, distortion can be detected. The area ratio is obtained, for example, as T201: T202: T203: T204 (that is, the upper side: right side: lower side: left side triangle area). At this time, the area of the triangle T201 ′ is normalized as 1.
The area ratio in the case of FIG. 8A is obtained as 1: 1: 1: 1.

On the other hand, FIG. 8B shows a case where distortion occurs in a state where the upper left corner of the form is lifted.
In FIG. 8B, side A′B ′ and side A′D ′ of the form A′B′C′D ′ are curved due to distortion. Here, as a distortion feature, a figure drawn by a diagonal line passing through the side and the corners of both ends thereof is obtained. Here, a triangle drawn by the corners at both ends of the side and the intersection of the diagonal lines is obtained. As a result, four triangles T201 ′, T202 ′, T203 ′, and T204 ′ are obtained.

These areas are not the same as in the normal state of FIG. Therefore, distortion can be detected by obtaining the area ratio of these triangles.
In the case of FIG. 8B, for example, it is obtained as 1: 0.8: 0.8: 1.

FIG. 9 shows another example of distortion. FIG. 9 is an example of trapezoidal distortion that occurs when a form is photographed from an obliquely upper angle. Here, for the sake of simplicity, it is assumed that the angle is generated only in the vertical direction, but the same applies to the case of the horizontal direction.
In the case of FIG. 9, the area ratio is obtained, for example, as 1: 1.5: 2: 1.5.

  By the way, as shown in FIG. 10, when the above-described barrel distortion or yarn winding distortion occurs, the corner position may not be changed. In this case, the area of the triangle formed by the diagonal line and the corner is not different from that in the normal state. : 1: 1: 1. For this reason, you may make it obtain | require the area of the figure not with a triangle but with a form edge and a diagonal line. For example, if T204 ″ (shaded portion) in FIG. 10A or T204 ″ (shaded portion) in FIG. 10B is used, a change in area due to distortion can be correctly extracted. In this way, for example, the area ratio becomes 1: 1.2: 1: 1.2 or 1: 0.7: 1: 0.7, and it can be detected that it is different from the normal state.

Here, the distortion pattern registered in the distortion pattern DB 24 includes the following information.
-Area ratio range-Permissible threshold The area ratio range is the range that the area of the figure on each side can take when the area of the figure on the upper side is 1, and the area ratio in the above examples of each distortion Is stored in such a value that each is included.
The allowable threshold is a value indicating how much the area ratio is allowed.
Further, the type of distortion is stored according to the area ratio. For example, when the areas of the right side and the left side are equal and the area of the figure by the lower side is large, “trapezoidal distortion with the upper side shrunk” is stored.

  Further, it is assumed that 1: 1: 1: 1 is also registered as a pattern when there is no distortion in the form image.

<Operation of Example 3>
Next, the operation of this embodiment will be described with reference to FIG.

  S101 to S106 are the same as those described in the first embodiment. However, the processing of the form distortion feature extraction unit 233 is as described above.

  S107: The form distortion determination unit 234 determines whether there is a pattern corresponding to the extracted form distortion feature in the searched distortion pattern. In this embodiment, the corresponding pattern is a distortion pattern in which the area ratio of the extracted form distortion feature is included in the range. If there is a corresponding pattern, the process proceeds to S108. If there is no corresponding pattern, the process proceeds to S109.

S108: It is determined whether or not distortion has occurred by determining whether or not the distortion characteristics of the form are within a threshold value. At this time, the extracted area ratio is compared with the area ratio allowed in each distortion pattern.
In this embodiment, the determination is made based on whether or not the size of the area ratio included in the extracted form distortion feature exceeds the threshold value included in the corresponding distortion pattern. At this time, the determination is made based on whether or not the difference between the largest number and the smallest number exceeds a threshold value.
When the corresponding distortion pattern is not generated, that is, when it is an empty distortion pattern, it is automatically determined to be within the threshold value.

  S109: When it is determined that distortion has occurred, a form reject is displayed. At this time, the type of distortion may be displayed according to the corresponding distortion pattern as in the above-described embodiment. For example, in the case of FIG. 9, a display such as “Image distortion due to“ trapezoidal distortion of form ”occurred. Please check the scanner” is displayed.

  S110 to S111 are the same as those described in the first embodiment.

  If it is sufficient to detect only the occurrence of distortion regardless of the type of distortion, comparison with the distortion pattern as described above is not performed, and distortion occurs when an area ratio of a predetermined value or more occurs. This may be detected.

  In the above example, the area ratio of figures on each side is obtained, but the difference in area may be obtained. At this time, by comparing the opposing figures across the intersection of diagonal lines, two pairs of figures can be compared with the upper and lower sides, and the right and left sides, and distortion can be detected by the presence or absence of the difference in area. This comparison method can detect distortion other than the barrel mold and the pincushion mold.

  Further, by comparing adjacent figures across diagonal lines, four pairs can be compared for each corner, and distortion can be detected by the presence or absence of the difference in area.

  According to the present embodiment, for each form side, in addition to the corners of both sides of the form side, the area ratio of the figure by the diagonal line connecting the corners is obtained as the form distortion feature. The comparison made it possible to specify the presence or absence of distortion and the type of distortion. In addition, only the presence or absence of distortion can be detected by comparing only the area ratio of the figure with a predetermined threshold.

  In the first and second embodiments described above, normalization may be performed when obtaining the area of the figure. For example, assuming that the length of the longest side of the form or the length of the outer periphery is 1, all lengths are normalized. Alternatively, the obtained area is normalized with the area of the form itself as 1. In this way, a constant threshold value can be used regardless of the form size, and the influence of changes in the form size can be eliminated.

System configuration Functional block of image processing apparatus Example of form distortion Processing flow Example of form distortion feature of embodiment 1 Example of form distortion feature of embodiment 1 Example of form distortion feature of embodiment 2 Example of form distortion feature of Example 3 Example of form distortion feature of Example 3 Example of form distortion feature of Example 3

Explanation of symbols

1 Non-contact scanner device 2 Image processing device 3 PC
DESCRIPTION OF SYMBOLS 11 Mat | matte part 12 Support member 13 Camera 21 Control part 22 Form detection process part 23 Distortion detection process part 232 Form edge detection part 231 Form angle detection part 233 Form distortion feature extraction part 234 Form distortion judgment part 24 Distortion pattern DB
25 Character recognition processing unit 251 Recognition area correction unit 252 Character recognition unit 26 Form format DB

Claims (13)

Detecting each side and corner of a four-side form image;
For one side and the corners at both ends thereof, obtaining a figure composed of the corners at both ends and the sides for the four sides;
The step of detecting distortion of the image of the form according to the size of the figure obtained.
Form image distortion detection method. The figure is
For one side and its corners, the corners at both ends are two vertices, and a point selected on the side is another triangle.
The form image distortion detection method according to claim 1. Determining that distortion has occurred when at least one of the graphic sizes exceeds a predetermined value;
The form image distortion detection method according to claim 1. When the combination of figures corresponds to a predetermined distortion pattern model, it is determined that distortion has occurred,
The form image distortion detection method according to claim 2. The distortion pattern model is composed of a combination of the size of the figure, the generation position, and the generation direction.
The form image distortion detection method according to claim 4. Detecting each side and corner of a four-side form image;
Dividing one side;
Obtaining a figure consisting of a line connecting the divided sides and dividing points;
When the combination of figures obtained for each side corresponds to a predetermined distortion pattern model, it comprises a step of determining that distortion has occurred,
Form image distortion detection method. The figure is
A triangle connecting the divided sides and corners or dividing points at both ends;
The distortion pattern model is composed of a combination of the size of the triangle, the generation position, and the generation direction.
The form image distortion detection method according to claim 6. The triangles are compared based on normalized size,
The form image distortion detection method according to claim 7. Detecting corners at both ends of each side of a four-side form image;
Obtaining a diagonal line; and
Obtaining a size of a figure connecting the corners of each side and the intersection of the diagonal lines; and
Detecting the distortion of the image of the form by the difference in the size of the figure,
Form image distortion detection method. The figure is either a triangle connecting the corners of each side and the intersection of the diagonal lines, or a figure surrounded by a line connecting each side and the intersection of the corners and the diagonal line,
The form image distortion detection method according to claim 9. The difference in the size of the figure is obtained by comparing the opposite figures centering on the intersection of the diagonal lines,
The form image distortion detection method according to claim 10. The difference in size of the figure is determined by comparing the neighboring figures by the diagonal line,
The form image distortion detection method according to claim 10. The size difference between the figures is a size ratio, and when the ratio corresponds to a predetermined distortion pattern model, it is determined that distortion has occurred.
The form image distortion detection method according to any one of claims 10 to 12.

Images