JP2021033814A - Image processing apparatus, program, image processing method, and document form processing apparatus - Google Patents

Abstract

To detect a fault on a document form at a low cost.SOLUTION: An image processing apparatus 100 for processing an image of a document form including ruled lines includes: a storage unit 120 storing information on ruled lines as reference ruled-line characteristics; a ruled-line extraction unit 112 which extracts information on ruled line from an image; a ruled-line comparison unit 113 which compares the ruled-line information extracted by the ruled-line extraction unit 112 with the reference ruled-line characteristics; and a determination unit 114 which determines a fault on the document form when there is a difference as a result of comparison of the ruled-line comparison unit 113.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing device, a program, an image processing method, and a form processing device that detect an abnormality in a form to be read.

A system that reads the contents of a form and executes processing is widespread. For example, an automated teller machine installed at a financial institution performs a transfer process in which the transfer amount stated on the transfer form is deposited into the account number written on the transfer form.
The transfer form is manually inserted into the automated teller machine (user of the automated teller machine), read as an image by a scanner in the automated teller machine, and the account number and transfer amount are recognized from the image. Upon receiving confirmation from the user that the recognized content is correct, the automated teller machine transports the transfer form to the hangar inside the automated teller machine and executes the transfer process.

The transfer paper to be inserted into the automated teller machine may be covered with foreign matter such as sticky notes, torn and partially chipped, or inserted with the edges broken. Even if the account number and amount of money can be read, such a transfer form will be clogged during transportation to the hangar, causing a malfunction of the automated teller machine. For this reason, it is desirable to avoid the occurrence of trouble during transportation by detecting an abnormality when the image is read by a scanner near the slot and returning the image to the slot. In the invention described in Patent Document 1, in a multifunction device provided with a color scanner, a color image of a document and a monochrome image are compared, and it is determined that there is a foreign substance in a region where the difference is large.

Japanese Unexamined Patent Publication No. 2010-246125

In the invention described in Patent Document 1, a color image is required for determining a foreign substance, and a color scanner is indispensable. In devices that process forms, including automated teller machines, it is particularly desirable to detect foreign matter using a monochrome scanner, which is inexpensive and easy to maintain, from the perspective of scanner cost and maintenance.
Adhesion of foreign matter or damage to paper is not limited to equipment failure, but also leads to omission of reading (recognition) of the description on the form. For this reason, not only automatic teller machines but also devices that read forms in general are required to detect foreign matter adhesion and form damage.

The present invention has been made in view of such a background, and it is an object of the present invention to provide an image processing device, a program, an image processing method, and a form processing device capable of detecting an abnormality in a form at low cost. And.

In order to solve the above-mentioned problems, the image processing device according to the present invention is an image processing device that processes an image of a form including a ruled line, and includes a storage unit that stores the information of the ruled line as a reference ruled line feature and the above. There is a difference as a result of comparison between the ruled line extraction unit that extracts the ruled line information from the image, the ruled line comparison unit that compares the ruled line information extracted by the ruled line extraction unit with the reference ruled line feature, and the ruled line comparison unit. Occasionally, it is characterized by including a determination unit for determining that there is an abnormality in the form.

In order to solve the above-mentioned problems, the program according to the present invention causes a computer to function as the above-mentioned image processing device.

In order to solve the above-mentioned problems, the image processing method according to the present invention is an image processing method executed by an image processing device that processes an image of a form including a ruled line, and the image processing device obtains information on the ruled line. , A storage unit for storing as a reference ruled line feature is provided, and a step of extracting ruled line information from the image and a step of comparing the extracted ruled line information with the reference ruled line feature are compared. At a certain time, it is characterized by including a step of determining that there is an abnormality in the form.

In order to solve the above-mentioned problems, the form processing device according to the present invention is a form processing device that processes a form including a ruled line, from a storage unit that stores the information of the ruled line as a reference ruled line feature, and from an insertion slot. A transport control unit that controls the transport of the input form inside the form processing device, an image acquisition unit that acquires an image of the form, a ruled line extraction unit that extracts ruled line information from the image, and the above. A ruled line comparison unit that compares the ruled line information extracted by the ruled line extraction unit with the reference ruled line feature, and a determination unit that determines that the form is abnormal when there is a difference as a result of comparison between the ruled line comparison unit and the ruled line comparison unit. When the determination unit determines that the form is abnormal, the transport control unit returns the form to the insertion slot.

According to the present invention, it is possible to provide an image processing device, a program, an image processing method, and a form processing device capable of detecting an abnormality in a form at low cost.

It is a functional block diagram of the image processing apparatus which concerns on 1st Embodiment. It is a data structure diagram of the form database which concerns on 1st Embodiment. This is an example of an image of a form acquired by the image acquisition unit according to the first embodiment. This is an example of a ruled line extracted by the ruled line extraction unit according to the first embodiment. It is a figure for demonstrating the ruled line feature which concerns on 1st Embodiment. It is a flowchart of the form registration process which concerns on 1st Embodiment. This is an example of a completed form to be read. It is a flowchart of the form reading process which concerns on 1st Embodiment. This is an example of a form with sticky notes. It is a figure for demonstrating the ruled line feature of the form on which the sticky note is attached which concerns on 1st Embodiment. The lower left is an example of a bent form. It is a figure for demonstrating the ruled line feature of the form which the lower left is bent according to 1st Embodiment. It is a block diagram of the form processing apparatus which concerns on 2nd Embodiment.

The image processing apparatus according to the embodiment (embodiment) for carrying out the present invention will be described below. The automated teller machine is one of the image processing devices. The ruled line information of the form to be read (reference form) is registered in advance in the image processing device. The image processing device scans the completed form image with a scanner and acquires ruled line information. The registered ruled line information is compared with the acquired ruled line information, and if the acquired ruled line is missing, the image processing device determines that the form is abnormal.
By doing so, the image processing apparatus can detect an abnormality in the form only with a monochrome image. No color scanner is required, and equipment costs and maintenance costs can be reduced.

≪Image processing device: Overall configuration≫
FIG. 1 is a functional configuration diagram of the image processing device 100 according to the first embodiment. The image processing device 100 includes a control unit 110, a storage unit 120, and an input / output unit 140. The storage unit 120 is composed of a RAM (Random Access Memory), an SSD (Solid State Drive), and the like. The storage unit 120 stores the program 121, the form database 130 (see FIG. 2), and the like. The CPU (Central Processing Unit) constituting the control unit 110 executes the program 121 to execute the image acquisition unit 111, the ruled line extraction unit 112, the ruled line comparison unit 113, the determination unit 114, the character recognition unit 115, and the field, which will be described later. The registration unit 116 is realized, and the form reading process (see FIG. 8 described later) is performed.

≪Image processing device: form database≫
FIG. 2 is a data structure diagram of the form database 130 according to the first embodiment. The form database 130 is, for example, tabular data. One record in the form database 130 indicates information related to the format of one form, and includes an ID (identifier, identification information) 131, a read field 132, and an attribute (column) of a ruled line feature 133.

ID131 is a form identification number, which is a number printed at a predetermined position (for example, upper left) of the form.
The reading field 132 is information indicating an area of information to be read from the form. The area is indicated by a reference position (for example, the upper left coordinate of the area), a width, and a height of the area in the coordinate system set on the image of the form. The read field 132 may include information in a plurality of areas.

The ruled line feature 133 is information related to the ruled line of the form. The form is divided into a plurality of areas, and the ruled line information included in each area is stored in the ruled line feature 133. The ruled line feature will be described with reference to FIG. 5 described later.
The ID of the form, which is the form of the form shown in the record 139, is "22". The upper left coordinate in one area to be read is (25,137), the width is "213", and the height is "60".

<< Image processing device: Input / output unit and control unit >>
Returning to FIG. 1, the description of the input / output unit 140 and the control unit 110 will be continued. The input / output unit 140 includes a scanner 141, a display 142, a keyboard 143, and a mouse 144. The scanner 141 reads the form and outputs the digital data of the image.

The control unit 110 is composed of a CPU, and includes an image acquisition unit 111, a ruled line extraction unit 112, a ruled line comparison unit 113, a determination unit 114, a character recognition unit 115, and a field registration unit 116.
The image acquisition unit 111 acquires digital data of the image of the form by using the scanner 141.

The ruled line extraction unit 112 extracts a ruled line from the image of the form and generates a ruled line feature. A ruled line is a vertical line or a horizontal line having a predetermined length or longer. The ruled line feature will be described with reference to FIG. 5 described later.
The ruled line comparison unit 113 compares the ruled line feature extracted from the completed form with the ruled line feature 133 of the form database 130 (see FIG. 2).

The determination unit 114 determines whether or not there is an abnormality in the form from the comparison result of the ruled line features.
The character recognition unit 115 recognizes and acquires the characters described in the form by using, for example, an optical character recognition technique.
The field registration unit 116 registers the field for reading the characters described in the form in the reading field 132 of the form database 130 according to the instruction of the operator of the image processing device 100.

≪Rules feature≫
The ruled line features will be described with reference to FIGS. 3 to 5.
FIG. 3 is an example of the image 200 of the form acquired by the image acquisition unit 111 according to the first embodiment. The form shown in FIG. 3 is a transfer paper, and "22" is printed as the ID of the transfer paper on the upper left.

FIG. 4 is an example of the ruled line 210 extracted by the ruled line extracting unit 112 according to the first embodiment. The ruled line extraction unit 112 extracts vertical lines and horizontal lines having a predetermined length or longer from the image 200 (see FIG. 3). In FIG. 4, the edge of the form is shown by a broken line, but this is not a ruled line.
In the account number and amount fields, the vertical line separating each digit is not extracted as a ruled line in FIG. 4 because it has a predetermined length or less except for a part connected to other columns. There are vertical and horizontal lines in the characters written on the form, but they are not recognized as ruled lines because they are less than the specified length.

FIG. 5 is a diagram for explaining a ruled line feature according to the first embodiment. The image of the form is divided into a plurality of areas vertically and horizontally as shown by the broken line in FIG. In FIG. 5, not the entire transfer form but only the payment handling slip on the left side is shown. Further, for ease of viewing, the extracted ruled line 210 (see FIG. 4) is shown by a thick line in FIG. The ruled line feature is information on the number and length of vertical and horizontal lines in each area. In the following, each region will be described as a square having a side of 10 lengths.

Region 221 includes one horizontal line of length "5".
Region 222 includes two horizontal lines of length "5" and one vertical line of length "10".
Region 223 includes one vertical line of length "10".
Region 224 includes one horizontal line of length "5" and one vertical line of length "8".
Area 225 does not include a ruled line.
The ruled line feature is information as a set of ruled line information in each of these areas. The ruled line extraction unit 112 extracts a ruled line from the form image and further generates a ruled line feature.

Hereinafter, the form registration process and the form reading process will be described. In the form registration process, the format of the form to be read is registered in the form database 130. The form for which the format has been registered is inspected by the form reading process, and if there is no abnormality, the characters in the registered area are read and output.

≪Form registration process≫
FIG. 6 is a flowchart of the form registration process according to the first embodiment. A process of registering the format of the form to be read (registering the reference form) will be described with reference to FIG.
In step S11, the image acquisition unit 111 acquires the image 200 of the form (see FIG. 3).
In step S12, the character recognition unit 115 recognizes and acquires the ID of the form. The ID is in a predetermined position, for example, in a predetermined area at the upper left of the form. Next, the character recognition unit 115 adds a record to the form database 130 and stores the acquired ID in the ID 131.

In step S13, the ruled line extraction unit 112 extracts the ruled line 210 (see FIG. 4).
In step S14, the ruled line extraction unit 112 generates a ruled line feature. Next, the ruled line extraction unit 112 stores the generated ruled line feature in the ruled line feature 133 of the record added in step S12.

In step S15, the field registration unit 116 stores the read field instructed by the operator of the image processing device 100 in the read field 132. In image 200 (see FIG. 3), the operator specifies the area of the account number column which is 5 + 1 + 7 digits on the payment handling slip on the left side of the image 200 and the area of the amount column which is 8 digits on the right side. To do. Upon receiving the designation, the field registration unit 116 stores the designated area in the read field 132 of the record added in step S12.

≪Form reading process≫
FIG. 7 is an example of the completed form 230 to be read. The account number, amount, payer's address, name, etc. have been entered.
FIG. 8 is a flowchart of the form reading process according to the first embodiment. The reading process of the completed form will be described with reference to FIG.
In step S21, the image acquisition unit 111 acquires the image of the form.

In step S22, the character recognition unit 115 recognizes the ID of the form in the predetermined area. If the character recognition unit 115 succeeds in recognition (step S22 → YES), the character recognition unit 115 proceeds to step S24, and if the recognition fails (step S22 → NO), the character recognition unit 115 proceeds to step S23.
In step S23, the control unit 110 outputs an error and ends the form reading process.

In step S24, the character recognition unit 115 determines whether or not the record corresponding to the form ID recognized in step S22 exists in the form database 130. If the record exists (step S24 → YES), the character recognition unit 115 proceeds to step S25, and if it does not exist (step S24 → NO), the character recognition unit 115 proceeds to step S23.

In step S25, the ruled line extraction unit 112 extracts the ruled line.
In step S26, the ruled line extraction unit 112 generates a ruled line feature.
In step S27, the ruled line comparison unit 113 acquires the ruled line feature 133 (also referred to as the reference ruled line feature) of the record of the form database 130 corresponding to the form ID recognized in step S22, and sets the ruled line feature generated in step S26. Extract areas with differences. Specifically, the ruled line comparison unit 113 compares the information of the ruled lines related to the areas at the same position. The ruled line information includes vertical line or horizontal line, length, and number of lines, and these are compared.

For example, the number of vertical lines and the number of horizontal lines are compared with each other, and if the number of read ruled lines is small, the ruled line comparison unit 113 determines that there is a difference. For example, if the number of read vertical lines is less than a predetermined number of the number of vertical lines in the reference ruled line feature, it is determined that there is a difference. The same applies to horizontal lines.
The ruled line comparison unit 113 may make a comparison by taking the correspondence between the vertical line of the reference ruled line feature and the read vertical line in consideration of the length. For example, the ruled line comparison unit 113 reads, for each vertical line included in the reference ruled line feature, a vertical line having the same length or a difference within a predetermined length even if it is short, the same ruled line. The vertical line of the reference ruled line feature is associated with the read vertical line. If there is no corresponding vertical line in the read vertical lines and the number of vertical lines of the reference ruled line feature is a predetermined number or more, the ruled line comparison unit 113 determines that there is a difference in the area. The same applies to horizontal lines.

In step S28, the determination unit 114 proceeds to step S29 if the number of regions determined to have a difference is within the threshold value (step S28 → YES), and proceeds to step S23 if the number exceeds the threshold value (step S28 → NO).
In step S29, the character recognition unit 115 reads the characters entered in the area in the read field 132 of the record of the form database 130 corresponding to the form ID recognized in step S22. If a plurality of areas are registered in the reading field 132, characters are read from each area.
In step S30, the character recognition unit 115 outputs the ID and the read character.

This completes the form reading process. If there is no abnormality in the form, no error (see step S23) occurs, and the image processing device 100 recognizes and outputs the character in the reading field 132 (see step S30).

≪Example of abnormality≫
The difference in the ruled line characteristics when there is an abnormality in the form will be described below.
FIG. 9 is an example of a form 240 with a sticky note 241 attached. A sticky note 241 is stretched in the center of the form, and some characters and ruled lines are hidden.
FIG. 10 is a diagram for explaining the ruled line characteristics of the form 240 (see FIG. 9) on which the sticky note 241 is stretched according to the first embodiment. Similar to FIG. 5, a plurality of areas in the payment handling slip portion are described. The hatched area 251 is an area where the sticky note 241 (see FIG. 9) is stretched, and the ruled line is not extracted. Therefore, in the area of the ruled line feature in the area 251 where there is a ruled line at the time of registering the form, there is no read ruled line, and the area is different from the ruled line feature registered in step S27 (see FIG. 8). Is judged. As a result, it is determined that the form 240 has an abnormality.

FIG. 11 is an example of a form 260 in which the lower left is bent. A part of the bent form covers the form, and some characters and ruled lines are hidden.
FIG. 12 is a diagram for explaining the ruled line characteristics of the form 260 (see FIG. 11) in which the lower left is bent according to the first embodiment. Similar to FIG. 5, a plurality of areas in the payment handling slip portion are described. The lower left ruled line is not extracted. Therefore, it is determined that the area is different from the ruled line feature registered in step S27. As a result, it is determined that the form 260 has an abnormality.

≪Characteristics of form reading process≫
The image processing device 100 extracts a ruled line from the image of the form and compares it with the registered ruled line feature to determine whether or not there is an abnormality in the read form. Specifically, the image processing device 100 divides the form into a plurality of areas and compares the ruled lines for each area. At the time of comparison, when the registered ruled line disappears (the ruled line cannot be read), the image processing apparatus 100 determines that there is a difference in the area.

By making such a determination, the image processing apparatus 100 can detect an abnormality such as a sticky note stuck, a bent form, or a torn form and a part of the form is missing. The ruled lines entered in the contact column and remarks column of the form are extracted as ruled lines, but the registered ruled lines are not gone (the ruled lines are increasing), so it is judged to be a difference. Therefore, it is not judged to be abnormal.

<< Second Embodiment: Form Processing Device >>
As the second embodiment, the form processing device 100Z will be described. An automated teller machine is also one of the form processing devices 100Z.
FIG. 13 is a configuration diagram of the form processing device 100Z according to the second embodiment. The form processing device 100Z is a device that reads and processes the form 290 input from the insertion slot 191 with the scanner 141. The form 290 is inserted from the insertion port 191, sandwiched between the rollers 192, and rotated by the roller 192, so that the form 290 is conveyed from the insertion port 191 to the hangar 194 along the transfer path 193. During the transport of the form 290, the scanner 141 acquires the image of the form 290 in the transport path 193 near the insertion slot 191.

The differences between the image processing device 100 (see FIG. 1) and the form processing device 100Z of the first embodiment will be described below.
The form processing device 100Z does not include the field registration unit 116. The form registration is performed by the image processing device 100 different from the form processing device 100Z, and the information of the form database 130 of the registration result is stored in the form database 130 of the form processing device 100Z.
The form processing device 100Z includes a transport control unit 117 that controls the roller 192. When the determination unit 114 determines that the form 290 has an abnormality, the transport control unit 117 controls the roller 192 to return the abnormal form 290 to the insertion port 191. By doing so, it is possible to avoid troubles such as paper jams that occur in the transport path 193.

The form processing device 100Z includes a communication unit 150 instead of the input / output unit 140. The ID read from the normal form 290 and the character information entered in the read field are output from the communication unit 150.

≪Other variants≫
The present invention is not limited to the above-described embodiment, and can be modified without departing from the spirit of the present invention. In the above-described embodiment, the ID is read to determine the form format. Information different from the ID, for example, the name of the form written at the top of the form may be read to determine the format of the form.

The image processing device 100 and the form processing device 100Z in the above embodiment include the scanner 141, but the scanner 141 may not be provided. An image read by a scanner provided in another device may be acquired and a form registration process or a form reading process may be executed. Further, regarding the form database 130, information on the form format is stored in the form database stored in another device in the form registration process, and the form format is stored in the form database stored in the other device in the form reading process. You may get the information of.

Although some embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omission and substitution can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in the present specification and the like, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

100 Image processing device 100Z Form processing device 110 Control unit 111 Image acquisition unit 112 Ruled line extraction unit 113 Ruled line comparison unit 114 Judgment unit 115 Character recognition unit 116 Field registration unit 117 Transfer control unit 120 Storage unit 121, 121Z Program 130 Form database 191 Insertion mouth

Claims (7)

An image processing device that processes the image of a form including ruled lines.
A storage unit that stores the ruled line information as a reference ruled line feature,
A ruled line extraction unit that extracts ruled line information from the image,
A ruled line comparison unit that compares the ruled line information extracted by the ruled line extraction unit with the reference ruled line feature.
An image processing device including a determination unit that determines that there is an abnormality in the form when there is a difference as a result of comparison by the ruled line comparison unit. The image processing apparatus according to claim 1, wherein the reference ruled line feature is information on a ruled line extracted by the ruled line extraction unit from an image of a reference form. The information on the ruled line is
The image processing apparatus according to claim 1, wherein each of the plurality of areas for dividing the image includes at least one of the vertical and horizontal types, numbers, and lengths of the ruled lines in the area. The ruled line comparison unit
For each of the areas, the reference ruled line feature is compared with the ruled line information extracted by the ruled line extraction unit from the image of the form.
When the area of the reference ruled line feature includes the number of vertical ruled lines, there is a difference if the number of vertical ruled lines in the area of the image is less than the number of vertical ruled lines in the area of the reference ruled line feature. Judging that
When the area of the reference ruled line feature includes the number of horizontal ruled lines, there is a difference if the number of horizontal ruled lines in the area of the image is less than the number of horizontal ruled lines in the area of the reference ruled line feature. Judging that
When the area of the reference ruled line feature includes the length of the vertical ruled line, the length of the vertical ruled line in the area of the image is compared with the length of the vertical ruled line in the area of the reference ruled line feature. If it is shorter than the specified length, it is judged that there is a difference, and it is judged that there is a difference.
When the area of the reference ruled line feature includes the length of the horizontal ruled line, the length of the horizontal ruled line in the area of the image is compared with the length of the horizontal ruled line in the area of the reference ruled line feature. If it is shorter than the specified length, it is judged that there is a difference, and it is judged that there is a difference.
The determination unit
The image processing apparatus according to claim 3, wherein when the number of areas determined to be different is larger than a predetermined number, it is determined that there is an abnormality in the form. A program for operating a computer as the image processing device according to claim 1. It is an image processing method executed by an image processing device that processes an image of a form including ruled lines.
The image processing device includes a storage unit that stores the ruled line information as a reference ruled line feature.
The step of extracting ruled line information from the image and
A step of comparing the extracted information of the ruled line with the reference ruled line feature, and
An image processing method comprising: a step of determining that there is an abnormality in the form when there is a difference as a result of comparison. It is a form processing device that processes forms including ruled lines.
A storage unit that stores the ruled line information as a reference ruled line feature,
A transport control unit that controls the transport of the form input from the insertion slot inside the form processing device, and a transport control unit.
An image acquisition unit that acquires an image of the form, and
A ruled line extraction unit that extracts ruled line information from the image,
A ruled line comparison unit that compares the ruled line information extracted by the ruled line extraction unit with the reference ruled line feature.
When there is a difference as a result of comparison by the ruled line comparison unit, it is provided with a determination unit for determining that the form is abnormal.
The transfer control unit is a form processing device characterized in that when the determination unit determines that the form is abnormal, the form is returned to the insertion slot.

Images