JP6319974B2 - Form reader and program - Google Patents

Description

  Embodiments described herein relate generally to a form reading apparatus and a program for correcting distortion of a form image and reading the form characters from the image.

  For example, in a form reading apparatus such as a computer, a technique for recognizing a form image (digital photograph or the like) obtained by a digital camera or the like using character recognition application software (hereinafter referred to as “OCR software”) is known. Yes.

  Normally, a photo taken with a digital camera shows the form in a distorted manner, so in a conventional image processing device, character recognition is performed after correcting the image using the outline (edge) of the partial image of the form. Has been done.

  Recently, there is a demand to install the above-described OCR software function in a mobile terminal with a camera such as a smartphone or a tablet, and to use it for reading a special form such as a car verification of an automobile.

Japanese Patent Laid-Open No. 2002-74351

  Forms such as car verification are stored in a console box of a vehicle, and are often stored in a form folder such as a car verification box and folded in half.

  If the car verification stored in such a state is taken out from the car verification case and photographed with a mobile terminal, the car verification has a storage shape of the car verification case and has a concave and convex shape such as a double fold. Even if an image is taken, the printed image is distorted.

  In this case, since the vehicle verification outer shape in the image does not become linear (constant shape), there is a problem that the image cannot be corrected well by the OCR software.

  In addition, even if some correction can be made using the outline of the form, since the image including the entire form is necessary for the correction at this time, the characters described in the form in the image are shown small, In addition, since the line of the form is distorted and the color is light, sufficient recognition accuracy is often not obtained with the form outline standard.

  The problem to be solved by the present invention is to provide a form reading apparatus and a program capable of recognizing characters with high accuracy from partial images of a form taken in a bent state.

The form reading apparatus according to the embodiment includes an imaging unit, a touch panel, a horizontal line detection unit , an adjustment unit , an ordinate average value calculation unit, an image correction unit, and a character recognition unit. The imaging unit images a part of the form to be read . The touch panel displays an image of a part to be read of the form imaged by the imaging unit and allows a touch operation. The horizontal line detection unit detects one or more horizontal lines in which black pixels are continuous from an end of an image of a part to be read of the captured form. The adjustment unit displays a correction curve that is displayed on the touch panel in a line shape corresponding to each of the detected one or more horizontal lines, and the line shape can be adjusted by a touch operation . The ordinate average value calculation unit calculates an average value of the coordinates in the vertical direction of the correction curve so that the correction curve adjusted to match the shape of the line with the horizontal line by a touch operation becomes a straight line . The image correction unit corrects the image of the form by enlarging or reducing the image so that the horizontal line is arranged at the position of the average value of the vertical coordinates. The character recognition unit recognizes characters with respect to the image of the form corrected by the image correction unit.

It is a figure which shows the structure of the tablet terminal of one embodiment of a form reading apparatus. It is a figure which shows an example of vehicle verification. It is a flowchart which shows operation | movement of a tablet terminal. It is a figure which shows a mode that a part of vehicle verification was displayed on the touch panel of a tablet terminal. It is a figure which shows a mode that the auxiliary line corresponding to the detected horizontal line was displayed on the touch panel. It is a figure which shows a mode that an auxiliary line is dragged to the vertical direction and an up-and-down fluctuation is decreased. It is a figure which shows a mode that the separated auxiliary line is traced and connected in the horizontal direction. (A) is a figure showing the state where the adjusted auxiliary line was displayed. (B) is a figure which shows a mode that the line was typically drawn with the average value of the coordinate of the vertical direction of each auxiliary line. It is a figure which shows a mode that the image which cut out a part of vehicle verification image to the vertical direction is expanded or reduced. It is a figure which shows the image of the corrected vehicle verification.

  Hereinafter, embodiments will be described in detail with reference to the drawings. FIG. 1 is a diagram showing a configuration of a tablet terminal which is one embodiment of a form reading apparatus.

  As shown in FIG. 1, the tablet terminal 1 of this embodiment is, for example, a computer having a recording medium such as a CPU, a memory, and an SSD. In the tablet terminal 1, the CPU reads the control software installed in the recording medium onto the memory and executes processing of the software, whereby the tablet terminal 1 functions as a form reading device.

  The tablet terminal includes a camera 11 as an imaging unit, a memory 12, a horizontal line adjustment unit 13, an ordinate average value calculation unit 14, an image correction unit 15, a character recognition unit 16, and a touch panel 17 as a display & operation unit.

  The camera 11 is, for example, a CCD camera or the like, and as shown in FIG. 2, the vehicle verification A as a character-reading form is imaged, and the captured image (image data) (hereinafter referred to as “vehicle verification image 21” ( (See FIG. 4)) is stored in the memory 12. The vehicle verification image 21 is an image such as a monochrome image.

  That is, the camera 11 optically reads the vehicle verification A and stores the read vehicle verification image 21 in the memory 12. That is, as the imaging unit, in addition to the camera 11 built in the terminal, for example, a digital camera including an externally connected CCD sensor or the like may be used.

  A vehicle verification image 21 is stored in the memory 12. The memory 12 stores a dictionary for character recognition. The memory 12 functions as a work area when image processing is performed by each unit.

  The horizontal line adjustment unit 13 detects a horizontal line with continuous black pixels from the vehicle verification image 21. That is, the horizontal line adjustment unit 13 functions as a horizontal line detection unit that detects one or more horizontal lines from the captured vehicle verification image 21.

  The horizontal line adjustment unit 13 displays auxiliary lines 23a to 23r for line adjustment (see FIG. 5) at positions on the screen corresponding to the detected horizontal line. More specifically, the horizontal line adjusting unit 13 displays auxiliary lines 23a to 23r for adjusting the positions of the lines corresponding to one or more detected horizontal lines on the touch panel 17, and auxiliary lines 23a to 23r by touch operation. It functions as an auxiliary line adjustment unit that adjusts the auxiliary lines 23a to 23r with the adjustment operation.

The ordinate average value calculator 14 calculates the average value of the coordinates in the vertical direction for each detected pixel on the horizontal line and stores it in the memory 12.
The image correction unit 15 corrects (shapes) the image by enlarging (stretching) or reducing the image so that the horizontal black pixels are arranged at the position of the average value of the vertical coordinates.

  The character recognizing unit 16 recognizes characters with respect to the vehicle verification image 21 corrected (shaped) by the image correcting unit 15 to read horizontal characters written or printed in the interval of the horizontal line of the vehicle verification A and read the text data. obtain.

  The character recognition unit 16 reads out the vehicle verification image 21 in the memory 12 corrected by the image correction unit 15 and performs layout analysis on the vehicle verification image 21, so that the document structure (the position of the image in the vehicle verification A) Character or character string is described in

  The character recognition unit 16 cuts out the vehicle verification image 21 into character string units and character units in accordance with the document structure analyzed by the layout analysis, and refers to a character recognition dictionary set in the memory 12 in advance, so that the vehicle verification A Recognize characters.

  The memory 12 stores a character (text) as a result of character recognition by the character recognition unit 16 and a character image corresponding to the character.

  The touch panel 17 has two functions of information display and input. The vehicle verification image 21 captured by the camera 11 is displayed on the liquid crystal screen, and an image (picture or photograph) displayed on the liquid crystal screen by the operator. By touching the point or area with the hand, information on the touched screen position is sensed and output as operation information (signal). That is, the touch panel 17 displays a vehicle verification image 21 captured by the camera 11 and can be touched.

Hereinafter, the operation of the tablet terminal 1 of this embodiment will be described with reference to FIGS. 3 to 10.
In the tablet terminal 1, the vehicle verification A that has been taken out from the vehicle verification slot or the like is spread on a desk or the like. Then, a portion of the vehicle verification A shown in FIG. 2 on which the character recognition target character is written (a part of the surface necessary for character recognition) is photographed by the camera 11.

  In this case, the camera 11 of the tablet terminal 1 is directed to the vehicle verification A, and the shutter button 22 is pressed so that the character reading target portion of the vehicle verification A is within the screen of the touch panel 17 as shown in FIG. Thus, the camera 11 captures the vehicle verification A, and the vehicle verification image 21 is stored in the memory 12 (step S101 in FIG. 3).

  The horizontal line adjustment unit 13 reads the vehicle verification image 21 stored in the memory 12, detects a horizontal line in which black pixels continue from the edge of the image in the vehicle verification image 21 (step S102), as shown in FIG. The touch panel 17 displays the vehicle verification image 21 and auxiliary lines 23a to 23r corresponding to the horizontal lines detected in the vehicle verification image 21 (step S103).

  The auxiliary lines 23a to 23r are displayed in a distorted state at the position of the touch panel 17 corresponding to the detected horizontal line. Further, since the detection performance of the horizontal line is not perfect, the auxiliary lines 23a to 23r may be displayed with a slight shift from the actual horizontal line. For example, the auxiliary lines 23a to 23r may be separated (broken) and displayed for the thin horizontal line.

  Therefore, in this example, the auxiliary lines 23a to 23r are adjusted by a touch operation with the user's finger (step S104).

  In this case, the auxiliary lines 23a to 23r of the touch panel 17 are dragged with a finger, and the auxiliary lines 23a to 23r are moved to the correct ruled line positions of the photographed vehicle verification image 21. After adjusting the auxiliary lines 23a to 23r, the correction button 24 is clicked to move to the next process (image correction process).

  As shown in FIG. 6, when an auxiliary line among the auxiliary lines 23 a to 23 r displayed on the touch panel 17, for example, the auxiliary line 23 f is shifted from the horizontal line of the photographed vehicle verification image 21, the auxiliary line When 23f is touched with the finger 61, the horizontal line adjusting unit 13 activates the touched auxiliary line 23f.

  When the auxiliary line 23f is dragged in the direction of the arrow 62 as it is with the touched finger 61, the horizontal line adjustment unit 13 moves to the position of the correct ruled line (horizontal line) of the vehicle verification A in which the auxiliary line 23f is captured. When activated, for example, the display form changes from other lines such as blinking, and the correction button 24 is displayed at the bottom.

  Further, when it is desired to correct one auxiliary line in various directions, for example, the auxiliary line 23n is touched with a finger 63 for a certain period of time (1 second or more) like the auxiliary line 23n. 23n is activated, and several points 64 for correcting the Bezier curve are displayed above and below the auxiliary line 23n along the auxiliary line 23n.

  When the user moves the desired point 64 with the finger 63, the horizontal line adjusting unit 13 can bend a part of the auxiliary line 23n according to the amount of movement of the point 64 and correct the auxiliary line 23n in a desired direction. The line 23n can be aligned with the horizontal line of the vehicle verification image 21.

  If some of the auxiliary lines 23a to 23r are separated (disconnected), the auxiliary lines that are interrupted by the finger 65 (for example, the auxiliary line 23e and the auxiliary line 23f) are selected as shown in FIG. Then, the horizontal line adjustment unit 13 activates the two selected auxiliary lines 23e and 23f. Thereafter, the horizontal line adjusting unit 13 connects the auxiliary line 23e and the auxiliary line 23f by tracing the auxiliary lines 23e and 23f with the finger 65.

  When the above operation is performed for other desired auxiliary lines 23a to 23r and the auxiliary lines 23a to 23r are adjusted, and the correction button 24 is clicked, the auxiliary lines 23a to 23r are positioned at the horizontal lines of the vehicle verification image 21. Almost overlap, and the positions of the auxiliary lines 23a to 23r are determined.

  When the correction button 24 is clicked, the ordinate average value calculation unit 14 and the image correction unit 15 determine that there is an instruction to execute correction (Yes in step S105), and each auxiliary line 23a that substantially matches the horizontal line. In order to correct the vehicle verification image 21 so that ˜23r is a straight line (the position of the average value of the ordinates of each line), the following processing is performed.

  First, the ordinate average value calculation unit 14 checks whether or not there are discontinuous portions on the auxiliary lines 23a to 23r (whether or not there is a line break). As a result of the confirmation, if there are discontinuous portions (discontinuities are present) on the auxiliary lines 23a to 23r (Yes in step S106), the ordinate average value calculation unit 14 displays discontinuous error portions (step S107). Returning to the process of step S104, the corresponding auxiliary line is readjusted.

  On the other hand, if there is no interruption as a result of the above confirmation (No in step S106), the ordinate average value calculation unit 14 calculates the average value of the ordinate in each auxiliary line 23a to 23r (step S108).

  In this case, as shown in FIG. 8B, the ordinate average value calculation unit 14 performs the vertical axis as shown in FIG. 8B for each pixel of each horizontal line corresponding to the auxiliary lines 23a to 23r of the vehicle verification image 21 shown in FIG. Average values (coordinates) 23a-av to 23r-av of the direction coordinates are obtained. For thick horizontal lines, an average vertical position (coordinates) is obtained for each of a plurality of pixels corresponding to the width of the horizontal line. That is, the ordinate average value calculation unit 14 calculates the average values 23a-av to 23r-av of the vertical coordinates for each auxiliary line 23a to 23r and stores them in the memory 12.

Subsequently, the image correction unit 15 enlarges (stretches) or reduces the image in the vertical direction so that the black pixels on the horizontal line are arranged at the average coordinate values 23a-av to 23r-av in the vertical direction, and verifies the vehicle. The image 21 is corrected (shaped) (step S109) and stored in the memory 12.
Typically, as shown in FIG. 9, each line (black pixel) of the image 91 obtained by cutting out a part of the vehicle verification image A in the vertical direction is moved to the position of the average values 23a-av to 23r-av. To enlarge or reduce the image vertically.

  That is, the image correction unit 15 enlarges (stretches), reduces, or deletes pixels other than the black pixels of the vehicle verification image 21 by the adjustment amount (moving amount, combined amount, etc.) of each auxiliary line 23a to 23r, and the horizontal lines are continuous. The vehicle verification image 21 is deformed (corrected) so that the vehicle verification image 21 is almost close to the straight line. When two lines separated in the horizontal direction are combined, pixels other than black pixels between the lines are deleted.

  As a result, as shown in FIG. 10, in the corrected image 92, since the characters are neatly arranged (aligned) horizontally, it is easy to separate the characters from the ruled lines, and the accuracy of character recognition can be improved. However, the user's appearance is also easier to see.

  When the vehicle verification image 21 is corrected in this way, the character recognition unit 16 reads the corrected vehicle verification image 21 from the memory 12, performs layout analysis on the vehicle verification image 21, and based on the layout analysis result. Then, the image of the character part is cut out, character recognition is performed on the cut out image (step S110), and the character recognition result consisting of a pair of text (character code) and character image is stored in the memory 12. This operation is performed for images of all character parts obtained as a result of layout analysis.

Thus, according to this embodiment, the following effects can be obtained.
Conventionally, since the image is corrected using four edges, top, bottom, left, and right of the form image, there is a problem that correction is difficult when the edges are complicatedly distorted. Paying attention to the fact that there is a long ruled line on the side and the ruled line is drawn almost to the edge of the left and right edges, and using the ruled line (horizontal line) of the captured vehicle verification image 21, Recognize characters with correction only in the vertical direction.

  Further, in this embodiment, an image of a part of the vehicle verification A (see FIG. 4) is captured instead of the entire vehicle verification A, and characters are recognized within the range of the partial image, so that a character recognition result can be obtained in a short time. it can. In addition, since characters can be photographed at a higher resolution than when the entire vehicle verification A is photographed, the performance of character recognition can be improved.

  In this embodiment, the image is not corrected in the horizontal direction, but the characters of the car verification A are printed in horizontal writing, so the characters are aligned in a horizontal line. With the current character recognition technology, it is possible to detect and recognize a character from an image with higher accuracy when the characters are aligned in a horizontal line. Therefore, by deforming (correcting or shaping) an image according to this embodiment, it is possible to improve character detection accuracy and recognition accuracy.

  As mentioned above, although embodiment of this invention was described, this embodiment is shown as an example and is not intending limiting the range of invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  In the above embodiment, the vehicle verification image A is displayed on the touch panel 17, the auxiliary lines 23 a to 23 r are displayed so as to be superimposed on the vehicle verification image A, and the line is adjusted by the user's touch operation, and then the image is corrected. Although an example has been described, when line adjustment by a user's touch operation is not necessary, the horizontal line adjustment unit 13 may be a horizontal line detection unit and may be configured as follows.

  In this case, the tablet terminal 1 is detected by a camera 11 (imaging unit) that captures a form such as the vehicle verification image A, and a horizontal line detection unit that detects one or more horizontal lines from the captured vehicle verification image A. An ordinate average value calculation unit 14 that calculates an average value of one or more horizontal lines in the vertical direction, and vehicle verification by enlarging or reducing the image so that a horizontal line is arranged at the position of the average value of the vertical direction coordinates. An image correction unit 15 that corrects the image A, a character recognition unit 16 that recognizes characters with respect to the vehicle verification image A corrected by the image correction unit 15, and an output unit (such as a display unit) that outputs the result of character recognition With. Thereby, a series of operations after imaging the vehicle verification A can be automated.

  In the above embodiment, each component is realized by a program installed in a storage such as a hard disk device of a computer. However, the above program is stored in a computer-readable electronic medium: electronic media, and the program is stored in the electronic medium. The computer may realize the functions of the present invention by causing the computer to read the information. Examples of the electronic medium include a recording medium such as a CD-ROM and a DVD-ROM, a flash memory, a removable media, and the like. Further, the configuration may be realized by distributing and storing components in different computers connected via a network, and communicating between computers in which the components are functioning.

  DESCRIPTION OF SYMBOLS 1 ... Tablet terminal, 11 ... Camera, 12 ... Memory, 13 ... Horizontal line adjustment part, 14 ... Vertical coordinate average value calculation part, 15 ... Image correction part, 16 ... Character recognition part, 17 ... Touch panel, 23a-23r ... Auxiliary line .

Claims (4)

An imaging unit that images a part of the form to be read ;
A touch panel that displays an image of a part to be read of the form imaged by the imaging unit and is capable of a touch operation;
A horizontal line detection unit that detects one or more horizontal lines in which black pixels are continuous from an edge of an image of a part of the target to be read of the captured form ;
An adjustment unit that displays a correction curve that is displayed on the touch panel in the shape of a line corresponding to each of the detected one or more horizontal lines and that can adjust the shape of the line by a touch operation;
An ordinate average value calculating unit that calculates an average value of vertical coordinates of the correction curve so that the correction curve adjusted to match the shape of the line with the horizontal line by a touch operation becomes a straight line ;
An image correction unit that corrects the image of the form by enlarging or reducing the image so as to arrange the horizontal line at the position of the average value of the vertical coordinates;
A form reading apparatus comprising: a character recognition unit that recognizes characters with respect to the image of the form corrected by the image correction unit. The adjustment unit is
2. The form reading according to claim 1, wherein when a part of the correction curve is interrupted, the interrupted correction curves are connected by selecting the interrupted correction curves by a touch operation and tracing them together. apparatus. The Turkey computers that have a touch operation is possible touch displays images of a portion of the reading target portion of the document captured by the imaging unit and the imaging unit that captures an image of a part of the reading target portion of the form ,
A horizontal line detection unit that detects one or more horizontal lines in which black pixels are continuous from an end of an image of a part of the reading target portion of the form imaged by the imaging unit;
An adjustment unit that displays a correction curve that is displayed on the touch panel in the shape of a line corresponding to each of the detected one or more horizontal lines and that can adjust the shape of the line by a touch operation;
An ordinate average value calculating unit that calculates an average value of vertical coordinates of the correction curve so that the correction curve adjusted to match the shape of the line with the horizontal line by a touch operation becomes a straight line ;
An image correction unit that corrects the image of the form by enlarging or reducing the image so as to arrange the horizontal line at the position of the average value of the vertical coordinates;
A program for causing a character recognition unit to function as a character recognition unit for the form image corrected by the image correction unit. The adjustment unit is
4. The program according to claim 3, wherein when a part of the correction curve is interrupted, the interrupted correction curves are connected by selecting the interrupted correction curves by a touch operation and tracing them so as to connect each other .

Images