JP4420440B2 - Image processing apparatus, image processing method, character recognition apparatus, program, and recording medium - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, a character recognition device, a program, and a recording medium, and more particularly to cell division of a table area in image data and determination of a row direction in a cell.

Conventionally, a character recognition device that converts character information written in a document image into a character code that can be handled by a computer has been used.
When performing such character recognition, the document image is first divided into a character area, an image area such as a photograph or figure, or a table area, and another process is often performed on each area.
For the table area, it recognizes the position of the ruled lines that make up the table, makes the frame (cell) enclosed by the ruled lines one character area, and determines whether the character line in the cell is horizontal or vertical The character recognition is performed by switching the extraction method of the character lines in each cell according to the determination result.

  In the determination method of Patent Document 1, a frame surrounded by line segments in the main scanning direction and the sub-scanning direction is extracted from the table region of the document image, and the length in the main scanning direction and the length in the sub-scanning direction of each frame. Thus, it is determined whether the character line in each frame is horizontal writing or vertical writing, and character extraction is performed by switching the extraction method of the character line in each frame according to the determination result. As a result, even in a table in which a horizontal writing frame and a vertical writing frame are mixed, the character lines in each frame are accurately extracted, and there is an effect that the accuracy of character extraction and character recognition in each frame is improved.

Further, the determination method of Patent Document 2 extracts a histogram by projecting a circumscribed rectangle of connected black pixels of all cells surrounded by a ruled line from the input table image in two vertical and horizontal directions. The number of lines, the number of horizontal lines, the vertical and horizontal line pitches, etc. are obtained (see FIG. 7), and it is determined from the information whether the characters in the cell are vertical writing or horizontal writing.
Japanese Patent No. 2931041 Japanese Patent Laid-Open No. 08-16715 JP 09-167246 A

  However, in the determination method of Patent Document 1, the direction of the character string is determined based on the length of the frame in the main scanning direction and the length of the sub scanning direction. As a result, there is a problem in that the direction of the character string is erroneously determined when the direction in which the length of the frame is long and the direction of the character string are different. For example, when there is a horizontally written character line in a vertically long frame, the direction of the character string is erroneously determined as vertically written.

In order to solve this, the method of Patent Document 3 determines the attribute of the image in the frame, determines the presence or absence of a horizontal ruled line, and more accurately determines the direction of the character string in the frame constituting the table. I did it. The attribute of the image in the frame is that there are 3 or more lines in the frame, or the height a of the character image in the frame and the character spacing b in the vertical direction of the character image have a relationship of b> 3a It is determined that the table image has no horizontal ruled lines. Otherwise, it is determined as a table image with a horizontal ruled line.
With this determination, even when there is a horizontally written character line in a vertically long frame, the direction of the character string is not erroneously determined as vertically written.

  Here, if the table of a color document or grayscale document has cells separated by color or shading, color information and shading information will be lost if binarization is performed for table area and character recognition. Adjacent upper and lower or left and right cells are combined to form a vertically or horizontally long cell. In the case of such a cell, the method of Patent Document 3 cannot correctly determine. For example, if the content of one cell separated by color is vertical writing and the cell below it is horizontal writing, the content of a cell in which these adjacent cells are combined can be determined as vertical writing or horizontal writing. Can not.

  Further, in the method using the histogram in the main scanning direction and the sub-scanning direction as in Patent Document 2, in the case of a Chinese character that is often made up of “deviation and 旁” such as Japanese, cell If the characters inside are all Kanji characters consisting of “bias and 旁”, the width cannot be obtained correctly, and it cannot be correctly determined whether the writing is vertical or horizontal.

  The present invention has been made in consideration of the above circumstances, and even when the ruled line of the table is drawn by color or shading, the cell delimiter can be correctly determined. An object of the present invention is to provide an image processing device, an image processing method, a character recognition device, a program, and a recording medium that can correctly estimate the direction of the character line.

In order to solve the above-mentioned problem, the invention according to claim 1, an area identifying means for identifying a binarized image of a document image and extracting a table area image, and dividing the table area image into cell images In the image processing apparatus having cell dividing means, when the document image is a color image or a grayscale image, an edge having a predetermined length or more is extracted from the document image corresponding to the table region image, and the table region image And a ruled line setting means for generating a new table area image by superimposing the edges as ruled lines, the cell dividing means dividing the table area image generated by the overlapping into cell images, and performing the image processing The apparatus extracts a circumscribed rectangle of a connected component of black pixels in the cell image divided by the cell dividing means, and determines whether the standard character size obtained from the circumscribed rectangle is a predetermined range value or not. Inside It is determined whether the number of rows is one or two, and a cell is determined from the number of rows, the vertical or horizontal length and aspect ratio of the cell image, and the position in the table of the cell image. It is characterized by having a line direction estimating means for estimating the direction of the character line .

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, the row direction estimation unit determines a row direction when the estimated row direction is different from a row direction set as a default value. The estimated line direction is reset .

The invention according to claim 3 is an image processing method for identifying a binarized image of a document image, extracting a table region image, and dividing the table region image into cell images, wherein the document image is a color image or a gray image. In the case of a scale image, an edge having a predetermined length or more is extracted from the document image corresponding to the table area image, and a new table area image is generated by superimposing the edge as a ruled line on the table area image. A table region image generated by superimposing is divided into cell images, a circumscribed rectangle of a connected component of black pixels in the divided cell image is extracted, and a standard character size obtained from the circumscribed rectangle is predetermined. It is determined whether or not the number of rows in a cell is one or two rows depending on whether or not it is a range value, the number of rows, the vertical or horizontal length of the cell image, and the aspect ratio, Cell from the position in the table of the cell image Characterized by being adapted to estimate the character row direction.

According to a fourth aspect of the present invention, there is provided a document image input unit for inputting a document image, a region identification unit for identifying a binarized image of the document image and extracting a table region image, and the table region image as a cell image. In a character recognition device having cell dividing means for dividing the cell image and character recognition means for recognizing the cell image, if the original image is a color image or a grayscale image, the original image corresponding to the table area image is used. An edge having a predetermined length or more is extracted, and a ruled line setting unit that generates a new table area image by superimposing the edge as a ruled line on the table area image, and the cell dividing unit generates the overlapping image is obtained by dividing the table region image to the cell image, the character recognition device, the extracting circumscribed rectangles of connected components of black pixels of the cell division means divided in the cell image, a standard sentence obtained from the outer contact rectangle It is determined whether the number of rows in the cell is one or two rows depending on whether the size is a predetermined range value, and the number of rows and the vertical or horizontal length of the cell image, And a line direction estimating means for estimating the character line direction in the cell from the aspect ratio and the position in the table of the cell image .

The invention according to claim 5 is a program for causing a computer to execute the function of the image processing apparatus according to claim 1 or 2 or the function of the character recognition apparatus according to claim 4.
The invention described in claim 6 is a computer-readable recording medium in which the program according to claim 5 is recorded.

According to the present invention, even when a ruled line of a table is drawn by color or shading in a color document or a gray scale document, cell separation can be correctly determined.
In addition, even in the case of a binarized image, or in the case where the cell division by color or shading has failed, the direction of the character line in the cell can be estimated using the information in the cell. As a result, it is possible to prevent failure of line extraction in character recognition processing.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic block diagram of a character recognition apparatus to which an image processing apparatus of the present invention is applied. The character recognition device shown here connects an image input device such as a scanner device and a display device such as a liquid crystal display, and recognizes characters in the image of the character region and the surface region read from the image input device to recognize character codes. And the recognition result can be used by other application software. At the time of the conversion, a display device is used for work support during recognition.

In FIG. 1, the character recognition device includes an input unit 10, a binarization unit 20, a preprocessing unit 30, a region identification unit 40, a character region processing unit 50, a table region processing unit 60, and an output unit 70.
The input unit 10 inputs a color document or a gray scale document from a color image input device (for example, a digital color copying machine, a scanner, a camera, etc.), performs A / D conversion processing, and stores it in the original image storage unit 15. Alternatively, image data may be received from application software of a digital camera or a document image analysis / recognition system, or may be input from a file that already stores image data.

The binarization unit 20 binarizes the color image data and the gray scale image data stored in the original image storage unit 15 and stores them in the binarized image storage unit 25.
The preprocessing unit 30 updates the binarized image storage unit 25 by performing skew correction and noise removal of the binarized image stored in the binarized image storage unit 25.

  The area identifying unit 40 identifies the binarized image stored in the binarized image storage unit 25 as a character area, a table area, a photo area, and the like, and the image divided for each area is positional information in the original image, The area information is stored in the area division result storage unit 45 in association with the size information and area classification (character area, table area, photo area, etc.). Further, the line direction of the original is also determined, and the determination result is stored.

  The character region processing unit 50 calculates the feature amount of the character cut out from the character region image among the divided regions stored in the region division result storage unit 45, and the feature amount and the feature amount of the standard pattern of the character recognition dictionary A candidate having a high similarity as a candidate for a recognition result, a candidate character code, its similarity, the character type of the candidate character, and the like as a recognition result, and associating the recognition result with a divided region, a region division result storage unit 45 is updated. The character area processing unit 50 may be called from the table area processing unit 60.

  The table area processing unit 60 divides the table area into cells among the divided areas stored in the area division result storage unit 45, identifies whether the area in the cell is a character area, a photo area, or the like, and The position information, size information, and cell classification (character area, photo area, etc.), and if these cells are character areas, the area recognition result storage unit 45 is updated with the character recognition result corresponding to the table area.

  The output unit 70 compares the recognition result data obtained in the region division result storage unit 45 with the original image in the original image storage unit 15 and displays it on the display device, and allows the user to select or correct a desired result. The final recognition result is output to an output device such as a printer or a storage device, or delivered to other application software.

  The overall configuration and basic processing contents of such a character recognition apparatus are the same as those of the conventional character recognition apparatus, but the table area processing unit 60 is based on the image processing apparatus of the present invention. Is different from the conventional one.

  In this embodiment, the image processing apparatus of the present invention is applied to the character recognition apparatus. However, for example, image identification, image editing, document filing, document processing, document management, The present invention can be easily applied to apparatuses and systems that perform document distribution and the like.

Next, the table area processing unit 60 (the image processing apparatus of the present invention) will be described in detail.
FIG. 2 is a block diagram showing a functional configuration of the table area processing unit 60. In the figure, the table area processing unit 60 includes a ruled line setting unit 61, a cell dividing unit 62, and a row direction estimating unit 63. ing.

The ruled line setting unit 61 obtains the region edge strength of the color original image or the grayscale original image corresponding to the table region stored in the region division result storage unit 45, and the edge strength is determined from a predetermined value. If it is strong (= there is an edge) and it is continuous (hereinafter referred to as a virtual ruled line), it is determined that there is a virtual ruled line, and a ruled line is added to the binarized image of the table area by OR processing, thereby dividing the area The result storage unit 45 is updated.
At this time, it is determined that cuts of about several pixels are continuous, and a case where the cut is continued for a predetermined length or more is determined as a ruled line. Therefore, the image other than the table area that is not extracted as the table area by the area identification process is not affected.

  Virtual ruled line processing is performed in color and grayscale original tables, in which cells are not separated by actual ruled lines, but only by differences in color and shading. Extraction may be difficult. In addition, when the ruled line of a color document or gray scale document (multi-valued document) is a thin actual ruled line, the ruled line disappears in the binarization process, and it is difficult to extract cells.

FIG. 3 is a flowchart showing a processing flow of the ruled line setting unit 61.
First, virtual ruled lines and ruled lines in the main scanning direction are set as follows.
A 2 × 3 pixel window (FIG. 4 (1)) is provided in the main scanning direction for all the pixels in the front region, and the pixel density in the sub scanning direction in the window is shifted while moving the window in the main scanning direction. Pixel continuations in which the difference exceeds a predetermined threshold are extracted (FIGS. 5A and 5B). When the length of the pixel continuation continues for a predetermined length or more, it is determined that the pixel continuation is a ruled line or a virtual ruled line (FIG. 5B) (step S01).
Here, if the pixel continuity with no density difference is equal to or smaller than the width of one window, it is determined that the pixels are continuous.

A ruled line or a virtual ruled line is written by OR processing at a position corresponding to the extracted pixel continuation in the binarized table area image data (step S02).
The above steps S01 and S02 are repeated until all the pixels of the image in the table area are processed (step S03).

  Next, a 3 × 2 window (FIG. 4 (2)) is provided in the sub-scanning direction, and the processing in steps S01 to S03 is repeated while shifting this window in the sub-scanning direction, so that ruled lines or virtual ruled lines in the sub-scanning direction are obtained. Set.

  Or, instead of the processing procedure as described above, for example, to reduce the processing time, to deal with a blurred image, or to deal with a dither image (color), for example, a 200 dpi color image is stored as a thin line reduced to 100 dpi It is also possible to create an image, binarize the thin line storage reduced image, extract virtual ruled lines, and extract cells.

  For example, when a 2 × 2 pixel is reduced to one pixel, the thin line storage reduced image is created by setting the darkest value in the 2 × 2 pixels as a representative color of the reduced image to be created. Even in the case of a manuscript written in black, a reduced image can be created without the thin lines disappearing (the thin lines may disappear if the thinning is simply performed).

  The cell division unit 62 divides the table area (binarized image with virtual ruled lines added) stored in the area division result storage unit 45 into cells, and the area in the cell is a character area or a photo area. The region division result storage unit 45 is updated by associating the cell position information, the size information, and the cell classification (such as the character region or the photo region) with the table region.

The line direction estimation unit 63 divides the divided cell image into a character area or a photograph area, and estimates the direction of the character line if it is a character area. The table area processing unit 60 performs character recognition according to the direction of the character line.
First, the row direction of the cells in the table area is automatically set as the row direction of the document specified by the user (if the user designates the row direction of the document as horizontal), or the row direction is automatically set. When the mode is set, the row direction obtained by the area identifying unit 40 is set as a default value.

  Next, when the predetermined condition (A) or (B) as shown below is satisfied, the row direction is set to “vertical”, and when different from the default row direction, the row direction is changed. Reset (replace).

(A) When all the following conditions (1) to (4) are satisfied, the row direction “vertical” is set.
(1) A vertically long cell having an aspect ratio larger than a predetermined value.
(2) The number of rows in the cell is one or two rows.
(3) The vertical length of the cell is shorter than ½ of the vertical size of the table.
(4) A cell is included in the left half of the table area (FIG. 6).

(B) When all of the following conditions (5) to (7) are satisfied, the row direction “vertical” is set.
(5) A vertically long cell having an aspect ratio larger than a predetermined ratio.
(6) The number of rows in the cell is one.
(7) The vertical length of the cell is longer than ½ of the vertical size of the table.

  Next, when the predetermined condition (C) or (D) as shown below is satisfied, the row direction is set to “horizontal”, and when different from the default row direction, the row direction is set to Reset (replace).

(C) When all the following conditions (11) to (14) are satisfied, the row direction “horizontal” is set.
(11) A horizontally long cell having an aspect ratio larger than a predetermined value.
(12) The number of rows in the cell is 1 row or 2 rows.
(13) The horizontal length of the cell is shorter than 1/2 of the horizontal size of the table.
(14) Cells are included in the upper half of the table area (FIG. 6).

(D) When all of the following conditions (15) to (17) are satisfied, the row direction “horizontal” is set.
(15) A horizontally long cell having an aspect ratio larger than a predetermined value.
(16) The number of rows in the cell is one.
(17) The horizontal length of the cell is longer than ½ of the horizontal size of the table.

  When none of the above conditions (A) to (D) is satisfied, the row direction is left as the default, and the row direction is not reset.

Here, in order to calculate the number of rows in the cell, a circumscribed rectangle of the black pixel connected component in the cell is obtained. Next, three standard sizes (the maximum value of the size of the square rectangle in the cell, the maximum value of the rectangular width of the horizontally long rectangle in the cell, and the maximum value of the height of the vertically long rectangle in the cell) are obtained from the circumscribed rectangle.
Here, ruled lines that divide the table area into cells may exist in the cells due to curling, etc., and in order to eliminate the adverse effects caused by this, when calculating the maximum value of a horizontal rectangle or vertical rectangle, the value The maximum value is obtained only for a rectangle whose value is smaller than a predetermined value.

  By predetermining three standard size range values that can be determined to be one or two rows in a cell, and examining whether the determined three standard sizes are predetermined range values, The number of rows in the cell is determined to be 1 row or 2 rows.

By configuring the present embodiment as described above, it is possible to correctly determine the cell delimiter even when the ruled line of the table is drawn by color or shading in a color document or a gray scale document.
In addition, even in the case of a binarized image, or in the case where the cell division by color or shading has failed, the direction of the character line in the cell can be estimated using the information in the cell. As a result, it is possible to prevent failure of line extraction in character recognition processing.

The present invention is not limited only to the above-described embodiments. The functions of the image processing apparatus and the character recognition apparatus according to the above-described embodiments are programmed, written in advance on a recording medium such as a CD-ROM, and then driven by a medium such as a CD-ROM drive mounted on the image processing apparatus and the character recognition apparatus. It goes without saying that the object of the present invention can be achieved by mounting the CD-ROM or the like in the apparatus, storing these programs in a memory or storage device of a computer, and executing them.
In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and the program and the recording medium recording the program also constitute the present invention.

  As a recording medium for storing the program, a semiconductor medium (for example, ROM, nonvolatile memory card, etc.), an optical medium (for example, DVD, MO, MD, CD, etc.), a magnetic medium (for example, magnetic tape, flexible disk, etc.) ) Or the like.

  Further, not only the functions of the above-described embodiment are realized by executing the loaded program, but also the above-described implementation by cooperating with the operating system or other application programs based on the instructions of the program. The case where the function of the form is realized is also included.

When distributing the program to the market, store the program in a portable recording medium for distribution, or store the program in a storage device of a server computer and connect it to another computer via the Internet. It can also be distributed by transferring to. In this case, the storage device of this server computer is also included in the recording medium of the present invention.
In the computer, the functions of the above-described embodiments are realized by installing a program on a portable recording medium or a transferred program and executing the installed program.

  The above-described computer can be applied as a single computer, but it can also be applied as a plurality of computer networks. A program used in the computer is recorded in another computer, and the remote computer is used. May be processed or executed in a distributed manner.

  As described above, by programming and distributing the functions of the above-described embodiment, cost, portability, and versatility can be improved.

It is a schematic block diagram of the character recognition apparatus to which the image processing apparatus of this invention is applied. It is a block diagram which shows the function structure of a table area process part. It is a flowchart which shows the flow of a process of a ruled line setting part. It is an example of the window in the main scanning direction and subscanning direction for extracting a virtual ruled line. It is an example of determination of a virtual ruled line. It is a figure explaining the position of the cell in a table | surface. It is a figure explaining the conventional method of determining whether the cell of a table | surface is vertical writing or horizontal writing.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Input part, 15 ... Original image storage part, 20 ... Binarization part, 25 ... Binary image storage part, 30 ... Pre-processing part, 40 ... Area identification part, 45 ... Area division result storage part, 50 ... Character area processing unit, 60 ... Table area processing unit, 61 ... Ruled line setting unit, 62 ... Cell division unit, 63 ... Row direction estimation unit, 70 ... Output unit.

Claims (6)

In an image processing apparatus having area identifying means for identifying a binarized image of a document image and extracting a table area image, and cell dividing means for dividing the table area image into cell images,
When the document image is a color image or a grayscale image, an edge having a predetermined length or more is extracted from the document image corresponding to the table area image, and the edge is superimposed on the table area image as a ruled line to create a new image. Having ruled line setting means for generating a table area image;
The cell dividing means divides the table region image generated by the superposition into cell images ,
The image processing apparatus extracts a circumscribed rectangle of a connected component of black pixels in the cell image divided by the cell dividing means, and determines whether the standard character size obtained from the circumscribed rectangle is a predetermined range value. To determine whether the number of rows in the cell is one or two, and in the table of the cell image, the number of rows, the vertical or horizontal length of the cell image, and the aspect ratio. An image processing apparatus comprising line direction estimation means for estimating a character line direction in a cell from the position of the character . 2. The image processing apparatus according to claim 1, wherein the row direction estimation unit resets the row direction to the estimated row direction when the estimated row direction is different from a row direction set as a default value. An image processing apparatus. In an image processing method for identifying a binarized image of a document image, extracting a table region image, and dividing the table region image into cell images,
When the document image is a color image or a grayscale image, an edge having a predetermined length or more is extracted from the document image corresponding to the table area image, and the edge is superimposed on the table area image as a ruled line to create a new image. A table area image is generated, the table area image generated by superimposing is divided into cell images, a circumscribed rectangle of a connected component of black pixels in the divided cell image is extracted, and obtained from the circumscribed rectangle It is determined whether or not the number of rows in a cell is one or two rows depending on whether or not the standard character size is a predetermined range value, and the length of the cell image and the length or width of the cell image An image processing method characterized in that the direction of a character line in a cell is estimated from the aspect ratio and the position in the table of the cell image . Document image input means for inputting a document image, area identification means for identifying a binarized image of the document image and extracting a table area image, cell dividing means for dividing the table area image into cell images, In a character recognition device having character recognition means for recognizing a cell image,
When the document image is a color image or a grayscale image, an edge having a predetermined length or more is extracted from the document image corresponding to the table area image, and the edge is superimposed on the table area image as a ruled line to create a new image. Having ruled line setting means for generating a table area image;
The cell dividing means divides the table region image generated by the superposition into cell images ,
The character recognition device extracts a circumscribed rectangle of a connected component of black pixels in the cell image divided by the cell dividing means, and whether a standard character size obtained from the circumscribed rectangle is a predetermined range value or not. To determine whether the number of rows in the cell is one or two, and in the table of the cell image, the number of rows, the vertical or horizontal length of the cell image, and the aspect ratio. A character recognition device comprising line direction estimation means for estimating a character line direction in a cell from the position of the character .   A program for causing a computer to execute the function of the image processing apparatus according to claim 1 or 2 or the function of the character recognition apparatus according to claim 4.   A computer-readable recording medium on which the program according to claim 5 is recorded.