JP2701350B2 - Document reading device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial application field) The present invention relates to a document reading apparatus that extracts an arbitrary character reading area in a document image of a book or the like and reads the character.
In particular, the present invention relates to a document reading device that reduces the burden on a user.

(Prior Art) It is important to automatically read a desired reading area in an existing document image such as a general book using a character recognition device in order to realize efficient storage and transmission of the existing document image. . When such an existing document is read, there is a problem that an area to be read is not uniquely determined depending on a use situation like a whole document or a part of a document such as an abstract. Further, the existing document image may include columns, figures, and tables.

Conventionally, the most common method for determining an area to be read from such an existing document image is a first method in which a reading area is determined as a rectangular area by designating two points on the document image.

Further, as shown in, for example, Japanese Patent Application No. 61-281377, "Image understanding method", a second method for obtaining an area to extract a document image in accordance with a grammar defined as a set of a plurality of rectangular areas is known. Are known.

On the other hand, as a method of dividing a general document image including a two-column form, a figure, a table, and the like into basic elements such as character lines, figures, and tables, and automatically extracting a desired area, for example, "Structural Analysis of Page Images Based on Split Detection Method" (IEICE Technical Report, Pattern Recognition and Learning PRL85-17, 1985-
A third method described in a technical paper (6, 63 pages to 70 pages) is known.

A fourth method of dividing a document image into basic elements such as character lines, figures, and tables, and then structuring the elements constituting the document image and the arrangement relation between the elements into a tree structure in a hierarchical manner,
For example, it is described and known in Japanese Patent Application No. 62-172199 “Document Image Analysis System” by the same person as the present inventor.

(Problem to be Solved by the Invention) In the first method described in the section of "Prior Art", it is necessary to designate at least two points for expressing a desired reading area as a rectangular area while always looking at a document image. In addition, there is a disadvantage that the burden on the user is increased in consideration of the case where the image display accuracy on the display screen is deteriorated. Further, in the case of a two-column document image or the like, it is necessary to specify the document image several times.

In the second method described in the section of the above-mentioned "Prior Art", defining the position and size of the rectangular area based on absolute or relative coordinates requires a lot of effort. In this case, the load on the user is further increased due to a change in the number of lines or the mixture of figures and the like.

In the third method by the same person as the inventor of the present application described in the section of the above "prior art", while a desired area can be automatically extracted from an existing document image, a reading area is fixedly determined. There was a drawback that the use form was fixed.

Further, in the fourth method by the same person as the inventor of the present invention described in the section of the "prior art", a method of structuring the elements constituting the document image and the arrangement relation between the elements as a tree structure is described. However, it does not specifically describe a method of determining a reading area selected by a user and reading characters.

As described above, the conventional method of determining the area to be read from the document image has a problem to be solved in any of the above-described four methods.

Therefore, an object of the present invention is to solve the above-described conventional problems by structuring the elements constituting a document from an input image and the arrangement relationship between the elements as a tree structure, and then using the tree structure to form a document image. By extracting the layout structure and specifying one point on the element area where the layout is displayed,
It is an object of the present invention to provide a document reading device that reduces the burden on a user.

Another object of the present invention is to allow a desired reading area required at various levels from an existing document image to be designated at one point and in a state where the designated position is relaxed, so that the burden on the user can be easily reduced. It is an object of the present invention to provide a document reading device which is reduced.

(Means for Solving the Problems) In order to solve the above-mentioned problems, a document reading apparatus provided by the present invention comprises: a region dividing means for decomposing a document image into basic element blocks such as character lines and figures; Document structuring means for sequentially structuring from the basic element blocks of the above, and structuring the element blocks constituting the document image and the arrangement relationship between the element blocks as a tree structure hierarchically expressed, and a predetermined level or re-expression Layout extraction means for extracting and displaying the layout structure of the element block selected by the area selection means using the tree structure, and detailed layout information should be expressed from one or a plurality of the layout-displayed element blocks. A re-display area selecting means for selecting an element block, and specifying a reading area at one point from one or a plurality of element blocks displayed in a layout. Area determining means for determining, and line reading means for sequentially reading only the character lines in the reading area in the vertical search order of the tree structure,
A character recognition unit that cuts out the read character line in units of one character, collates the character line with a recognition dictionary, and sequentially outputs character read results.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings.

FIG. 1 is an example shown for explaining a method of dividing a horizontally written document image into basic elements and then structuring the elements into elements constituting the document. Also, FIG.
FIG. 11 is an example showing a result of generating a tree structure of elements constituting a document image and an arrangement structure between the elements by dividing and structuring basic elements from the document image shown in FIG.

In FIG. 1, hatched circles indicate characters. A document image is used as a basic element (in the figure, the symbol Si (i = 1, 2, ... 2)
For example, the method of dividing into a character line block indicated by 0) is described in “Structural analysis of page image based on split detection method” by the same person as the above-mentioned invention of the present application (IEEE Technical Report PRL85-17 , 1985-6). It is needless to say that, by using the above-mentioned conventional technology and the like, for example, even if figures and the like are mixed or vertical writing is performed, it can be divided into basic elements. Next, structuring is performed sequentially from the basic elements,
Processing for generating the elements constituting the document image and the arrangement structure between the elements as a tree structure will be described.

First, a sentence block is an important element constituting a document image. Here, if a sentence block is defined as a set of character lines in which character lines having the same character arrangement direction are arranged at a predetermined pitch or less, the sentence block described below has a structure in paragraph units in a normal document. It may be regarded as a transformed area.

For example, the character line of FIG. 1 S _3, S _4, S ₅ is text block T ₂
Will be structured according to the above definition. In the following description, a combination area of basic element blocks such as a text block, a character line, a figure, a table, and a photograph will be referred to as a virtual block. For example, a character row block S ₆ and combining region M ₃ sentences blocks T ₃ of FIG. 1 is a virtual block. next,
As the arrangement relationship between the element blocks that constitute the document image,
Introduce a vertical relationship, a horizontal relationship, and an inclusion relationship. For example, in the figure, the character line blocks S ₁ and S ₂ are in a vertical relationship,
Virtual block M ₃ and the sentence block T ₄ is on the left and right relationship. Further, the containment relationship between the virtual block M ₂ and the virtual block M _3, which means 2-column region.

By structuring the area including the arrangement relationship described above, a tree structure as shown in FIG. 2 can be generated for the document image shown in FIG. In FIG. 2, the nodes indicated by circles in the figure indicate the respective element blocks, and the symbols in the nodes respectively indicate the element blocks in FIG. 1 (where the symbol P is a one-page area). Also, the arrow in the figure ↓
→ refers to an arrangement relationship of a vertical relationship and a horizontal relationship, respectively. For example, it can be easily understood from FIG. 2 that one page region P includes two virtual blocks in a left-right relationship. In addition, position, size,
Assume that it has an element name, a pointer indicating the arrangement relationship, and the like.

Now, the tree structure of Figure 2 starting from the first page region P performs normal vertical search, considering the case where sequential retrieving a character row block Si (i = 1 ... 20) , the first character line S ₁ found , then find the character line S _2, it will be the last to find a character line S _3. That is, when the vertical relationship is satisfied, the character lines can be sequentially read from top to bottom. When the horizontal relationship is satisfied, the character lines can be sequentially read from left to right in the example of horizontal writing in FIG. Character strings can be detected in the order in which sentences should be read.

The structuring method including the arrangement relation of each element of the document image as shown in FIG. 2 is described in, for example, Japanese Patent Application No. 62-172199 “Document Image Analysis Method” by the same person as the present application. This can be realized by using the described method.

FIG. 3 is an example in which the tree structure in FIG. 2 is searched vertically, text blocks or basic element blocks are extracted, and the layout is displayed according to their position and size information. Therefore, a method for specifying an area of the document reading apparatus according to the first invention of the present application will be described with reference to FIG.

In the layout display shown in FIG. 3, each element block may be identified for each element name using color information, a graphic pattern, or the like. Further, for example, a reduced document image can be easily displayed in association with the layout display of FIG. Symbols a, b, c, and d indicated by arrows in the figure each indicate an example of a user-specified position on the display screen. In addition, as the pointing device of the display screen,
A known device such as a mouse can be used, but is not limited thereto.

Now, the arbitrary point in the text block T ₂ as shown by the arrow a is designated as a reading area, a character row block S ₃ included in the text block T ₂ shown in FIG. 2, S ₄ , S ₅
Are sequentially read, and then one character in each character line block is sequentially cut out and recognized. Thus, each character image in the text block T ₂ is converted into a character code string as a sentence. Note that a conventionally known technique can be used for the above-described character extraction and character recognition.

Similarly, by specifying the arbitrary point in the document block T ₃ as shown by an arrow b as the reading area, it can be easily converted to each character image in the text block T ₃ sequentially to the character code string .

Next, the virtual block M ₂ representing the two-column set shown in FIG.
The case where the whole sentence, that is, the character line Sk (k = 6... 20) is converted into a character code string by one designation will be described. In the case of a display level element blocks shown in FIG. 3, for example, specifying the position of the arrow c as read area, location of the arrow c is included in the virtual block M ₂ of FIG. 2, the back under for the virtual block M ₃ and not included in the text block T _4, virtual block M ₂ by the designation of the arrow c
Can be determined. That is, the tree structure shown in FIG. 2 is searched, and the tree structure can be obtained as an element block including the specified position detected last.

Similarly, when the position of arrow d is designated, the entire document image, that is, all character lines in the document image are converted into character code strings in the order of vertical search in the tree structure in FIG.

As described above, according to the first aspect of the present application, the user can be burdened by the fact that the designation position is considerably reduced from the designation of two points required for obtaining a rectangular area to the designation of one point and the designated position is considerably reduced. It can be significantly reduced. However, in the case of the designation of the arrow c shown in FIG. 3, the designated position is slightly restricted as compared with the case of the arrow b or the like.
Needless to say, the reading area designation by one point may be performed several times.

FIG. 4 is an example shown for explaining the area specifying method of the document reading apparatus of the second invention of the present application. In the first invention of the present application, a method of designating a reading area by one point of an element block that is not displayed based on the position of each displayed element block, such as a reading area specification indicated by an arrow c in FIG. Offered. On the other hand, the second invention of the present application provides a technique for sequentially changing the display level and displaying the area, thereby specifying an area by one point of a desired element block. As a result, it is necessary to perform display several times by specifying redisplay, but the restriction on the designated position of one point is considerably relaxed.

FIG. 4 (a) shows a layout display of one page area P in FIG. 2, and when pointing is performed at the position of arrow e as redisplay designation, as shown in FIG. 4 (b). FIG. 2 shows a virtual block under one page area P in FIG.
M ₁ and M ₂ are displayed. On the other hand, when the reading area is designated at the position of arrow e as described in FIG. 3, each character line in the document is sequentially converted into a character code string. Similarly, the fourth
In FIG. (B), when performing pointing as redisplayed specified at the location of arrow f, indicating virtual block M virtual block M ₃ in the back of a ₂ and text block T ₄ in the second diagram in Figure 4 (c) Is displayed as follows.

In the display shown in FIG. 4, the display magnification may be changed and displayed. Further, similarly to FIG. 3, a reduced document image may be displayed in association with the layout display.

FIG. 5 is a block diagram showing one embodiment of the present invention.

In the figure, reference numeral 1 denotes an image memory for storing a document image as quantized image information. Reference numeral 2 denotes an area dividing unit. The area dividing unit 2 has a function of dividing a document image stored in the image memory 1 into basic element blocks such as character lines, figures, and tables, and stores the result in the structured data storage unit 4. The document structuring unit 3 sequentially reads and updates the contents of the structured data storage unit 4 to determine the element blocks constituting the document image and the arrangement relationship between the element blocks as described in FIG. It is generated as a structure and stored in the structured data storage unit 4. Layout extractor 5
Searches the tree structure stored in the structured data storage unit 4 in accordance with the inside of a redisplay selection unit 15 described later, and transfers each element block information to be displayed to the display unit 12. The display unit 12 displays, based on the transferred element block information,
As shown in FIG. 3, layout display is performed on a display screen (omitted in the figure). It is assumed that the display unit 12 also has a function of reading a document image from the image memory 1, performing image reduction, and displaying the document image on a screen in association with a layout display. Next, as described in FIG. 3 with reference to the layout display on the screen, when the user specifies a reading area using a pointing device such as a mouse, the pointing input position information is re-input from the area specifying unit 7. Display selection section
Transferred to 15. The area determination unit 8 searches the tree structure stored in the structured data storage unit 4 for an element block to be read in accordance with the pointing input position information, and transfers the row to the row reading unit 9. The line reading unit 9 vertically searches the tree structure stored in the structured data storage unit 4 only for character lines included in the element block to be read, sequentially reads out the tree structure, and sequentially transfers the tree structure to the character cutout unit 10. . The character cutout unit 10 sequentially cuts out an image in units of one character from a document image stored in the image memory 1 according to the sequentially transferred character block information, and transfers the image to the recognition unit 11. The recognition unit 11 checks the recognition dictionary 13 with the sequentially input character images, converts them into character codes, and sequentially stores them in the recognition result storage unit 14.

In this embodiment, as described above, in order to specify the area of a desired element block while sequentially changing the display level, for example, as described in the description of the area specification of the arrow e in FIG. 4A. In addition, together with the input position information specified by the pointing device, the re-display information indicating whether the input position information indicates the re-display specification or not (ie, indicates the reading area specification) is output from the area specifying unit 7 at the same time. Then, the redisplay selection unit 15 is transferred. The redisplay selection unit 15 transfers the input position information to the layout extraction unit 5 if the input position information is the redisplay region, and transfers the input position information to the region determination unit 8 otherwise. As described with reference to FIG. 5, the function of the layout extraction unit 5 for searching the tree structure stored in the structured data storage unit 4 and transferring each element block information to be displayed to the display unit 12 includes: It has the same function as the embodiment of the first invention of the present application,
The search processing for each element block to be displayed is different. That is, the element blocks including the input position information transferred from the re-display selection unit 15 are first transferred to the display unit 12 and detected as re-display element blocks from the displayed element blocks. Search the tree structure stored in 4
A plurality of element blocks behind one of the redisplay element blocks are taken out as element blocks to be displayed and transferred to the display unit 12.

It is assumed that the layout extraction unit 5 transfers one page area as the initial element block information to be transferred to the display unit 12.

(Effects of the Invention) As described above, according to the document reading apparatus of the present invention, the elements constituting a document and the arrangement relation between the elements are structured as a tree structure from the input image, and the layout is displayed according to the tree structure. By designating the element region with one point and with the designated position relaxed, the burden on the user can be remarkably reduced, and characters in a desired region can be easily read from the existing document image.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining a method of dividing a document image into basic elements and then structuring the elements into elements constituting the document. FIG. 2 is a diagram showing an example of a result of generating a document arrangement structure obtained for the document image of FIG. 1 as a tree structure. FIG. 3 is a view for explaining an area designation method applied to the document image of FIG. 1 in the document reading apparatus of the first invention of the present application. FIG. 4 is a diagram for explaining an area designation method applied to the document image of FIG. 1 in the document reading apparatus of the second invention of the present application. FIG. 5 is a block diagram showing one embodiment of the present invention. In the figure, 1 is an image memory, 2 is an area dividing section, 3 is a document structuring section, 4 is a structured data storage section, 5 is a layout extracting section, 7 is an area specifying section, 8 is an area determining section, and 9 is a line. Reading unit, 10 is a character cutout unit, 11 is a recognition unit, 12 is a display unit, 13
Is a recognition dictionary, 14 is a recognition result storage unit, and 15 is a redisplay selection unit.

Claims (1)

(57) [Claims] 1. A region dividing means for decomposing a document image into basic element blocks such as character lines, figures, etc., and sequentially structuring the plurality of basic element blocks to form an element block constituting a document image and an inter-element block. A document structuring means for structuring the arrangement relationship as a tree structure hierarchically expressed, and a predetermined level or a layout structure of element blocks selected by a redisplay area selecting means described later are extracted using the tree structure, Layout extracting means to be displayed, re-display area selecting means for selecting an element block in which detailed layout information is to be expressed from one or a plurality of layout-displayed element blocks, and one or more layout-displayed one or more element blocks. Area determination means for designating and determining a reading area at one point from an element block, and a vertical search of the tree structure sequentially for only character lines in the reading area. And reading row readout means in the order,
A document reading apparatus comprising: a character recognition unit that cuts out a read character line in units of one character, collates the character line with a recognition dictionary, and sequentially outputs character reading results.