JP2001076095A - Information processor and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prepare electronic document data which suitably saves and delivers a document image and are easily electronically documentated. SOLUTION: The layout of an inputted document image is analyzed (S103) to recognize a texture area and a picture area. The overlap area where the recognized text area and picture area overlap with each other is detected (step S104), and the storage style of the document image is switched, according to whether the overlap area is detected, thereby storing document image data (S105 and S106).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an information processing apparatus and method for handling document originals read by optical reading means and reusing the data.

[0002]

2. Description of the Related Art Conventionally, when a document image is stored and distributed,
Images have been stored and distributed using black and white binary images using MMR compression and color images using JPEG compression. Further, in recent years, image recognition technology has been improved, and a demand for not only saving and distributing an image but also reusing it as an electronic document has been increased, and is being realized.

Normally, when a document image is reused, it is converted into an image format suitable for recognition processing, layout analysis processing is performed, and character recognition processing is performed on a character portion to generate an electronic document. However, both the layout analysis processing and the character recognition processing sometimes make a determination error. In particular, in the character recognition processing, even if the accuracy is 99%, the remaining 1% error will inevitably occur. This is not simply caused by the character recognition processing, but may be caused by pre-processing image binarization processing or layout analysis processing. In any case, it is almost impossible to completely reproduce it. For this reason, it is common to store and distribute image information using image compression that has been conventionally performed.

[0004]

However, in the conventional method, for example, when storing and distributing a color document image, JPEG compression for relatively faithfully compressing the original image is often used. If it is a JPEG compressed image, for example, the device on the receiving side will perform decompression processing in order to confirm the contents. At this time, it is necessary to confirm that the original image is high resolution or large in size. A large amount of memory is required, and the time spent for the decompression process increases.

[0005] Therefore, in the case of a document image covering a plurality of pages, the receiving side sometimes discards the data without confirming the content.

Therefore, in order to reduce the file size of the color document image and reduce the memory capacity required for the decompression processing, the color document image is distributed to black and white binary and distributed. However, such processing results in loss of color information or blackening or fading of the photographic part during binarization, resulting in a significant deterioration in image quality. Further, in the case of a document image with a base, the image may look like noise. The use of a black-and-white binary image is used on a compromise that the character portion is important in the document, so that the character portion only needs to be clearly readable. A good example is sending by fax. FAX ultimately deals only with black and white documents, which is often sufficient.

[0007] In the case of a black and white binary image, as a typical example, a simple binary image and an image binarized by error diffusion are often handled. In the former case, the text part is beautiful, but the photo part is blackened. On the other hand, the latter
In MMR compression, the file size becomes large, and background noise appears in the character portion. In particular, in the latter case, it is very difficult to make an electronic document for the purpose of reuse as described above.

[0008] In any case, when an image is handled, there is a considerably high barrier to the electronic document which can handle the document image as a code. In fact, it was not much reused.

Therefore, an electronic document format satisfying the following requirements has been desired.・ Color images remain in color,
The size must be small and the reproduced image must be beautiful. -Even in a black-and-white image with a background, the necessary parts must be clarified so that the background noise does not appear and the image is not ugly. -All of them must be reusable electronic documents that can be imported into application software on a computer.

Accordingly, it is an object of the present invention to provide an information processing apparatus and method capable of generating an electronic document format satisfying the above requirements.

That is, an object of the present invention is to provide an information processing method and apparatus capable of creating document image data suitable for storing and distributing a document image and which can be easily converted into an electronic document.

[0012]

An information processing apparatus according to the present invention for achieving the above object has, for example, the following arrangement. That is, analysis means for performing layout analysis on the input document image and recognizing at least a text area and a picture area, and detection means for detecting an overlapping area where the text area and the picture area recognized by the analysis means overlap. A storage unit for switching the storage mode of the document image based on whether the detection unit detects an overlapping area and storing the document image data.

[0013]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

In this embodiment, PAF (Page Analysis
Format) is temporarily set.
The problem is solved by using the PAF as an intermediary.

<PAF> In PAF (puff), a document image is image-analyzed, and the area in the document image that can be coded is
Code as much as possible. Areas that are difficult to code
Save as a partial image. FIG. 1 is a diagram illustrating an electronic document format (PAF) according to the present embodiment. 11 is P
Although the entire AF configuration is shown and not shown, property information of the entire document (for example, an item for managing the number of pages of the document PAF, etc.) and management information (for example, at what time and what scanner is inputted) ) Are stored.

[0016] The Text Object section 12 stores the Te in the document image.
The area determined as the xt area is held. The Text area is
This area has the highest compression efficiency. The text area holds data in two storage methods. One is Text
This is a method of storing an area as a binary image. This is black and white 2
Of course, the value image is T
Only in the ext area, a binary image is stored. On the other hand, another storage method is a method in which a text area is subjected to character recognition processing to be converted into a character code and stored. The text area encoded with the character recognition code becomes a reusable document image.

The Picture Object unit 13 stores a natural image such as a photograph or an image area which cannot be vector-coded by analysis as a partial image. Here, when the original image is a binary image, it is held as a partial binary image, and when it is a multi-valued image, it is held as a partial multi-valued image.

The drawing object section 14 is a section for holding an area such as a line, a line drawing, and an enclosing frame that can be converted to a line vector. For example, in the case of a horizontal line, its start position, end position, thickness, line type, and the like are held.

The table object section 15 is a section for holding an area determined as a table area in a document image. For example, in addition to the number of vertical and horizontal cells in the table, its cell configuration information, table frame information,
Text written inside the cell is also held.

An electronic document format stored in the above-described format based on the result of layout analysis of a document image and the analysis result is called PAF.

FIG. 2 is a diagram showing adaptive compression in which appropriate compression is performed for each object and stored. As described above, when an image is stored, a compression method suitable for the area is selected, compressed, and stored. This reduces the file size. Text Object area 12 and Table
The Object area 15 may be stored as a binary image. In this case, all of the images except for the drawing object unit 14 indicating a line are a group of partial images. In order to make the following description easy to understand, the above-mentioned format is stored in IMG-PAF.
(Image puff). The format in which the result of performing the character recognition processing on the area determined to be the image area including the Text from the layout analysis processing result is set to O-format.
Called PAF.

By saving the electronic document PAF as described above instead of the document image, the file size can be reduced or the file can be reused.
In addition, there are other advantages such as enabling a full sentence search.

However, in performing such PAF conversion, not all document images can be appropriately converted into partial objects. Since the document image shown in FIG. 2 is a document laid out relatively neatly, it can be efficiently reduced to a file even if it is PAF-formatted. It is also possible to faithfully reproduce a reproduced document image. However, in the case of a document image that is more complicated, even if the IMG-PAF obtained as a result of the layout analysis is stored as it is, the processing of the overlapping area becomes complicated, and the reproduced electronic document may become useless. . In this regard, FIG.
This will be described with reference to FIG.

FIG. 4 is a view showing a document image in which the respective areas are arranged neatly as a result of the layout analysis. FIG. 5 shows a document image with a slightly complicated layout, and FIG.
This shows a document image layout analysis result in which a TEXT section (text area) overlaps or enters an ICT section (image area). When a complicated analysis result as shown in FIG. 6 is stored as it is as a partial area image, there are many overlapping parts, and the file size becomes large,
Also, at the time of reproduction, the reproduced image is such that the Text portion in the image is unusually raised. That is, it is necessary to determine in advance which document image may be a partial storage method, and which document image returns an inefficient and unsightly image.

In the present embodiment, a document image as shown in FIG. 4 is treated as a subdivided partial image as much as possible, and a complicated document image as shown in FIG. Any document image can be PAFed to create its benefits by adding a classification technique to treat it as an image. Furthermore, in the process of saving a partial cut-out image for saving each partial image, it is determined that a document image that cannot be reproduced well can be kept as the original image. This makes it possible to preserve the original image information as much as possible when layout analysis and character recognition are likely to produce bad results. Can be

Hereinafter, the processing of this embodiment up to the PAF conversion will be described. FIG. 12 is a block diagram illustrating the configuration of a system that executes document image processing according to the present embodiment. In FIG. 12, reference numeral 101 denotes a CPU;
02 and a control program stored in the RAM 103 to execute various processes. Reference numeral 104 denotes an external storage device, which is appropriately stored in the RAM 103 for execution by the CPU 101.
It stores various application programs and document image data to be loaded into the. Reference numeral 105 denotes a display, which performs various displays under the control of the CPU 101. 10
An operation unit 6 includes a keyboard, a pointing device, and the like. A bus 107 connects the above-described components. It is assumed that a control program for realizing processing described below with reference to the illustrated flowchart is stored in the external storage device 104, loaded into the RAM 103 as needed, and executed by the CPU 101. Of course, these control programs are stored in the ROM 103
May be stored.

FIG. 3 is a flowchart for explaining the procedure of the PAF processing according to the present embodiment. First, a document image is input in step S101. The document image input here is a scanner image or an image file. In step S102, preprocessing is performed. In the preprocessing, if the document image is a multi-valued image, a binarization process is performed on the image. For example, in the case of an RGB color image, first, a black-and-white gray scale conversion is performed and then a binarization process is performed. This binarization process is not a binarization process for printing but a binarization process for a layout analysis process to be performed later. For example, simple 2 with 128 as the slice level
A binarized image is created by a binarizing process or a binarizing process in which an appropriate threshold is obtained based on a result of histogram analysis of a partial area in the document image and the slice is used as a slice level. The reason for using the binarization processing is that a binary image is generally used in the layout analysis processing and the character recognition processing that will be described later. An additional reason is to reliably remove the background color. This is because it is difficult to determine the area of the character in the background and the character cutout. On the other hand PI
It is preferable that the CT unit is a region uniformly blackened. If black is flying, TE in the PICT section
This is because an erroneous recognition such as an XT part is easily performed.

Further, if the document image is inclined, it is easy to make a recognition error. Therefore, the inclination of the document image is corrected. Also, a document orientation correction process is performed so that the document image does not face left or upside down.

In step S103, a layout analysis process is performed. Here, the processing method of layout processing is
It is possible to use a known technique such as layout analysis based on contour tracing, such as a contour tracing method that detects a block of black pixels in an image and follows the contour, or sequentially detects a number when a black pixel area is detected. There is a labeling method to add. Then, the attribute of the area is determined from the size, position, and the like of the detected mass of black pixels.

As a result, when the layout analysis processing is performed,
TEXT (character), TITLE (title), CAPT
ION (caption), LINEART (line drawing), P
ICTURE (natural image), FRAME (frame), LINE
The attribute information of each block recognized for each attribute such as (line) and TABLE (table) and its rectangular address information are output as a layout analysis result. Text Objec in Figure 1 above
Areas belonging to the t section 12 are TEXT (character), TITL
E (title) and CAPTION (caption). Similarly, the Picture Object unit 13
(Natural image), LINEART (line drawing), Drawing Object
Part 14 includes FRAME (frame), LINE (line), and
Table applies to the Table Object section 15.

Next, in step S104, an image understanding process is performed. This process is an important process in the present embodiment, and details thereof will be described later. As a result of this image comprehension processing, the document images are classified in order to perform appropriate PAF conversion.

In step S105, an IMG-PAF conversion process is performed. In this IMG-PAF processing, a partial image is cut out according to the rule of the image understanding processing in step S104. If the original image is a multi-level image, the PICT unit cuts out the multi-level image as it is.

Next, in step S106, the IMG
-Perform PAF storage processing. Here, step S105
Performs adaptive compression on the images cut out in step (1). Then, the compressed image is stored together with the layout information and is converted into IMG-PAF.

Subsequently, in step S201, IM
The data stored in the G-PAF format is read again. In step S202, a text area is extracted from the layout information. In the text area, a binary clipped image is stored. If this image is compressed, it is expanded. In step S203, a character recognition process is performed on the image stored in the text area. In this character recognition processing, language determination is performed as preprocessing. Generally, Japanese-English distinction is performed. Next, group orientation discrimination is performed to discriminate between vertical writing and horizontal writing. Then, for example, in the case of Japanese, the processing is executed using a Japanese character recognition engine.

In the O-PAF conversion process in step S204, the character code and character recognition information obtained in step S203 are incorporated into the PAF again. In this case, the image in the text area may be deleted as needed. Then, in the O-PAF storage processing in step S205, the O-PAF obtained in step S204 is stored.

The software for realizing the above flow is hereinafter referred to as PAF Capture. The point of this embodiment is to determine what classification a document image belongs to and what kind of image storage should be performed using the layout analysis result in step S103. This process is called an image understanding process.

<About Classification of Document Images>
The layout can be classified by focusing on the layout. FIG.
FIG. 3 is a diagram showing a classification system of document images according to the present embodiment. The document image 51 first includes an alignment layout image 52,
It is largely classified into two types of non-aligned layout images 53. The aligned layout image 52 is a document in which the rectangles of the respective attributes as shown in FIG. 4 are laid out neatly.
For example, patent publication documents belong to this category.

The non-aligned layout document 53 refers to a document image in which the PICT area and the Text area as shown in FIG. 6 overlap or are included. Thus, the non-aligned layout image 53 is a document image in which it is difficult to completely separate areas.

The non-aligned layout image 53 can be further separated into an overlapping rectangular layout 54 and other layouts 55. The layout of FIG. 6 is included in the overlapping rectangular layout 54. In the case of the layout as shown in FIG. 6, a fairly good reproduction result can be obtained as long as the image cutout rule is established. Other layout 5
5 include, for example, an image without characters,
If the result of the region discrimination is extremely distorted due to the image being tilted, it is classified here. The document images classified into the other layouts 55 do not need to be forcibly changed to partial area images, but may be left as original images.

In this embodiment, the alignment layout image 5
2. Extract a document image that fits the classification of the overlapping rectangle layout 54, and prevent the other parts from being provided to the processing system for storing the cut-out image, thereby preventing improper cutting-out of the image.

FIG. 8 is a diagram showing an overlapping pattern of the PICT section and the TEXT section. In FIG. 8, (a) shows PI
This is an example in which a TEXT section is completely included inside a CT section. FIG. 9A shows an image example in which a rectangle as shown in FIG. 8A is obtained. FIG. 8B shows a PIC.
This is the case where a part of the TEXT part overlaps the T part, and FIG. 9B shows an example of the image. These two types of patterns are used when the layout analysis
The overlap flag of the XT section is set. However,
Different flags are set for the cases (a) and (b). FIG. 8C shows PI in the PICT unit.
FIG. 9B shows an example of an image obtained by analyzing the presence of a CT unit. Similarly, FIG. 8D shows a case where the PICT unit partially overlaps another PICT unit, and FIG. 9D shows an example of the image.

Using the above image examples, step S1
The image understanding process of No. 04 will be described. FIG. 10 is a flowchart illustrating the procedure of the image understanding process according to the present embodiment. First, a layout analysis is performed in step S301. In step S302, an overlap flag is added to the layout analysis result. This overlap flag is added when the PICT unit and the TEXT unit overlap as shown in FIGS. 8 and 9 (a) and (b).

In step S303, an overlap flag is checked for the layout analysis result. Pict
If there is no overlap between the part and the TEXT part, the process proceeds to step S304, and it is determined whether or not a TEXT part exists in the image. If the TEXT portion exists, it is determined that the image is the aligned layout image, and the process proceeds to step S30.
Then, the process proceeds to step S105, where the image data is sent to the partially cut-out image processing system for executing the IMG-PAF conversion in step S105. On the other hand, step S
If there is no TEXT area in 304, it is determined that the layout image is another layout image, and step S310 is performed.
Proceed to.

If it is determined in step S303 that there is an overlap, the process proceeds to step S305,
A TEXT part that at least partially overlaps the PICT part is detected, and is extended to a PICT area including a protruding part of the TEXT part. For example, in the case of the PICT unit in FIG. 6, the PICT unit is changed to the one in FIG. Here, a rectangle is connected without being a large circumscribed rectangle.

In step S307, it is detected whether there is another rectangle other than the overlapped rectangle. If there is no layout, the process proceeds to step S310 to determine another layout. In the case of this document image, the PICT unit and the TEXT
This is because it is determined that it is better to keep one document image as it is, rather than separate and complicated.

If it is determined in step S307 that there is another rectangle, the flow advances to step S308 to determine whether the recombined rectangle occupies 80% or more of the document image. This 80% is an example, and the determination is made in consideration of the position, width, and height of the rectangle. If it is determined that the layout is 80% or more, it is determined that the layout is another layout, and the process proceeds to step S3.
Proceed to 10. In this case, there are many regions where the PICT unit and the TEXT unit are complicated and complicated, and it is determined that it is better not to divide the region. On the other hand, if it is less than 80%, the process proceeds to step S309, and it is determined that the layout is an overlapping rectangular layout.

The document images determined to be the aligned layout or the overlapping layout are both IMG in step S105.
-Save a partial image in the partial image saving processing system in the PAF processing. At that time, in the case of the aligned layout, there is no problem, and partial image clipping is performed, and optimal compression is performed for each. The document image of the overlapping layout is the TEXT and PI
In the overlapping part of CT, connection processing is performed and PIC
T section. The PICT unit and the overlapping part of the PICT unit perform the connection processing in the same manner to form the PICT unit. P
When there is a TEXT part and a PICT part in the ICT part, the larger one includes the smaller one and saves the image.

By using such an image understanding rule, the original image can be automatically saved when it is impossible to prevent the IMG-PAF converted file from becoming an unnatural partial image.

By the above-described processing, in the case of an aligned layout image, a very small file is obtained. In addition, the document images belonging to this classification can be further converted into a more usable electronic document by character recognition processing. on the other hand,
The overlapping layout image is reproduced while leaving the original image portion as much as possible. Thereby, the unnatural TEXT part and PICT
It is possible to eliminate boundaries between departments.

[Other Embodiments] Next, other embodiments will be described. In the above-described embodiment, the image understanding rule is used when saving the IMG-PAF, but the document image automatic classification technology can also be implemented as an automatic classification method of image filing. In this case, an image understanding process is performed on the result of the layout analysis, and the classification shown in FIG. 7 is obtained.
Then, the information of the classification is described in the management information of the file. Then, the document images belonging to the aligned layout image 52 and the overlapping rectangular layout image 54 are classified at least as reusable documents. Other document images belonging to the layout 55 are classified into categories handled as image data. This document will be treated as an image.

This classification is convenient for setting an area for storing a file. You can also search
The time to hit is faster if the images are accessed sequentially from those belonging to the aligned layout images. For example, if the document image of other layout is not included in the search item,
That amount of processing is reduced.

As described above, according to the automatic classification of document images according to the present embodiment, a human may decide that this image should be an image, or that this image may be stored as a partial image. It is not necessary to judge this. In addition, as in the case of an aligned layout image, even if character recognition is performed, and even if it is coded and reused, a very good electronic document can be obtained. There is an effect that can be.

Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus including one device (for example, a copying machine, a facsimile, etc.) Device).

Another object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and a computer (a computer) of the system or the apparatus. It is needless to say that the present invention can also be achieved by a CPU or an MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. Also,
When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also the operating system (OS) running on the computer based on the instructions of the program code.
It goes without saying that a case where the functions of the above-described embodiments are implemented by performing some or all of the actual processing, and the processing performs the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written into the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, the program code is read based on the instruction of the program code. Needless to say, the CPU included in the function expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0056]

As described above, according to the present invention,
This makes it possible to create electronic document data that is suitable for storing and distributing document images and that can be easily converted into electronic documents.

[Brief description of the drawings]

FIG. 1 is a diagram showing a simple structure of an electronic document PAF.

FIG. 2 is a diagram illustrating adaptive compression of IMG-PAF.

FIG. 3 is a flowchart illustrating a procedure of a PAF process according to the embodiment;

FIG. 4 is a diagram showing a layout analysis result of an aligned layout image.

FIG. 5 is a diagram illustrating a layout analysis result of a complicated aligned layout image.

FIG. 6 is a diagram illustrating a layout analysis result of an overlapping rectangular layout image.

FIG. 7 is a diagram showing classification of document images.

FIG. 8 is a diagram illustrating an overlapping pattern of a PICT unit and a TEXT unit.

FIG. 9 is a diagram illustrating an overlapping pattern of a PICT unit and a TEXT unit.

FIG. 10 is a flowchart illustrating a procedure of an image understanding process according to the embodiment.

FIG. 11 is a diagram illustrating the connection between the PICT region and the TEXT region.

FIG. 12 is a block diagram illustrating a system configuration for implementing document image processing according to the present embodiment.

Claims (13)

[Claims] An analysis means for performing layout analysis on an input document image to recognize at least a text area and a picture area, and detecting an overlap area where the text area and the picture area recognized by the analysis means overlap. An information processing apparatus, comprising: a detection unit; and a storage unit configured to switch a storage mode of the document image and store the document image data based on whether an overlapping area is detected by the detection unit. 2. The image processing apparatus according to claim 2, wherein the storage unit stores the text area and the picture area detected by the detection unit as separate partial images when the detection unit does not detect the overlapping area. The information processing device according to claim 1. 3. The information processing apparatus according to claim 1, wherein the storage unit treats the overlap area detected by the detection unit as one picture area. 4. The information processing apparatus according to claim 2, wherein the storage unit stores the document image data by performing an appropriate compression process on each of a text area and a picture area. 5. The method according to claim 1, wherein the storage unit stores the document image without dividing the document image into partial images when the ratio of the overlapping area detected by the detection unit in the document image is larger than a predetermined threshold. The information processing apparatus according to claim 1. 6. The information processing apparatus according to claim 1, further comprising a character recognition unit that executes a character recognition process on a partial image of the area stored as the text area. 7. An analysis step of performing a layout analysis on an input document image to recognize at least a text area and a picture area, and detecting an overlap area where the text area and the picture area recognized in the analysis step overlap each other. An information processing method comprising: a detecting step; and a storing step of switching a storage mode of the document image based on whether an overlapping area is detected in the detecting step and storing the document image data. 8. The storing step stores the text area and the picture area detected in the detecting step as separate partial images when no overlapping area is detected in the detecting step. The information processing method according to claim 7. 9. The information processing method according to claim 7, wherein in the storing step, the overlapping area detected in the detecting step is treated as one picture area. 10. The information processing method according to claim 8, wherein in the storing step, the document image data is stored by performing an appropriate compression process on each of a text area and a picture area. 11. The storage step, wherein when the ratio of the overlapping area detected in the detection step in the document image is greater than a predetermined threshold value, the document image is stored without being divided into partial images. The information processing method according to claim 7, wherein: 12. The information processing method according to claim 7, further comprising a character recognition step of performing a character recognition process on a partial image of the area stored as the text area. 13. A storage medium for storing a control program for causing a computer to implement a process of storing document image data, wherein the control program performs a layout analysis on an input document image and at least a text area and a picture area. Code of an analysis step of recognizing an area, code of a detection step of detecting an overlap area where the text area and the picture area recognized in the analysis step overlap, and whether or not an overlap area is detected in the detection step. And a code for a storage step of storing the document image data by switching the storage mode of the document image based on the storage medium.