JP2000306103A - Method and device for information processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the data volume of a document picture and to hold document data in such a form that data can be easily used again. SOLUTION: A multi-valued picture original 101 is optically read to obtain original image data. Character areas 106 to 108 and image areas 102 to 105 are extracted from obtained original image data, and respective position information are acquired. Image data in character areas are subjected to character recognition, and recognition results are stored in a text layer 116. Image data in image areas are subjected to compression processing, and processing results are stored in a picture layer 112. Data obtained by compressing the whole of original image data are stored in an overall picture layer 111.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus and method for processing image data obtained by optically reading a document original, and more particularly to a method suitable for processing on the premise of reusing read image data. The present invention relates to an information processing apparatus and method.

[0002]

2. Description of the Related Art In recent years, when a paper document is stored or reused on a computer, it is common to read the document image with a scanner and convert the document into image data and hold it. . In order to convert such image data into a character code so that it can be used by word processing software or the like, character recognition processing is performed on the obtained image data to perform coding. At this time, the character information is coded, but the image included in the document is cut out from the image data and held so that it can be pasted into the document. Therefore, in order to store a document manuscript on a computer, it is necessary to perform a layout analysis process of the document image prior to the character recognition process, and the accuracy of the process is becoming increasingly important.

Further, with the advancement of technology, a language analysis is attached to the character recognition process, and the know-how of the typesetting technology is incorporated in the layout analysis process, so that each process can obtain a highly accurate one. It is becoming.

[0004] Under such circumstances, it has become possible to save a document manuscript using the results of layout analysis processing and character recognition processing. If the document image is stored in this form, character search of the document becomes possible, and the possibility of reusing the document image can be improved. As this type of storage format, a file format of a specific application is often used. For example, Microsoft's RTF (Rich Text Format), JustSystem's Ichitaro (trademark) format,
Alternatively, recently, the HTML format is often used on the Internet. Further, although the format becomes a little complicated, it may be stored in the SGML format.

On the other hand, there is a document which can be called an image original unlike a text original document. As the name implies, it is a document mainly composed of images. It is desired that the image be kept as close as possible to the original, the data amount be reduced, and the image handling be excellent. In the document manuscript,
In many documents, it is sufficient that the contents can be confirmed even if the image part is slightly different from the original, but in an image document, the reproducibility of a picture image is regarded as important.

[0006] For this reason, it is common to compress and save the entire surface of an image document. In recent years, an image format called FlashPix (an image format proposed by Kodak Company) has been generalized as an image format for storing images. In this FlashPix, an image is divided into block areas of 64 × 64 pixels, and encoding is performed for each block. In the coding for each block, a coding method can be selected for each block. Then, such encoding is performed by changing the resolution of the image step by step, and each result is held. Another feature is that the image management method can be stored in detail. This image format realizes an image handling method that reduces the amount of data and satisfies the requirements for a high-resolution image when outputting to a printer and a low-resolution image when displaying on a display.

The FlaxhPix format is an image format mainly suitable for a photographic image such as a one-sided natural image. In addition, as the encoding method used for each block, JP
EG encoding, JBIG encoded MMR encoding, and the like. An image document is also stored in the FlaxhPix format.

[0008]

A document image mainly composed of characters sometimes has a picture area, and in such a case, importance is placed on the reproducibility of the picture area. Should. Alternatively, there is a case where a character area exists in an image original mainly composed of a picture, and there is a demand that the part is to be converted into a character code.

As described above, there is a demand for a document image storage format that realizes image reproducibility and character encoding. However, there are the following problems in a format that satisfies both, and a method of creating the format. (1) In general, the image of one entire page is compressed and held, so that when the compression ratio is increased, the deterioration of the image becomes remarkable. (2) A processing method and a format for analyzing layout of one page and dividing and storing the divided pages are not established. (3) In the prepared document image analysis data, if the analysis result is not satisfactory, is it in a format that can be easily corrected? (4) How to exchange data with other application software formats. (5) How to search as document image data

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to reduce the amount of data of a document image and to retain document data in a form that facilitates data reuse.

[0011]

An information processing apparatus according to the present invention for achieving the above object has, for example, the following arrangement. That is, a reading means for optically reading a document manuscript to obtain manuscript image data, an analyzing means for extracting a character area and an image area from the manuscript image data and obtaining respective position information, Recognition means for performing character recognition on the image data of the obtained character area; compression means for performing compression processing on the image data of the image area; recognition result data obtained by the recognition means; A storage unit for storing the compressed data and the position information obtained by the analysis unit;

[0012]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

In this embodiment, an image processing apparatus which analyzes a document image and saves (converts into a micro document) in data having a reusable format will be described.

In this embodiment, an image processing apparatus satisfying the following items will be described. (1) When a document document image is input by a scanner, all image formats such as a binarized image, a monochrome grayscale image, an RGB 256-color palette color image, and an RGB full-color image can be input. (2) The image of each format is converted into image data that can be subjected to layout analysis. (3) A layout analysis process is performed on the image in (2), and a layout analysis process is performed for each attribute such as a Text area and a Picture area in the document image. (4) In the layout analysis processing, the order recognition of the Text area and the priority analysis of the Picture area are performed. (5) Perform character recognition processing for the Text area. (6) The picture in the Picture area is cut out from the input original image. (7) The cut-out image is compressed by a specified encoding method. For example, create an image in FlashPix format. In the case of the FlashPix format, necessary information is described in the information description section based on the layout analysis processing result and the character recognition processing result. (8) Full compression encoding of the original image. For example, 1/100
Target compression ratio of the order. (9) The entire compressed data, cut-out image data, layout analysis data, and character recognition data are collectively saved as one image analysis format. (10) In the case of the image display search, the whole compressed data or the cut-out image data is displayed and searched. (11) In the case of a character code search, a search is performed using a character recognition result. In that case, a fuzzy search including candidate characters is also performed. (12) A document image to be re-created from the analysis format is created using a character code, a line vector, and a pasted image. If the created document image is inappropriate, the document image is corrected using the entire image compression data. (13) A general file format, an RTF format, an Ichitaro format, an HTML format, an XML format, and the like are created by converting this analysis image format. (14) Reverse conversion conversion is performed from the general file format to the present analysis format.

In order to realize the above functions, in the present embodiment, the document image is stored as parts according to the contents. Parts that can be coded and vectorized are coded and vectorized. Compression encoding is performed on a natural picture or picture area where high-definition information is to be held so that a high-definition image can be reproduced. By saving a document image in such a format, the large memory capacity of the original image can be reduced, the necessary area (parts) does not deteriorate the information, and the character code can be automatically obtained. Occurs.

FIG. 18 is a block diagram showing the configuration of the image storage device according to the present embodiment. In FIG. 18, 11
Denotes a CPU, which executes various processes of the image storage device in the present embodiment. A ROM 12 stores a boot program executed by the CPU 11 and various parameters. Reference numeral 13 denotes a RAM which provides a load destination of a control program to be executed by the CPU 11,
11 provides a work area for executing various processes.

A display 14 performs various displays under the control of the CPU 11. An input unit 15 includes, for example, a keyboard and a pointing device, and realizes various operation inputs to the image storage device. Reference numeral 16 denotes an external storage device, which is configured by, for example, a hard disk, and implements a control program for realizing an image storage process described later,
The image data generated by the image storing process is stored. A scanner 17 reads a document to be stored and provides document image data. A printer 18 records a visible image based on the input image data. A system bus 19 connects the above-described components.

First, a micro document which is a document data storage format of the present embodiment will be described.
FIG. 1 is a diagram illustrating a data structure of a micro document according to the present embodiment.

In FIG. 1A, reference numeral 101 denotes a multi-valued image original. Although shown here as a line drawing for convenience of illustration, the multi-valued image document 101 is an RGB color image or an image in which density values in other formats are stored. In the multi-valued image document 101, reference numerals 102 to 105 denote photographic images,
06 to 108 are character images. It is assumed that the image data of the multi-valued document image data 101 is stored at a high resolution of 300 DPI or more in order to reproduce the state of the original document as much as possible. In this case, if an 8-bit multi-valued image for each of RGB is used, an A4 document has a memory amount of 30 MByte or more.

In the present embodiment, such a document image is subjected to image analysis, and is separated into parts according to the contents written inside. Then, each part is stored in a layer as shown in FIG. The data of the layer structure including the whole part becomes the micro document of the present embodiment. In the following, each part in the document is referred to as an object in the document.

FIG. 1B conceptually shows a layer where each object is managed. The document image shown in (1) of FIG.
An object is stored in each layer according to the image content.

The entire image Layer 111 at the highest level is
This is a hierarchy for storing the result of compressing the entire document image by JPEG compression or the like. In order to handle a document image as image data, it is sufficient to expand this data. Full image
In the layer 111, in order to increase the compression ratio, even if the image deterioration is somewhat large (that is, even if the image quality is somewhat sacrificed), the compression is performed at least about 1/100 as a guide.
Since the entire image layer 111 stores the entire image as an image, even if the image is slightly deteriorated, it is useful for correcting the result of another layer storing the document analysis result,
It is also useful as a thumbnail image that displays the entire page as a small image. In addition, when the image data of the full-size image Layer 111 is restored and displayed on a display, it is assumed that an image that is satisfactory as a document image can be obtained. On the other hand, this entire image Layer 111
It is unavoidable that jaggedness or the like appears when printing is performed using the above data.

The Picture Layer 112 stores image data of a region having a Picture attribute obtained from a document analysis result described later. Here, “Picture” refers to an area where the density of a photograph or the like changes greatly and is determined to be a natural image. FIG.
In (1), the portions indicated by 102 to 105 are Pictur
It becomes a region having the attribute e.

FIG. 2 is a diagram showing image data obtained by cutting out the Picture area 103 of the multi-valued image document 101. FIG. 3 shows a picture area 1 of the multi-valued image document 101.
It is a figure which shows the image data obtained by cutting out 04.
A compression method capable of reproducing a high-definition image is applied to an image cut out as a Picture area. That is, the cut-out image is compressed so that the image can be reproduced clearly both on the display and during printing. For example, when outputting to a 400 DPI printer, the picture area (Picture area) is
If the data stored in the er112 is called and transferred to the printer, the image area can be reproduced clearly.

As a method of compressing the Picture area, the above-described JPEG compression method may be used, but in the present embodiment, the Picture area is stored in a FlashPix data format. Hereinafter, FlashPix will be briefly described.

FIG. 4 is a diagram showing tile division of an image by FlashPix. In FlashPix, encoding is performed in each area divided into 64 × 64 dot blocks. In addition, as shown in FIG. 5, an image whose resolution is changed stepwise is stored together in advance. For this reason, the printer has 300 DP
In the case where an image of I is used and an image of 75 DPI is used for display, a low-resolution image can be directly obtained without performing a process of creating a low-resolution image from a high-resolution image. Since the image has a stepwise resolution, the data amount increases accordingly. However, since the image is a thinned-out image with a low resolution, the increase amount is slight.

As can be seen from the above description, FIG. 2 shows that the image data in the
This is an example of hijacking the Pix format and saving the image at a gradual resolution. Similarly, FIG. 3 shows an example in which the image data in the Picture area 104 is hijacked into the FlashPix format and the image is stored at a stepwise resolution. The image data of each part thus obtained is stored in the Picture Layer 112.

Next, in the Line Art Layer 113 of FIG.
Unlike the above-mentioned Picture Layer 112, the image of the area determined to be a line drawing is stored. Unlike a natural image, a line drawing area is an image created by computer software and has a relatively small change in density and many blank areas.

This line drawing area can be stored with such information if line drawing vector conversion can be performed. In the present embodiment, the data obtained by the vector conversion is stored in the Line Vectore Layer 114. However, the line drawing vector conversion cannot be completely converted by the current technology, and is generally handled as image data. Although CAD drawings and simple line drawings are easy to vectorize, it is more certain that they are converted by post-processing from a state once stored as image data. Therefore, in the present embodiment, as shown in FIGS. 2 and 3, hierarchical multi-resolution image storage in the FlashPix format is performed for line drawings.

The Table Layer 115 stores the results of table extraction and analysis. The special layer for the table is provided for the following reason. That is, the numerical value of a table has a special meaning in a document, as in the case of a spreadsheet software field. For this reason, when only a table in a document is required, it is provided as one layer so that only this layer (Table Layer) needs to be referred to. FIG. 6 is a diagram for explaining a configuration example of the table. As shown in FIG. 6, 3 × 3
Is a simple table, the outer frame Table (a), each inner frame Cell (b), the character string Element (c) for each line, and the range Region (d) surrounded by a line Divided into such parts. Actually, there are more complicated tables, so it will be more complicated form, but it will be described here in this degree.

The Text Layer 116 is a region for performing character recognition processing on the image data of the character region and the analyzed region (regions 106 to 108) in the document image and storing the result. Here, not only the character code information but also information such as candidate characters, decoration information, size, font, etc., the number of character lines, vertical writing, horizontal writing information, etc. obtained by the character recognition processing are stored. If only the first candidate of the result of character recognition is extracted and stored, a raw plain text can be obtained, but it is highly likely that the result includes a misrecognized character.

The Layout Layer 117 is an area for storing layout information as a result of layout analysis of a document image. The layout layer 117 stores rectangle information (including the size and position of the rectangle) and other additional information of each area for each attribute.

Next, how to create the information in the form of a Micro Document as described above will be described. In the following description, the multi-valued image document 10 shown in FIG.
1 will be described as an example. The document image may be a color image or a black-and-white multi-valued image. In any case, as shown in FIG.
The entire image after reading the document has the most information and the largest image data.

FIG. 7 is a flowchart illustrating a procedure for generating a micro document according to the present embodiment. In FIG. 7, the processing shown in steps S52 to S55 shows a procedure for creating image data suitable for layout analysis.

First, when a multi-value image is input in step S51, brightness information is created in step S52. When the multi-valued image is an RGB color, if the color is an image suitable for the CIE color space, the brightness information can be created according to a well-known predetermined arithmetic expression. In addition, if the image is simply obtained by the scanner characteristics, there is no problem even if the GREEN image is simply the brightness information. Since Green image data is located at the center of the color wavelength, characters, line rectangles, and the like often belong to this color. Strictly speaking, brightness information is generated for each color, but the description is omitted here.

The brightness image obtained in step S52 is equivalent to a monochrome gray scale image. Therefore, in the case of a monochrome grayscale image, the processing may be started from step S53.

In step S53, a background removal process is performed.
This is because if the density of the background exists to some extent in the case of a grayscale image, subsequent layout analysis cannot be executed unless the background and the lines drawn on the background are separated. As a simple method, there is a simple binarization method in which separation is performed at a fixed threshold. However, for each region of the image,
An adaptive binarization method that changes the threshold is most suitable. This method is suitable for character extraction for each area because the threshold value is changed in a part of the image.

In step S54, the image data from which the background has been removed is binarized. This process is performed simultaneously with the background removal in step S53, but is separately illustrated in FIG. 7 for explanation. As a result, finally, in the multi-valued image 101 shown in FIG. 1, the image portions of the regions 102 to 105 are relatively blackened, and the regions 106 to
A binarized image in which the character portion 8 clearly appears. In step S55, the binarized image is held.

In step S56, a layout analysis process is performed on the binarized image held in step S55 to obtain rectangular information as shown in FIG. Each rectangle 102 '
To 108 'correspond to the regions of the images 102 to 108 in FIG. 1, respectively. Note that a well-known layout analysis technique can be applied to the layout analysis processing. The brief description is as follows. That is, the image data is
The inspection is performed from the upper left, and when a black pixel is detected, an image around the black pixel is examined, and an image is extracted by a method called contour line tracking. The entire image is examined by this method, and the attribute of the region is determined from the obtained contour information, the state of the black and white pixels of the image, the position information of the black pixel rectangle, and the like. Although not always accurate, a very high precision layout analysis result can be obtained by the technical development to date. And
In step S57, each object is stored in the corresponding layer according to the attribute determined in step S56.

Next, each attribute will be described. FIG. 9 shows each attribute obtained by the layout analysis processing. In FIG. 9, the attributes are "TEXT", "PECTURE", "TABLE", "LIN".
"TEXT" is a character area, "PICTURE" is a natural image and line drawing area, "TABLE" is a table area, "LINE" is a line area,
“FRAME” indicates a frame area. “NOISE” is an unconfirmed area, which is simply determined as noise or added to, for example, an area that cannot be determined as a “character area”. The “NOISE” attribute is further classified into two types: a case where the analysis is further performed to change the attribute to another attribute, and a case where the attribute is erased as noise.

The "TEXT" area stores the text "TX_TEX".
T ”, title“ TX_TITLE ”, caption“ TX_CAPTIO ”
Detailed attributes such as “N” and in-table characters “TB_CELL” are added.Captions indicate titles and annotations of figures, tables, etc. The “PICTURE” area is a natural picture “PI_PICTUR”.
Detailed attributes such as “E” and the line drawing “PI_LINE_ART” are added.The “TABLE” area has a table frame “TB_FRAME”, an area “TB_REGION” surrounded by table rules, and a character “TB_CEL” in the table.
L ”, one-line character block“ TB_ELEMEN ”in CELL in the table
A detailed attribute such as "T" is added.

Note that TABLE has been described with reference to the example of FIG. 6, but "TB_REGION" has not been described. “TB_REGION” indicates area information surrounded by a ruled line in a table in which no ruled line is drawn. In some tables, meaningful CELLs are not separated by lines. Just because they are not separated by lines,
If you use one CELL, you will get strange results if you take it to spreadsheet software. Therefore, the correct CELL state is analyzed by table analysis processing, and REGION is also required as information for that purpose.

The "LINE" area has three types of detailed attributes: a horizontal line "HLINE", a vertical line "VLINE", and an oblique line "SLINE". The “FRAME” area has outer frame information when there is an outer frame whose four corners are surrounded by lines.

The following describes which of the layers described with reference to FIG. 1 the attribute given to each area in the analysis processing described above. Each layer is a collection of similar document image parts.

FIG. 10 is a diagram for explaining the storage state of image data in the Picture Layer 112. here,
The state in which the image data itself cut out from FIG. 1A is stored by rectangles 102 ′, 103 ′, and 104 ′ (of course, the image data of rectangle 105 ′ is also stored. Omitted).
As described above, the Picture Layer 112 contains image parts (only a part thereof) cut out as PI_PICTURE from the original image shown in (1) of FIG. This image is stored, for example, in the aforementioned FlashPix format.
Of course, other compression methods such as JPEG may be used.

FIG. 11 is a diagram showing a data storage state in the Text Layer 116. The Text Layer 116 also stores a character recognition processing result described later. The character area rectangles 105 'to 108' are numbered in the layout analysis processing by a method of analyzing the order of a document in order called order recognition. This is because the sentence is divided into the respective areas, and the sentence becomes very difficult to read unless it is arranged in order when reconstructing.
Note that a well-known order recognition technique can be applied to this order recognition.

The information stored for each area as described above has a layer structure, and each layer is associated with another layer to be one analysis result. FIG. 12 is a diagram showing a data structure in a micro document format according to the present embodiment. Here, the association of each layer (description) is performed by a link. For example, information indicating where an actual cut-out image is stored is added to image information of a certain image area. The layout description layer has a format in which attributes and rectangle information of each area are collectively managed and linked to a detailed layer. The layout description 602, TEXT description 603, table description 604, Pictur
e description 605, Line Art description 606, LineVector description 6
07 and the front image description 608 are respectively Layout Layer 11
7, Text Layer 116, Table Layer 115, Picture L
ayer112, Line Art Layer113, Line Vector Laye
r114 corresponds to the front image Layer 111.

Next, the PICTURE layer will be further described. Natural images such as photographs and pictures drawn with draw software have a relatively high density of black pixels in the binarization method. On the other hand, a picture drawn with a line has a relatively low density of black pixels. Using this property and some information, PI_PICTURE and
Separate into PI_LINE_ART.

In particular, in the PI_PICTURE section, in the micro document of the present embodiment, there is a lot of information for which a high-definition output must be obtained even when printed. Naturally, image rectangle information, color, resolution information, etc. are added. In the present embodiment, if there is a character in the image and the area can be extracted, the character information can be added. Further, it is also possible to add the order of the image having priority in the document. As described above, information required in the FlashPix format can be obtained from the layout analysis and character recognition results.

Next, how to reconstruct a document image from this micro document will be described.

A procedure for reconstructing a document image from a micro document according to the present embodiment will be described below with reference to the flowchart of FIG.

When execution of document reproduction is started in step S801, parts are collected from each layer in step S802. In step S803, Layout
The collected parts are arranged according to the layout information stored in Layer 117, and the document is reconstructed.

Here, as for the character area, the character string of the first candidate as a result of the character recognition processing is reproduced in a font possessed by the computer at an equivalent position. Lines and frame areas are converted into line drawing commands based on the start, end points, thickness, etc., and lines are drawn. The table draws its frame from the table frame information, and writes characters, numbers, and the like in the cell in the same manner as the character information.

For the Picture area, the cut-out image is called and the image is pasted at an equivalent position. The resolution of the image used at this time is set in step S804.
To S806. That is, in step S804, it is determined whether to perform display display or print output. If print output is to be performed, the process proceeds to step S805, and a high-resolution image is adopted as an image to be pasted in the Picture area. On the other hand, if it is output to the display device, the process proceeds to step S806, and Pictur
Adopt a low-resolution image as the image to be pasted to the e-region. In this embodiment, since the data is stored in the FlashPix format, for example, if the display is a display,
In step S806, an image of about 70 DPI which is affordable for display is selected and displayed. On the other hand, when printing
In step S805, a high-resolution image is selected and printed. This is because the low-resolution image used for the display becomes a rough picture when printed.

As described above, the document manuscript is reproduced only with the analysis result, and this processing is completed (step S805).

As described above, FIG. 7 and FIG.
The flow of the document image saving process and the document image reconstruction process according to the present embodiment will be further described with reference to FIG.

An original image data 901 as shown in FIG. 1A is prepared. In the layout analysis 902, binary image data obtained by binarizing the document image data 901 is generated, layout analysis is performed, and a layout analysis result 903 as shown in FIG. 8 is obtained. In the character recognition / image encoding 904, character recognition is performed for a portion determined to be a character portion by layout analysis, and image coding is performed for a portion determined to be a Picture portion. In the document part conversion 905, parts conversion for one page of the original image is executed, data in the form of a microdocument shown in FIG. 1B is constructed, and stored in the external storage device 906.

Next, the reconstruction of the original image is performed by using Picture Layer
This is realized by arranging the image data held in r907 and the text data held in Text Layer 908 in accordance with the layout information held in Layout Layer 117. Here, as shown in 909, Pict
Display and search on the display 14 can be performed using only the image held in the ure Layer 907. Alternatively, as shown at 910, display, correction, and search can be performed only with the Text Layer 908. Further, as shown in 911, Picture La
The document analysis result can be printed out by the printer 18 using the yer 907 and the Text Layer 908. At that time, the image part uses the one with the higher resolution.

Next, the correction of the analysis document data will be described. FIG. 15 is a diagram illustrating an example of a display screen for correction confirmation. In FIG. 15, reference numeral 161 denotes a screen on which an original image is displayed and a layout analysis result is displayed thereon. In the above, each part is expressed as an object. On this screen, the object can be deleted, the result of order recognition can be corrected, the size of the frame can be changed, and the frame can be separated or combined. . The layout analysis result can be mainly corrected.

Reference numeral 162 denotes the analysis document screen described above.
All parts on the document are displayed at the same time. The correction here is mainly a correction to the result of character recognition. The characters indicate the first candidate, but each character has a candidate character on the back. Correction candidates can be found by using language processing, and replacement of the candidates can be performed. The document analysis result of this layout reproduction is
Or, it is converted into a format such as SGML, and can be transferred to commonly used software.

FIGS. 16 and 17 are diagrams for explaining the image priority. When the layout analysis of the document manuscript is performed, each part is numbered and managed in one order system according to the position of the analysis result. This is important to maintain the sentence-figure relationship. FIG. 11 shows the numbering of character information.
Is as described above.

The Picture section also has its own management number. This is called image priority. FIG.
4 909, the image parts can be retrieved by themselves and displayed on the display 14. This is one means of quickly searching for a desired document. In this case, there is a disadvantage that the search becomes slower when displayed in the order recognition order. Thus, for the Picture part, a takeover order can be added according to the rule of larger image area as shown in FIG. 16, or an order of document layout + image size can be added as shown in FIG. In the order of larger image areas in FIG. 16, the cutout images are numbered in the order of larger vertical and horizontal sizes. A vertically long figure exists on the left side of the image 161 in FIG. 14, and this is a garbage image that can be often formed at the end of the image. In this manner, the priority of an image in which only one of the aspect ratios is extremely large is set low as shown in FIGS.

The document layout + image size order shown in FIG. 17 is an order that takes into account both the order of the sentence added by the order recognition and the image size. In the case of order recognition, the order is determined according to the position of the text and the position of the image, but small images are treated equally. As a result, a small image that does not matter has a higher priority according to its position. The above image priority is important information stored in the micro document ((2) in FIG. 1) of the present embodiment.

As described above, according to the present embodiment, the format and the creation of the layout analysis of the document image, the separation and the layer management of each part, and the provision of the reuse means utilizing the respective characteristics are provided. A method is provided.

In the above embodiment, the document image is subjected to layout analysis and separated into individual parts. However, a method may be used in which the document image itself is treated as an image and stored for each resolution. In this case, a hierarchical image as handled by FlashPix is stored in the entire image layer in (2) of FIG. The feature of this storage method is that information of the result of character recognition and information of the result of table recognition can be inserted into the management information of FlashPix from the document image. This means not only storing a document for each resolution, but also analyzing and adding the contents thereof, which is different from general image storage. Such storage means is also possible.

As described above, according to the above embodiment, the data amount of the document image can be reduced and the reusable TEXT can be used.
There are advantages such as digitization, vectorization, and compatibility with spreadsheet software. As a further major feature, a search means for a large number of stored document images is provided, so that a user can quickly find out. The search environment has an effect that it can provide a form that can be used not only in a very high-performance computer but also in a low-performance computer environment with a small amount of memory.

Even if the present invention is applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, and the like), an apparatus (for example, a copying machine, a facsimile, etc.) comprising one device Device).

An object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (or CPU) of the system or apparatus.
And MPU) read and execute the program code stored in the storage medium.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD
-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the OS (Operating System) running on the computer based on the instruction of the program code. ) May perform some or all of the actual processing, and the processing may realize the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instructions of the program code, It goes without saying that the CPU included in the function expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0073]

As described above, according to the present invention,
The data amount of the document image can be reduced, and the document data can be held in a form that makes it very easy to reuse the data.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a data configuration of a micro document according to an embodiment.

FIG. 2 is a diagram showing image data obtained by cutting out a Picture area 103 of a multi-valued image document 101.

FIG. 3 is a diagram showing image data obtained by cutting out a Picture area 104 of a multi-valued image document.

FIG. 4 is a diagram showing tile division of an image by FlashPix.

FIG. 5 is a diagram illustrating storage of image data at multiple resolutions by FlashPix.

FIG. 6 is a diagram for explaining the structure of table data.

FIG. 7 is a flowchart illustrating a procedure for generating a micro document according to the embodiment.

FIG. 8 is a diagram illustrating rectangle information obtained by a layout analysis process.

FIG. 9 is a diagram showing each attribute obtained by a layout analysis process.

FIG. 10 is a diagram illustrating a storage state of image data in Picture Layer 112.

FIG. 11 is a diagram showing a data storage state in a Text Layer 116.

FIG. 12 is a diagram showing a data structure in a micro document format according to the present embodiment.

FIG. 13 is a flowchart illustrating a procedure for reconstructing a document image from a micro document according to the embodiment.

FIG. 14 is a diagram illustrating generation of a microdocument and reconstruction of a document image according to the present embodiment.

FIG. 15 is a diagram showing an example of a display screen for correction confirmation.

FIG. 16 is a diagram for explaining an image priority order.

FIG. 17 is a diagram illustrating an image priority order.

FIG. 18 is a block diagram illustrating a configuration of an image storage device according to the present embodiment.

Claims (21)

[Claims] A reading means for optically reading a document manuscript to obtain manuscript image data; an analyzing means for extracting a character area and an image area from the manuscript image data and acquiring respective position information; Recognition means for performing character recognition on the image data of the character area obtained by the means; compression means for performing compression processing on the image data of the image area; recognition result data obtained by the recognition means; An information processing apparatus, comprising: storage means for storing the obtained compressed data and the position information obtained by the analysis means. 2. The information processing apparatus according to claim 1, wherein the compression unit uses different compression processing as needed for the image data of each image area extracted from the original image data. . 3. The storage means stores data relating to the document manuscript in a hierarchical data format.
2. The hierarchical structure according to claim 1, wherein the hierarchical structure has a first hierarchy for storing the recognition result data, a second hierarchy for storing the compressed data, and a third hierarchy for storing the position information. Information processing device. 4. The information processing apparatus according to claim 3, wherein the storage unit also stores, in the first hierarchy, data representing an image of a character area corresponding to the recognition result data. 5. The information processing apparatus according to claim 3, wherein the storage unit further includes a fourth layer for storing data obtained by compressing the entire original image data. 6. The information processing apparatus according to claim 3, further comprising display means for performing display based on data stored in the first hierarchy based on a user's instruction. 7. The information processing apparatus according to claim 3, further comprising display means for performing display based on data stored in the second hierarchy based on a user's instruction. 8. The information processing apparatus according to claim 3, further comprising a search unit configured to perform a search using the recognition result data stored in the first hierarchy. 9. The analyzing means assigns a priority order based on the arrangement or size to each of the extracted character areas and image areas, and assigns a priority to each data stored in the first hierarchy and the second hierarchy. The information processing apparatus according to claim 1, wherein a number based on the priority order is assigned. 10. A character image is generated from the recognition result data stored in the first layer, and a decompressed image is generated by expanding compressed data stored in the second layer. 2. The information processing apparatus according to claim 1, further comprising a generation unit that arranges the character image and the restored image based on the stored position information to generate a document image. 11. A reading step of optically reading a document manuscript to obtain manuscript image data; an analyzing step of extracting a character area and an image area from the manuscript image data and acquiring respective position information; A recognition step of performing character recognition on the image data of the character area obtained in the step, a compression step of performing compression processing on the image data of the image area, a recognition result data obtained in the recognition step, An information processing method, comprising: a storage step of storing obtained compressed data and position information obtained in the analysis step in a storage unit. 12. The information processing method according to claim 11, wherein in the compression step, different compression processing is used as necessary for the image data of each image area extracted from the original image data. . 13. The storage step of storing data relating to the document manuscript in a hierarchical data format, wherein the hierarchical structure stores the recognition result data in a first format.
The information processing method according to claim 11, further comprising a hierarchy, a second hierarchy for storing the compressed data, and a third hierarchy for storing the position information. 14. The information processing method according to claim 13, wherein, in the storing step, data representing an image of a character area corresponding to the recognition result data is also stored in the first hierarchy. 15. The method according to claim 1, wherein in the storing step, data obtained by compressing the entire original image data is stored as a fourth layer in the layer structure.
3. The information processing method according to 3. 16. The information processing method according to claim 13, further comprising a display step of performing a display based on data stored in the first hierarchy based on a user's instruction. 17. The information processing method according to claim 13, further comprising a display step of performing display based on data stored in the second hierarchy based on a user's instruction. 18. The information processing method according to claim 13, further comprising a search step of performing a search using the recognition result data stored in the first hierarchy. 19. The analyzing step determines a priority order for each of the extracted character areas and image areas based on the arrangement or size of the extracted character areas and image areas, and assigns a priority to each of the data stored in the first hierarchy and the second hierarchy. 12. The information processing method according to claim 11, wherein a number based on the priority is assigned. 20. A character image is generated from the recognition result data stored in the first layer, and a decompressed image is generated by expanding compressed data stored in the second layer. 12. The information processing method according to claim 11, further comprising a generating step of arranging the character image and the restored image based on the stored position information to generate a document image. 21. A storage medium storing a control program for causing a computer to execute processing for saving data relating to a document document, wherein the control program reads the document document optically and obtains document image data. Extracting a character region and an image region from the original image data, and performing a character recognition on the image data of the character region obtained in the analysis step. A code of a recognition step, a code of a compression step of performing compression processing on the image data of the image area, recognition result data obtained in the recognition step, compressed data obtained in the compression step, and And a code for a storage step of storing the obtained position information in the storage means. Storage medium.