JP2001094760A - Information processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device that can easily and quickly correct an electronic document by using an image of an original. SOLUTION: Information of an area whose correction is designated is extracted in an image of an original that is divided into areas (S1-S5), after a user corrects the extracted information, the corrected information is again converted into an image, the image of the designated area included in the original image is deleted and the image reflecting the correction is again embedded to the area from which the image is deleted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus capable of partially correcting data of a document manuscript obtained by an optical image reading apparatus such as a scanner.

[0002]

2. Description of the Related Art Conventionally, in an office environment where electronic information is handled, data of a document manuscript (electronic document) read into an information processing device such as a personal computer (PC) by an optical image reading device such as a scanner. ) Is often used. In such a case,
The user obtains the image data of the original document in the PC by reading the original document of the electronic document with a scanner or the like, and converts the image data into the electronic document using general OCR software.

FIG. 12 is a diagram showing a flow of processing when a general electronic document is created by an information processing apparatus.

As shown in FIG. 1, a layout analysis processing module 202 performs a layout analysis on document image data (original image) 201 obtained in advance by a scanner or the like, thereby converting the document image into a character area, a graphic area, Separate for each area such as a table area.

Next, the character recognition processing module 203 performs general character recognition processing on the character areas and the like divided by the layout analysis processing module 202.

[0006] The document format conversion module 204 converts the recognition result in the character recognition processing module into a generally distributed electronic document format, thereby converting it into a text document 205 as an electronic document. The converted text document includes, for example, RTF (Rich Text) supported by Word (word processing software of Microsoft Corporation).
Electronic documents such as “t Format” are known.

Here, a general O which supports RTF is used.
The flow of image processing by CR software will be outlined.

FIG. 13 shows a GUI (Graphic User Inter) in general OCR software.
FIG.

In FIG. 1, an image to be converted is displayed in a left window, and a frame representing a layout analysis result after area division is displayed. On the other hand, in the right window, the result of character recognition for the character area included in the left window is displayed. The original image resulting from this character recognition is a document image as shown in FIG. Then, by performing the RTF saving process using the obtained layout information and the character recognition result, an electronic document (electronic document) as shown in FIG. 15 according to general word processing software can be obtained.

[0010] However, in the obtained electronic document, the character recognition result is often partially incorrect or the table layout is slightly different as shown in FIG. This is because it is difficult to convert an electronic document so that it completely matches the original document.

In the example shown in FIG. 15, the character "R" is missing from the original image of FIG. 14 called "recognition sample" recognized as a character area. Is not lost, the recognition result protrudes from the document frame of the character area, and is often hidden by the shadow.For example, if the width of the character frame is slightly adjusted, a character that appears to be missing appears. Often comes. This is due to a font size of a recognition result, a shift in recognition between characters, and the like.

Although not illustrated in FIG. 15, for example, in the case of a long document, a document image may not be included in the vicinity of the last line due to a misalignment between characters or lines. If the existing character string protrudes in the same way and is hidden by shadows and cannot be seen, the lowest ruled line in the table area is not displayed, or the characters in the cells that make up the table are centered And sometimes the left-justified ones are mixed. These are all caused by erroneous recognition in each area after the area division.

[0015] If the recognition error is of the degree included in FIG. 15, the user can easily correct it by using general word processing software (document editor) while being converted into an electronic document. It is inefficient to correct individual parts while comparing the document with the original image (original image).

In this case, it is conceivable to modify the original image itself. When modifying the image itself, the user encloses a portion of the image to be modified with a rectangle using general retouching software, and encloses the rectangle. A series of operations of erasing the image data of the blank portion and pasting the original character to the erased rectangular region must be performed for each region to be corrected, which is a rather troublesome operation.

Accordingly, an object of the present invention is to provide an information processing apparatus capable of easily and quickly correcting an electronic document using a document image.

[0016]

In order to achieve the above object, an information processing apparatus according to the present invention has the following configuration.

That is, analysis means for analyzing the layout of the input document image, extraction means for extracting information on an area designated as a correction target area from the analysis results by the analysis means, and extraction means Correcting means for correcting information on the extracted correction target area,
Erasing means for erasing, from the original image, an image area represented by the information on the correction target area corrected by the correction means, and a correction target area corrected by the correction means in an area where the image is deleted by the erasing means. Image updating means for inserting an image represented by the information about

Further, for example, the erasing means may erase an image area within a rectangle surrounding characters included in the area, from among the image areas represented by the information on the correction target area.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an information processing apparatus and an information processing method according to the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram of an information processing apparatus according to the present embodiment, and a general personal computer may be employed.

In the drawing, reference numeral 22 denotes a display such as a CRT,
Reference numeral 23 denotes a keyboard as input means. A ROM 24 stores a boot program and the like. 25 is
This is a RAM for temporarily storing various processing results. Reference numeral 26 denotes a storage device such as a hard disk drive (HDD) for storing a program for realizing a document correction process described later. Reference numeral 27 denotes a communication interface for communicating with an external device via the communication line 30. Reference numeral 28 denotes a general scanner that optically reads a document including a document, a table, and the like to generate digital image data (document image) of the document. These components are connected via an internal bus 29, and the CPU 21 controls the entire information processing apparatus according to a program stored in the storage device 26.

FIG. 2 is a diagram showing the flow of processing when an original image is corrected by the information processing apparatus according to the present embodiment.

As shown in FIG. 1, in the layout analysis processing module 102, document image data (original image) 1 obtained in advance in the information processing apparatus by using a scanner or the like.
By performing a layout analysis on the document image 01, the document image is separated into regions such as a character region, a graphic region, and a table region. At this time, a general structure analysis is performed on the table area to generate table structure information indicating the structure of the table area.

Next, the character recognition processing module 103 performs general character recognition processing on characters included in the character area and the table area divided by the layout analysis processing module 102.

Next, the object extraction module 104
In the recognition result in the character recognition processing module, it is possible to extract a desired area that the user wants to correct,
Performs object extraction processing. Here, the object will be described with reference to the example of the document image shown in FIG.
The structure of the object is Title_T at the top of the figure.
ext, a text area below the title 1, a text area at the lower left of the text area 2, a picture (Pict) slope area on the right side of the text area, and a table (Tabl) at the bottom.
e) The object extraction module 104 has a configuration of an object called an area, and the object extraction module 104 extracts and corrects only a desired area from these objects.

Next, the object correction module 105
Now, for objects that the user has specified to extract,
It is a module that can modify characters and tables.

Next, the image correction module 106 deletes characters, table rule lines, and the like drawn on the original image (original image), and places an image corresponding to an object such as a corrected character on the deleted portion. By writing, a document image 107 that has been corrected is obtained.

By causing the information processing apparatus to execute each of the above modules, the user can easily correct the input original image as shown in FIG. 15 to a correct original image as shown in FIG. it can.

Next, the function of each of the above-described modules will be described in detail with reference to an example in which a user modifies a table area included in a document image. This will be described later with reference to FIG. 8).

FIG. 3 is a flowchart showing a document image correcting process by the information processing apparatus according to the present embodiment.
The processing executed by the CPU 21 according to a program stored in the storage device 26 in advance is shown.

In the figure, step S1: scanner 2
The document image data (document image) illustrated in FIG. 14 is read using the storage device 8 or from the storage device 26 (the document image may be obtained from an external device via the communication line 30). In this embodiment, a black-and-white binary image is to be corrected.

Step S2: By performing a layout analysis on the original image read into the information processing apparatus in step S1 by a general method, the original image is divided into character areas, graphic areas, table areas, and the like. In addition to the separation, table structure information is generated by performing structural analysis on the table area.

Step S3: A general character recognition process is performed on the character portion included in each of the regions divided in step S2.

Step S4: The result of the character recognition processing in step S3 is stored in the storage device 26 or the like.

Step S5: From the recognition results stored in step S4, an object specified by the user as a correction target is extracted using the keyboard 23 or the like. In the present embodiment, it is assumed that a table area included in the document image is specified.

At this time, the CPU 21 does not extract the image data of the table area to be corrected from the document image, but extracts the table structure information as a result of the structural analysis in step S2 as an object of the designated table area. And character information included in the table.

Step S6: The rectangular area (target area) corresponding to the area of the object specified by the user in step S5 can be modified by the user.

That is, based on the table structure information and the character information extracted in step S5 for the designated table area, the CPU 21 displays the display 2 as shown in FIG.
2 to display a table worksheet and allow a user to input the table worksheet. In the present embodiment, by activating general OCR software, the result of the layout analysis in step S2 is displayed in the left window as shown in FIG. 5, and the right window shown in FIG. This shows a state in which the numbers indicating the names and scores in the table worksheet have been changed.

Step S7: A rectangular area (target area) corresponding to the area of the object designated by the user in step S5 is erased from the original image (original image).

Step S8: Generate an image (corrected image) corresponding to the information corrected by the user in step S6 in the target area, and write the generated image in the target area. Thus, in the present embodiment, only the table area is modified (changed) from the content of FIG. 4 to the content of FIG.
Is obtained.

Step S9: The data of the original image in which the corrected image has been written in the target area in step S8 is stored in the storage device 26 or the like.

Here, a procedure for correcting a table area performed by the above-described document image correcting process of FIG. 3 will be described in detail.

When the layout analysis is performed in step S2, the document image is separated into a plurality of types of regions having various attributes such as a character region, a table region, and a photograph region. As a layout analysis method, a well-known analysis method based on contour tracing or an analysis method based on a labeling method may be adopted at present.

FIG. 6 is a diagram showing the structure of attribute information generated as a result of layout analysis in this embodiment.

As shown in the figure, the type of attribute generated as a result of the layout analysis in this embodiment is T
ext, Picture, Table, Line, No
Each region of the issue can be taken.

The Text area represents an area including characters in a document image, and as shown in FIG.
ext), title (TX_TITLE), caption (TX_CAPTION), and in-table character (TB_CELL) as sub-attributes.

The Picture area represents an area of a photograph or a picture in a document image, and as shown in FIG.
_PICTURE), line drawings and figures), etc. (PI_DRA
Each attribute of W) is assigned.

The Table area represents a table area in a document image, and as shown in FIG. 6, an outer frame (TB_FRAM).
E), an area surrounded by ruled lines in the table (TB_REGIO)
N), table cell (TB_CELL), character row unit (TB
_ELEMENT) (table structure information).

The Line area represents a line image in a document image. As shown in FIG. 6, each attribute of a horizontal line (HLINE), a vertical line (VLINE), a diagonal line (SLINE), and a line (Frame) that is not a closed area is shown. Granted.

The noise area is set for an area of the document image that cannot be recognized as each of the above areas.

FIG. 7 is a diagram showing a structure of attribute information generated as a result of layout analysis in the present embodiment when data is stored, and shows an address structure when stored in the RAM 25 or the storage device 26.

In the same figure, the region 1002 to the region 10
06 stores the results of the layout analysis shown in FIG.

The area 1007 includes a text area 1002
Is stored in step S3. An area 1007 in which the character recognition result is stored further includes areas of outer frame information (Area), line information (Line), character information (Character), and character code information (Code) of the character area. And
The character code information also includes information on candidate characters for character recognition.

In the area 1008, Pictur
The image area recognized as the e area is cut out from the document image and stored.

Next, from the area information performed using the results of the layout analysis shown in FIG. 7 and FIG. The case of extraction will be described as an example.

FIG. 8 is a diagram for explaining the flow of the process of extracting a correction target area in the present embodiment.

When the user designates a table area as the area to be corrected, one of the areas 1002 to 1001 of the data structure shown in FIG.
From 007, the structure information 1005 of the table area and the character recognition information 1010 included in the table area are extracted.

Next, the structure information 100 of the extracted table area
5 and the character recognition information 1010, a table worksheet 1105 is created. Table worksheet created 11
At 05, a cell for which character storage is specified by the cell information of the structure information 1005 occupies one cell of the worksheet, and the inside of the cell is included in the character code information (Code) of the character recognition information 1010. Filled with the corresponding character. For example, the character “previous” included in the table 1106 in FIG. 8 occupies the upper two lines of the worksheet.
-4, there is a blank cell to the left of “earlier period”. In general, the number of characters in the cells constituting the table worksheet is not so large, and in this case, both the Area information and the Line information in the character information 1010 indicate an area that is not much different from the character information.

1106 shows character frame information in addition to the character information to be embedded in the table worksheet 1105. This frame is based on the character information to be embedded in the table worksheet 1105. Based on the character information, the result of the character cutting process of cutting out the rectangular character space in units of one character, and the adjacent characters according to the result of the character cutting process are regarded as one frame. It shows the result of calculating a rectangle in accordance with the result of the combining process for combining.

Reference numeral 1107 denotes a state in which the number rectangle is erased based on the calculated character frame information and the user's correction (change) designation information, and the inside of the number box is cleared. (Fill with white).

Reference numeral 1108 denotes a table obtained by writing, as an image, the new characters (numeric characters) input in step S6 in the portion of the character frame cleared as in 1107. Here, as a method of converting a new character input by the user to the table worksheet 1105 into a black-and-white binary image of the same type as the original image, character rectangular information is obtained by character recognition of the input character. It is sufficient to obtain the image by performing a general font rendering process on the character area using a font corresponding to the size information included in the character rectangle information.

As described above, in the information processing apparatus according to the present embodiment, when a document image is partially corrected, the original image is easily corrected by embedding a new image in the image portion to be corrected of the document image.・ Can be changed.

Although the present embodiment has been described with reference to the modification of the table area, the text area may be modified. In this case, when the character string is shifted, the characters to be corrected may be moved one by one. Also, Pictu
When the re area is to be corrected, the image stored in the image storage area 1008 in FIG. 7 may be corrected using image retouching software and embedded in the corresponding position of the document image. 1> In the above embodiment, an example in which only the character information included in the table area is changed has been described. However, in the present modification, a case in which the table structure is further changed will be described.

FIG. 9 is a diagram for explaining a case where, for a table area to be corrected, character information and a table structure included in the table area are changed.

In FIG. 9, reference numeral 1201 denotes an image of a table area included in a document image before correction. For example, in this table
When adding a column, one blank line is inserted as shown at 1202, and desired characters are inserted into the blank line as shown at 1203. The input operation of a new character is performed by the user on the table worksheet.

Adding a row using a table worksheet is preferable because it is easy for the user to operate, but it involves a change in the table structure, and it is necessary to reflect the change in the original table area information.

Therefore, in this modification, when the range of the original document image is not affected by other objects even if the range of the original document image is expanded, the table area itself is temporarily deleted and the frame of the table is redrawn. This is a process of writing a ruled line in an image.

The writing process of the ruled line may be a method of writing a new line or a method of writing by cutting out a line from the table area of the original document image. In the latter case, the new line may be copied using the line of the previous line.

Next, when the ruled lines of the table are drawn, characters are inserted into the cells constituting the table. When inserting a character into a cell, for a character portion that has not been corrected by the user and is not related to the line correction, the character portion is cut out from the original document image and a new character portion is cut out. Insert into the corresponding cell in the table. on the other hand,
Although the character is not corrected, the line is corrected (1202 in FIG. 9).
For example, as for the characters of the original image located in the portion where the line has been added), the character “total” in 1202 has been moved down compared to before the correction. Instead of pasting the included characters, paste them at the position lowered by the added character line. And
A new character (number) input by the user is written in the added line.

FIG. 10 is a flowchart showing a process of correcting a table area according to the first modification of the present embodiment.

In the figure, steps S21 to S23: When a correction target in a document image is designated by the user (step S21), information on the table area is extracted in the same manner as step S5 in FIG. 3 (step S22). ), A table worksheet is generated and displayed based on the extracted information as shown by 1201 in FIG. 9 (step S23).

Step S24: For the displayed table worksheet, the user inputs a correction (including a correction of characters) accompanying a change in the table structure as shown in, for example, 1202 and 1203 in FIG. At this time, the state of the internal processing flag is changed according to whether or not a correction accompanied by a change in the table structure and a character correction have been input for use in a subsequent processing step. At this time, the presence or absence of character correction
It is stored in units of one character according to the internal processing flag.

Step S25: By detecting the state of the internal processing flag stored in step S24, it is determined whether or not the displayed table worksheet has been modified so as to change the table structure. , YES
When the table structure has been changed, the process proceeds to step S30, and when NO (no change in the table structure), the process proceeds to step S2.
Proceed to 6.

Step S26: If the determination in step S25 is NO, the state of the internal processing flag stored in step S24 is detected to determine whether or not the character has been corrected in the displayed table worksheet. Judge.

Step S27: If it is determined in step S26 that there is no character correction, the process is terminated.

Steps S28 and S29: If it is determined in step S26 that the character is corrected, the state of the internal processing flag stored in step S24 is detected, and the corrected character portion is set to 1
Correction is performed in units of characters, and in step S29, the newly input character is stored in text region 1002 (FIG. 7) in units of one character.

Steps S30 and S31: If the determination in step S25 is YES, the entire currently extracted table area is temporarily erased (step S30), and the table framework is changed to a new line or the original document is written. The image is redrawn by cutting out a line from the table area of the image (step S31). Steps S32 and S33: The character portions that have not been subjected to character correction by the user and are not related to the line correction are: The original document image is cut out and inserted (step S32), and the corrected character is inserted in units of one character into the character portion where the character has been corrected by the user.

Step S34: The front worksheet is converted into an image and embedded in a corresponding portion of the original document image to obtain a corrected document image.

As described above, in this modification, the character information and the table structure of the table area included in the original image can be reflected.

<Modification 2 of Embodiment> In Modification 1, when displaying the ruled lines of the table in steps S30 and S31, even if the range of the original document image is expanded, it does not interfere with other objects. However, in the present modification, a case where the object overlaps with another object will be described.

FIG. 11 is a flowchart showing a process of correcting a table area according to the second modification of the present embodiment.
10 is a flowchart in which portions different from those of the first modification shown in FIG.

In the figure, when it is determined in step S25 that the table structure has been modified, it is determined in step S35 whether or not it overlaps with another object. In the case of YES, the cut-out table area is reduced. , So that the original position is obtained even after the expansion
The process proceeds to step S30 if NO. In this determination, by comparing the coordinate information or the side information indicating the area included in the area information after the table correction with the coordinate information or the side information indicating the area included in the area information of another object, What is necessary is just to judge whether there is an overlapping part. As a method of reducing the cut-out table region, for example, a general image reduction process such as a linear thinning process or an interpolation thinning process may be adopted.

In this modification, when the image is reduced in step S36, when the character without correction is copied in step S32, the character in the reduced image is used. For the newly added character, the character at the point corresponding to the reduction ratio of the image in step S36 is used.

According to this modification, it is possible to prevent the table area to be corrected from overlapping another area.

[0085]

[Other Embodiments] Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus (for example, a copying machine) Machine, facsimile machine, etc.).

Further, an object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and a computer (a computer) of the system or the apparatus. It is needless to say that the present invention can also be achieved by a CPU or an MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. Also,
When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also the operating system (OS) running on the computer based on the instructions of the program code.
It goes without saying that the present invention includes a case in which the functions of the above-described embodiments are implemented by performing part or all of the actual processing.

Further, after the program code read from the storage medium is written into the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, the program code is read based on the instruction of the program code. Needless to say, a case is also included in which a CPU or the like provided in the function expansion card or function expansion unit performs part or all of actual processing, and the processing realizes the functions of the above-described embodiments.

[0088]

As described above, according to the present invention,
An information processing apparatus capable of easily and quickly correcting an electronic document using a document image is provided.

[Brief description of the drawings]

FIG. 1 is a block diagram of an information processing apparatus according to an embodiment.

FIG. 2 is a diagram showing a flow of processing when a document image is corrected by the information processing apparatus according to the embodiment.

FIG. 3 is a flowchart illustrating a document image correction process performed by the information processing apparatus according to the embodiment.

FIG. 4 is a diagram illustrating an example of a table worksheet created based on character information and structure information of a table area extracted from a document image.

FIG. 5 is a view showing a GUI of OCR software on which a result of layout analysis and a table worksheet are displayed.

FIG. 6 is a diagram illustrating a structure of attribute information generated as a result of layout analysis according to the embodiment.

FIG. 7 is a diagram showing a structure of attribute information generated as a result of layout analysis in the embodiment when data is stored.

FIG. 8 is a diagram illustrating a flow of a process of extracting a correction target area according to the present embodiment.

FIG. 9 is a diagram illustrating a case where character information and a table structure included in a table area to be modified are changed.

FIG. 10 is a flowchart illustrating a correction process of a table area according to a first modification of the embodiment.

11 is a flowchart in which a part different from the flowchart of the first modification shown in FIG. 10 is extracted from the flowchart showing the correction processing of the table area in the second modification of the embodiment.

FIG. 12 is a diagram showing a flow of processing when a general electronic document is created by the information processing apparatus.

FIG. 13: GUI in general OCR software
FIG.

FIG. 14 is a diagram illustrating an example of a document image (document image).

FIG. 15 is a diagram exemplifying a case where the document image illustrated in FIG. 14 is converted into an electronic document according to general word processing software;

Claims (7)

[Claims] An analyzing means for analyzing a layout of an input document image; an extracting means for extracting information on an area designated as a correction target area from an analysis result by the analyzing means; Correcting means for correcting the extracted information on the correction target area; erasing means for erasing, from the original image, an image area represented by the information on the correction target area corrected by the correcting means; And an image updating unit for inserting an image represented by the information on the correction target region corrected by the correction unit into the region where is deleted. 2. The information according to claim 1, wherein the erasing unit erases an image area within a rectangle surrounding a character included in the area included in the information about the area to be corrected. Processing equipment. 3. When the area specified as the area to be corrected is a table area, the correction means can correct information on the table area by a man-machine interface in a table worksheet format. The information processing apparatus according to claim 1. 4. The method according to claim 1, wherein when the area specified as the area to be corrected is a character area, the correction means can correct information on the character area by a text editor type man-machine interface. The information processing apparatus according to claim 1, wherein 5. The image processing apparatus according to claim 1, wherein when the area specified as the area to be corrected is a picture area, the correcting means can correct information on the picture area by a man-machine interface in an image retouching format. The information processing apparatus according to claim 1, wherein 6. When the structure of the table area is changed by the correction information input through a man-machine interface in the form of a table worksheet, the image updating means may include the correction means. 6. The information processing apparatus according to claim 5, wherein a part of the image area other than the correction target area included in the document image is moved according to the correction information input in step (a). 7. The image updating means, wherein the structure of the table area is changed by correction information input through a man-machine interface in the form of a table worksheet. 2. The method according to claim 1, wherein when it is determined that the correction target area is updated according to the correction information input and interferes with other surrounding areas, the correction target area is reduced prior to the update. An information processing apparatus according to claim 1.