JP4974367B2 - Region dividing method and apparatus, and program - Google Patents

Description

  The present invention relates to an area dividing method and apparatus for identifying and extracting areas such as a character area, a table area, and a photograph area from image data of a document, and a computer program for executing the method.

  When image data generated by reading a document is processed and reused as text data, it is important to acquire layout information indicating the existence range of sentences written in characters. For example, when character recognition of a document is performed, the document is read as image data with an image scanner or the like, and layout information is obtained by dividing the area into character areas and graphic areas. The character area is converted into text data by performing character recognition processing. In this case, the area division is used as a pre-process for the character recognition process. For area segmentation, the processing method for areas with different attributes in the document image is optimal not only for character recognition pre-processing, but also when document image data is filed on an optical disk or transmitted by facsimile. It is an important technology for making it easier.

  Various techniques such as a method using projection (Patent Document 1) and a method using black pixel integration (Patent Document 2) have been proposed as techniques for automatically dividing an area. However, with these automatic area dividing techniques, it is very difficult to accurately divide a complex layout or an irregularly formatted document.

  In addition, as a technique for performing region division using manual information, the user selects an arbitrary region in the document image with respect to the region dividing device, and points are set inside each selected region. There is an area dividing method and apparatus that designates and performs area division of a document image using the position of the designated point as evidence in an area dividing apparatus (Patent Document 3). According to this technique, it is possible to improve the accuracy of region division by actively using the user's region designation information.

However, in the technique described in Patent Document 3, since region identification is performed by designating each point in the region to be divided in the document image, almost all regions existing in the document image are not designated. and, it can not be exact area identification. That is, even if there is only one area that needs to be identified among the 10 areas, accurate identification cannot be performed unless the remaining 9 areas that do not need to be identified are selected. Therefore, if and that correct the results of automatic segmentation class Osamu, not suitable for simple use in the case requiring identification of one or two areas. In addition, it is difficult to specify an accurate point (position specification) on a small-sized screen such as a display unit of a PDA (personal digital assistant) or an operation panel of an MFP (multifunction printer).

JP-A-5-266250 JP-A-5-274472 JP-A-9-128479

  The present invention has been made to solve such a problem. The purpose of the present invention is to accurately identify a region by roughly specifying a region to be identified when performing region division using manual input. Is to make it possible.

The region dividing method of the present invention includes a step of displaying a document image, a step of acquiring position information of a user-specified region that is specified by a user and is a part of a region to be divided of the displayed document image, based on the document image data in the user-specified region, the possess and performing extraction of the divided region of interest, the step of performing the extract unloading includes a feature extraction step of extracting internal information of the user-specified region, the based on the extracted information, the user-specified region character candidate, that the table candidates, and organic and performing an attribute classification step of classifying the figure or photograph candidate area extraction processing according to the classified each candidate which is a region dividing method according to claim.
Program of the present invention, the computer is a program for executing the steps of the area dividing method of the present invention.
The area dividing apparatus of the present invention is an area dividing apparatus having a computer in which the program of the present invention is installed.

According to the present invention, when performing region division using manual input, it is possible to accurately extract a region by specifying a rough position for a region that needs to be identified. Further, by performing attribute discrimination before region extraction and using an optimum extraction method according to the discriminated attribute (character, table, figure or photograph), an optimum extraction result can be obtained.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a schematic block diagram showing the configuration of the area dividing apparatus according to the first embodiment of the present invention. This area dividing device is configured to execute the area dividing process by digital processing such as a microprocessor.

  The area dividing device includes a scanner 1 that optically reads a document and converts it into document image data that is an electrical signal, a CPU 2 that controls the entire area dividing device and performs various processes, and area-divided document image data. , A display 4 for displaying document image data input from the scanner 1 and area-divided document image data, a printing apparatus 5 for printing the area-divided document image data, a mouse, A pointing device 6 such as a tablet and a pen, a program storage ROM / RAM 7 that stores various programs used when the CPU 2 operates, and a work area RAM 8 that temporarily stores data and programs when the CPU 2 operates. CD-ROM / FD drive for reading programs from CD-ROMs and FDs And a blanking 9, which are connected to the bus 10.

FIG. 2 is a flowchart showing the operation of the area dividing apparatus of this embodiment.
First, image input is performed in step S1. Here, the sample document 11 shown in FIG. The document image data of the sample document 11 output from the scanner 1 is sent to the display 4 in step S2, and the image of the sample document 11 is displayed.

  Next, in step S <b> 3, the user inputs the designated area by operating the pointing device 6 while viewing the image of the sample document 11 displayed on the display 4. Here, as shown in FIG. 4, a part of the table 11a is surrounded by a circle 12 to roughly specify the position of the table 11a. The area designated by the user (hereinafter referred to as the user designated area) is displayed on the display 4 in step S4.

  Next, in step S <b> 5, the CPU 2 converts the user specified area into an area that can be easily handled by the CPU 2. For example, the user may handle the rectangular data based on the maximum and minimum coordinates of the area surrounded by the circle 12, or cut and handle the user-specified area itself.

  Next, in step S6, information inside the user designated area is extracted. For example, the image data inside the user-specified area is binarized, and a projection histogram is taken on both the X and Y axes (see Japanese Patent Publication No. 7-95335). The continuous value close to 0 in the projection histogram inside the user-specified area can be regarded as information between characters and between lines. In step S7, the information is divided into regions. In this example, when a continuous value of 0 in the projection histogram continues, it is possible to determine that the vicinity is an area break. (For character areas).

  In step S8, the result of area division is displayed on the display 4. Here, the table 11a of the sample document 11 is extracted as a table area, and a frame 13 indicating the table area is displayed outside the table area. If the extracted table area does not match the table area 11a of the sample document 11, the user corrects it using the pointing device 6 in step S9. The region segmentation result corrected as necessary in this manner is output in step S10 for use in a character recognition device or the like.

  As described above, according to the region dividing apparatus of the present embodiment, the user does not specify the entire region to be extracted accurately while viewing the document image on the display 4, and only specifies roughly a part of the region. Therefore, the entire area can be extracted with high accuracy. Also, coupled with its high speed, interactive correction is possible.

[Second Embodiment]
FIG. 6 is a flowchart showing the operation of the area dividing apparatus according to the second embodiment of the present invention. In this figure, steps that are the same as or correspond to those in FIG. The block diagram of the schematic configuration of the area dividing apparatus according to the present embodiment and third to fifth embodiments described later is the same as that of the first embodiment (FIG. 1). Further, in the present embodiment and the third to fifth embodiments described later, the document read from the scanner 1 is also the same sample document 11 as in the first embodiment, and the user specified area is the same as in the first embodiment. Round 12.

  In the present embodiment, before region extraction, attribute determination (character, table, figure or photograph) of the user-specified region is performed in step S11. Attribute discrimination can be handled by known techniques such as Japanese Patent Nos. 3344774 and 3215163, which are patents of the present applicant.

FIG. 7 is a flowchart illustrating an example of attribute determination processing.
First, in step S21, image data in the user designated area is input, and then in step S22, the image data is binarized. However, when the image data in the user-specified area is binary, this binarization process is not performed.

  Next, in step S23, black pixel connected components are extracted, and in step S24, white pixel connected components are extracted. Then, a black pixel ruled rectangle is extracted in step S25 using the extraction result in step S23, and a white pixel ruled rectangle is extracted in step S26 using the extraction result in step S24. Here, the black pixel ruled rectangle is extracted only by long black pixel connection in each of the horizontal direction and the vertical direction, and the white pixel ruled rectangle is extracted only by long white pixel connection in each of the horizontal direction and the vertical direction. To do. Finally, in step S27, it is determined whether the table is a rule base or a feature amount base.

8 to 10 are diagrams for explaining an example of discrimination based on the rule base.
As shown in FIG. 8, a part of a table composed of four horizontal ruled lines 21 and four vertical ruled lines 22 exists in the user-specified area, and nine (3 horizontal x 3 vertical) partitioned by the ruled lines. ), The characters “XXX”, “YYY”, and “ZZZ” are entered in the upper cell.

  In this way, when the table is drawn with a white background and black characters, the ruled line is a black pixel, so as shown in FIG. A ruled line rectangle 23 and a vertical black pixel ruled line rectangle 24 are detected.

  However, since the white pixel ruled rectangle can only exist between the ruled lines in the horizontal direction and the vertical direction, there is no long white pixel connection and tends to be thick. FIG. 10A shows the white pixel ruled line rectangle obtained in step S26 from the table of FIG. 8 by inverting it to black for easy understanding. Since the vertical ruled line is too thick, it cannot be obtained due to the limitation of the aspect ratio of the rectangle in this case. In FIG. 8, since there are characters inside the cell, if the number of characters is large and there is no sufficient margin on the left and right of the character, for example, as shown in FIG. 10B, the character “ When the lengths L1 and L2 of the left and right margins of “XXX” are less than a threshold value that considers white pixel connection to be long (the same applies to “YYY” and “ZZZ”), the detected white pixel ruled rectangle is shown in FIG. 10C. It will be like that.

  As described above, after performing the attribute determination in step S11, in step S12, creation of a region range (region extraction process) according to the determined attribute is executed.

  According to this embodiment, it is possible to obtain an optimum extraction result by performing attribute discrimination before region extraction and using an optimum extraction method according to the discriminated attribute (character, table, figure or photograph). it can.

[Third Embodiment]
FIG. 11 is a flowchart showing the operation of the area dividing apparatus according to the third embodiment of the present invention. In this figure, the same or corresponding steps as those in FIG. 6 (second embodiment) are denoted by the same reference numerals as those in FIG.

  In the present embodiment, the internal characteristics of the user-specified area are classified in step S13, and an area range is created (area extraction process) in step S14 according to the classification result. In other words, the classification of the user-specified area is not a classification according to the purpose of use of the subsequent user, but a classification that is useful for the successful segmentation of the area is performed, and the optimal area segmentation method inferred from internal information is selected To do.

  This embodiment is effective when, for example, there are many long horizontal lines but they are not table regions. An example is shown in FIG. In this example, a long horizontal line 21 exists under the characters “◯”, “Δ”, and “□”. If the user encloses a part of the area shown in this figure with a circle 22, in the case of the second embodiment, it may be determined as a “table area” by the attribute determination means in step S11. When the region range creation method for the table region in step S13 is as in the fourth embodiment described later, a range narrower than the region considered by the user is output as a result.

  Considering that such a case may occur, there is also a merit in outputting attributes that are effective for the region extracting means, instead of outputting attributes that the user uses in subsequent processing by the attribute determining means. is there. In the example as shown in FIG. 12, it is better to output the result of which one of the plurality of region extraction methods located after the output is more appropriate than outputting the attribute of the table simply because there are many long horizontal lines. The result is more user-friendly. The true attribute can be determined again after the area is determined.

  According to the present embodiment, since the optimum region extraction method is selected according to the classification result of the internal characteristics of the user-specified region, accurate region extraction can be performed even in the case of FIG.

[Fourth Embodiment]
FIG. 13 is a flowchart showing the operation of the area dividing apparatus according to the fourth embodiment of the present invention. In this figure, the same or corresponding steps as those in FIG. 6 (second embodiment) are denoted by the same reference numerals as those in FIG.

  In this embodiment, the attribute of the user designated area is determined in step S15, and when it is determined to be a table, the table area is extracted in step S16. Here, in step S15, unlike step S11 of the second embodiment, only the discrimination of the table and the discrimination of the background, white or black are performed.

FIG. 14 shows a flowchart of a specific configuration example of step S16.
In step S31, the document image data is read into the work area RAM 8, and in step S32, ruled lines are extracted from the entire surface of the document image with the foreground color. This ruled line extraction method uses existing methods such as creating a ruled line candidate by using a Hough transform, or by using only pixels that are longer than a threshold among consecutive pixels (called a run) to obtain a connected component. You can use.

  Of the ruled line rectangles obtained on the entire screen in this way, those included in the user designated area are extracted in step S33. FIG. 15A is an example of the extraction result of step S33, and ruled lines 32 (two horizontal ruled lines and two vertical ruled lines are displayed as solid lines) included in the user-specified area 31 determined by the freehand curve are extracted.

  Next, in step S34, a ruled line that intersects or contacts the ruled line included in the user-specified area (extracted in step S33) is extracted. FIG. 15B shows the result of extracting a ruled line (broken line 33 in FIG. 15A) that intersects or is in contact with the ruled line 32 extracted in step S33.

  Finally, in step S35, the table area is determined from the maximum and minimum values of the ruled line coordinates extracted in step S34. As a result, a table region indicated by a solid line in FIG. 15B is extracted.

  As described above, according to this embodiment, when the user-specified area is a table area, a highly accurate extraction result can be obtained. In FIG. 14, in order to simplify the process, the ruled lines on the entire screen are first extracted. However, a method of searching from the vicinity of the found ruled lines may be used. There is no significant impact on

[Fifth Embodiment]
FIG. 16 is a diagram for explaining the operation of the area dividing apparatus according to the fifth embodiment of the present invention. The basic operation flow of the region dividing apparatus of this embodiment is the same as that in FIG. 6 (second embodiment), and the region range when the classification result (attribute determination in step S11) is determined to be a table. It is an invention about how to obtain (corresponding to step S12).

  Here, a connected component in the user specified area is obtained, and the connected component is in contact with the user specified range. Then, the search range is expanded with respect to the connected component that is in contact with the position of the designated area, and if the pixel continues in the expanded range, the connected component is grown (enlarged). As this process is repeated, the pixels connected to the pixels within the designated area range become clear, and the area surrounding them is extracted.

  In the case of FIG. 16, a rectangular area 42 (displayed by a broken line) surrounding the user-specified area 41 determined by the freehand curve is simply set as the user-specified area. Further, the black pixels in the region 42 (a part of each of the three horizontal ruled lines, a part of one vertical ruled line, and eight o) are considered as pixels included in the user-specified area. Then, the connected component including these pixels is operated to finally extract the connected component rectangle.

  FIG. 17 is a diagram illustrating an example of a rectangular extraction method using a connected component of pixels. As shown in A of the figure, black runs (shown as black rectangles) are extracted in the main scanning direction, and as shown in B, the connected components of the black runs are integrated to create rectangular areas 51 to 53. . Next, as shown in C, a rectangle is grown also in the sub-scanning direction, and rectangular regions 54 and 55 are created. At this time, even if the rectangles are overlapped, those in which the runs are not connected (in this case, the rectangular region 53) are not integrated and are treated separately.

  Here, it is assumed that the background is simply a white background and a black foreground, but a multi-valued image and a lightness inverted image may also be detected. In that case, if it is a multi-valued image, binarization is performed first to adjust the conditions. Next, a step of discriminating between the foreground and the background is performed according to the processing flow shown in FIG. This can be dealt with by assigning the obtained foreground color and background color to white or black, respectively, and performing the processing of this embodiment described above.

  What can be said in common with the first to fourth embodiments described above is that the region intended by the user can be extracted (cut out) at high speed, and an easily distinguishable table (simple configuration table). ), It is possible to extract even roughly. For a table that is difficult to be automatically identified, a more accurate extraction result can be obtained by making it closer to the area actually required when designated by the user. In addition, the relationship between the difficulty level of this table and the trade-off between automatic area extraction has the advantage that it is easy for the user to learn and use.

It is a schematic block diagram which shows the structural example of the area division | segmentation apparatus of the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the area dividing apparatus of the 1st Embodiment of this invention. It is a figure which shows the sample document of the 1st Embodiment of this invention. It is a figure which shows an example of the aspect which designates a part of table area | region of the sample document of the 1st Embodiment of this invention. It is a figure which shows the table area extracted from the sample document of the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the area | region division apparatus of the 2nd Embodiment of this invention. It is a flowchart which shows an example of the attribute discrimination | determination process in the 2nd Embodiment of this invention. It is a figure which shows an example of the table | surface in the 2nd Embodiment of this invention. It is a figure which shows the black pixel ruled line rectangle extracted from the table | surface of FIG. 8 by the attribute discrimination | determination process of FIG. It is a figure which shows the white pixel ruled line rectangle extracted from the table | surface of FIG. 8 by the attribute discrimination | determination process of FIG. It is a flowchart which shows operation | movement of the area dividing apparatus of the 3rd Embodiment of this invention. It is a figure which shows an area | region where use of the area | region dividing device of the 3rd Embodiment of this invention is suitable. It is a flowchart which shows operation | movement of the area | region dividing device of the 4th Embodiment of this invention. It is a flowchart which shows the specific structural example of the extraction process of the table | surface in FIG. It is a figure which shows the ruled line and table | surface extracted by the extraction process of FIG. It is a figure for demonstrating operation | movement of the area dividing apparatus of the 5th Embodiment of this invention. It is a figure which shows an example of the rectangle extraction method using the connection component of a pixel.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Scanner, 2 ... CPU, 4 ... Display, 6 ... Pointing device.

Claims (5)

A step of displaying a document image, a step of obtaining position information of a user-designated area that is a part of a division target area of the displayed document image designated by a user, and document image data in the user-designated area based on, it possesses and performing extraction of the divided region of interest,
The extracting step includes a feature extracting step of extracting information inside the user-specified area, and classifying the user-specified area into character candidates, table candidates, figures, or photo candidates based on the extracted information. area dividing method which is characterized in that chromatic and attribute classification step, and performing region extraction processing in accordance with each candidate classified. The area dividing method according to claim 1, wherein:
The feature extraction step includes a rectangle extraction step for extracting a connected component rectangle of black pixels or white pixels from a binary image, and a ruled line extraction step for extracting ruled lines. The attribute classification step is based on these extraction results. predetermined feature amount closest or in the area dividing method according to claim Rukoto obtain classification results of the attribute to which category space. The area dividing method according to claim 1, wherein:
The step of performing the region extraction process is the largest connected rectangle by growing from the connected components of the pixels included in the user-specified region to a position where the connection is lost with respect to the region that is a table candidate in the attribute classification step. A region dividing method characterized in that the region is a user-specified table region . The program for making a computer perform each process of the area | region division method described in any one of Claims 1-3 . An area dividing apparatus having a computer in which the program according to claim 4 is installed .

Images