JP2002236921A - Document image recognition method, document image recognition device and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To specify a character color without performing an OCR by distinguishing an area of a color document image. SOLUTION: A document image is input as a color digital image (step S1). A background color of the document image is specified (step S2). Picture elements other than a background area are extracted from the document image by using the background color, concatenated components are generated by integrating the picture elements, and the concatenated components are sorted and distinguished by shape characteristics and color characteristics into areas such as characters, ruled lines, charts, and photographs. Character rectangle data is acquired to be used as a distinguished character area (step S4). A representative color is calculated on the basis of a picture element value of picture elements left after removal of picture elements equivalent to the background color in the character rectangle, and the representative color is determined as the character color of the character rectangle.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document image recognition method, a document image recognition device, and a computer-readable recording medium, and more particularly, to a technique for identifying a color of a color document image by identifying a region. It is applied to the extraction of a title / keyword or the generation of a summary sentence using the information on the obtained character color.

[0002]

2. Description of the Related Art Japanese Patent Laid-Open No.
In Japanese Patent Application Laid-Open No. 121733 (document image processing apparatus), after a color image is area-identified, small areas belonging to a character area are classified by color for each chunk of the area. This is the C,
A criterion is determined based on the average value of the densities of the M, Y, and K colors, and the character color is determined based on whether the density is higher or lower than the criterion. However, since the character color determination criterion is a criterion relating to density, it is difficult to adapt to a character area (green ground + black character) where the density difference between the background and the character is small.

Japanese Patent Laid-Open Publication No. Hei 8-123901 (character extraction device and character recognition device using the device) once binarizes a color image and extracts characters from the binary image. Therefore, no color information is reflected in character extraction. As described above, it is difficult to accurately extract a character area in which the density difference between the background and the character is small, and it is not possible to cope with a character area written with white characters on a black background.

[0004]

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems. 1. Specify the character color without OCR; 2. Correspond to characters with a small density difference between the background and the characters; 3. Specify white characters. 4. To provide a method of presenting a character color identification result that is easy for the user to understand; 5. Extract document titles, keywords, etc. without OCR; The purpose of this is to efficiently create a summary by minimizing the OCR. Regarding the above 2 and 3, any character rectangle can be extracted from a color image by using the method described in Japanese Patent Application No. 2000-032298, and the problem can be solved by giving a character color to it. Can be.

[0005]

According to a first aspect of the present invention, a document image is input as a color digital image, color information is extracted from the document image, a background color of the document image is specified, and the background color is determined. Then, pixels other than the background area are extracted from the document image as small chunks, and the extracted small chunks are integrated to generate a connected component. What is claimed is: 1. A document image recognition method for classifying an image into a region such as a figure / photograph, wherein a character color of a character region is specified from a color digital image for a character region which is a region classified as the character. It is.

According to a second aspect of the present invention, in the first aspect of the present invention, when the character color is specified, when the background color is present in the character rectangle classified as the character area, the background color in the character rectangle is determined. Assuming that a pixel corresponding to a pixel value substantially equal to 0 is 0 and other pixels are 1, a representative color is determined from a set of pixels having a value of 1, and the determined representative color is set as the character color of the character rectangle. It is characterized by the following.

According to a third aspect of the present invention, in the first aspect of the present invention, when a character color is specified, a character color is given to each of the classified character rectangles, and a distribution chart for each character color is created. A color that satisfies a specific condition based on the distribution map is set as a representative character color of the document.

According to a fourth aspect of the present invention, in the first aspect of the present invention, when presenting an area such as a character / ruled line / figure / photo as a rectangle in a document to a user, the color of the rectangle representing the character area Is the character color specified by the first aspect.

A fifth aspect of the present invention is characterized in that, in the third aspect of the present invention, if there is a character string described in a color other than the representative character color of the document, the character string is used as a document title or a keyword. It is what it was.

According to a sixth aspect of the present invention, in the third aspect of the invention, if there is a character string described in a color other than the representative character color of the document, the entire area including the character string is subjected to OCR,
It is characterized in that a summary sentence of a document is created from character codes obtained from a plurality of R regions.

According to a seventh aspect of the present invention, there is provided image input means for inputting a document image as a color digital image, extracting color information from the document image, specifying a background color of the document image, and using the background color. Pixels other than the background area are extracted from the document image as small blocks, and the extracted small blocks are integrated to generate a connected component. What is claimed is: 1. A document image recognizing device having an area classifying means for classifying an image into a region such as a photograph, wherein a character color for specifying a character color of a character region from a color digital image for a character region which is a region classified as the character It is characterized by having a specifying means.

According to an eighth aspect of the present invention, in the invention of the seventh aspect, when specifying the character color, when the background color is present in the character rectangle classified as the character area, the background color in the character rectangle is determined. Assuming that a pixel corresponding to a pixel value substantially equal to 0 is 0 and other pixels are 1, a representative color is determined from a set of pixels having a value of 1, and the determined representative color is set as the character color of the character rectangle. It is characterized by the following.

According to a ninth aspect of the present invention, in the invention of the seventh aspect, when a character color is specified, a character color is given to each of the classified character rectangles, and a distribution chart for each character color is created. A color that satisfies a specific condition based on the distribution map is set as a representative character color of the document.

According to a tenth aspect of the present invention, in the invention of the seventh aspect, when an area such as a character / ruled line / figure / photo is presented to a user as a rectangle in a document, the color of the rectangle representing the character area Is the character color specified by claim 7.

According to an eleventh aspect of the present invention, in the ninth aspect of the present invention, if there is a character string described in a color other than the representative character color of the document, the character string is a title of the document or a title / keyword which is a keyword. It is characterized by having extraction means.

According to a twelfth aspect of the present invention, in the ninth aspect of the invention, if there is a character string described in a color other than the representative character color of the document, the OC which performs OCR on the entire area including the character string is provided.
R means and a summary sentence creating means for creating a summary sentence of a document from character codes obtained from a plurality of OCR areas.

According to a thirteenth aspect of the present invention, there is provided an image input step of inputting a document image as a color digital image, extracting a color information from the document image, specifying a background color of the document image, and using the background color. Pixels other than the background area are extracted from the document image as small blocks, and the extracted small blocks are integrated to generate a connected component. A computer-readable recording medium that records a program that causes a computer to execute a document image recognition method including an area classification step of classifying an area such as a photograph, and for a character area that is an area classified as the character, A computer-readable recording medium storing a program for causing a computer to execute a character color specifying step of specifying a character color of a character area from a color image

According to a fourteenth aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for causing a computer to execute the document image recognition method according to the second aspect, wherein the character area is specified when a character color is specified. When there is a background color in the character rectangle classified as, the pixel corresponding to the pixel value substantially the same as the background color in the character rectangle is set to 0, the other pixels are set to 1, and the pixels having the value of 1 are set. A computer-readable recording medium storing a program for causing a computer to execute a step of determining a representative color from a set and setting the determined representative color as the character color of the character rectangle.

According to a fifteenth aspect of the present invention, there is provided a computer-readable recording medium storing a program for causing a computer to execute the document image recognizing method according to the third aspect. Giving a character color to each of the character rectangles, creating a distribution map for each character color, and causing the computer to execute a step of setting a color satisfying a specific condition as a representative character color of the document based on the distribution map. It is a computer-readable recording medium on which a program is recorded.

According to a sixteenth aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for causing a computer to execute the document image recognition method according to the fourth aspect, wherein an area such as a character / ruled line / figure / photo is recorded. A computer-readable recording medium storing a program for causing a computer to execute a step of setting the color of the rectangle representing the character area to the character color specified by claim 1 when presenting the rectangle to the user as a rectangle in a document. It is.

According to a seventeenth aspect of the present invention, there is provided a computer-readable recording medium storing a program for causing a computer to execute the document image recognition method according to the fifth aspect, wherein the recording medium is described in a color other than the representative character color of the document. If there is such a character string, it is a computer-readable recording medium that records a program for causing a computer to execute a title / keyword extraction step using the character string as a document title or a keyword.

According to an eighteenth aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for causing a computer to execute the document image recognition method according to the sixth aspect, wherein the medium is described in a color other than the representative character color of the document. If there is a character string, an OCR that OCRs the entire area including the character string
A computer-readable recording medium in which a program for causing a computer to execute a step and a step of creating a digest of a document from character codes obtained from a plurality of OCR regions is recorded. [Detailed description of the invention]

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention relates to Japanese Patent Application No. 2000-032.
As an application of the method described in Japanese Patent Publication No. 298, a character color in a color document image is automatically specified. The present invention has an advantage that OCR is not required because the character color can be directly specified from the color image. For relatively large characters, the character color can be reproduced almost in character units, and for small characters, the character color can be specified in line or area units. In particular, it can sufficiently cope with white characters.

(Embodiment) FIG. 1 is a diagram showing an example of the configuration of a document image recognition apparatus to which the present invention is applied. In FIG. 1, 1 is an image input means, 2 is an area classification means, and 3 is a character Color identification means, 4 is a title / keyword extraction means, 5 is a title / keyword DB, 6 is an OCR means, 7 is a summary sentence creation means, and 8 is a summary sentence DB. First, a color document image is input from the image input unit 1, the background color of the entire document is specified by the area classification unit 2, area identification processing is performed on the color document image, and information on a character rectangle is obtained. Hereinafter, a character color is specified by the character color specifying means 3 for each of the obtained character rectangles.

FIG. 2 is a flowchart for explaining an example of processing for specifying a character color in the document image recognition apparatus to which the present invention is applied. First, a color document image is inputted from the image input means 1 (step S1), and the area classifying means 2
In step S2, the background color of the entire document is specified.
An area identification process is performed on the color document image (step S3), and information on a character rectangle is obtained (step S4).
Next, the pixels corresponding to the background color are removed from the character rectangle on the original color image, and the remaining pixel values are obtained (steps S5 and S6). The representative color of the obtained pixel value is set as the character color of the character rectangle (step S7). In the embodiment shown in FIG. 2, the average value of all pixel values is used as the text color. However, a histogram of density values may be generated and the median may be used as the text color.

FIG. 3 is a diagram showing an example of how a character color of an arbitrary character rectangle to which the present invention is applied is specified.
0 is a character rectangle, and the character rectangle 10 is composed of pixels 10a (dark gray pixels) corresponding to the background color and pixels 10b (white pixels) and 10c (light gray pixels) constituting the character. When specifying the character color, processing is performed only on the area identified as a character as a result of the area identification. In the diagram shown on the left side of FIG. 3, the original character rectangle 10 includes the character A. The background color of the character A is obtained in advance in step S2. At this time, the diagram shown in the middle of FIG. 3 shows a state in which the pixel 10a corresponding to the background color has been removed from the character rectangle 10 and the remaining pixels (pixels 10b and 10c) have been obtained. One representative color is obtained from the obtained pixels, and is set as the character color of the character rectangle 10. The diagram shown on the right side of FIG. 3 shows a state in which the color of the pixel 10b is specified as a character color. Here, as means for specifying the character color, a histogram of the average value or the density value of each of the RGB planes may be generated, and the median may be used as the character color.

When specifying the representative character color of the document, after the character colors have been specified for all the character rectangles obtained by the area classifying means 2, the only representative character color representative of the document is determined. For example, a histogram relating to luminance is created, and a color corresponding to the median value is set as a representative character color.

FIG. 4 is a diagram showing an example of a GUI screen for presenting a character color specification result to a user. Character color specifying means 3
After obtaining all the character colors of the input document from, the result is presented to the user. FIG. 4 shows an actual window.
When the screen of s is shown and all the character rectangles identified by the area classification means 2 are displayed on the screen, the color specified by the character color specification means 3 is presented as the color of the rectangle. Here, it is difficult to understand in the drawing because the printing is monochrome, but in the upper drawing of FIG. The lower part of FIG. 4 shows a screen displaying the entire document.

FIG. 5 is a diagram showing an example of the operation when the mouse pointer is over a character rectangle. FIG. 5 (A)
Indicates that the mouse pointer is not over the character rectangle,
Usually, a rectangle is not displayed or is displayed in a dotted line or a light color. FIG. 5B shows a state in which the mouse pointer is over the character rectangle, and the color of the rectangle is changed to the character color, or the rectangle is displayed with the character color translucently filled.

FIG. 6 is a flowchart for explaining an example of a process for specifying a title / keyword in the document image recognition apparatus to which the present invention is applied. Character color specifying means 3
A method for extracting a title / keyword from a document image at high speed using the result obtained from the above will be described. First, character rectangle data is obtained from the area classification means 2 (step S11). Character rectangle data other than the representative character color of the document is obtained from the character color specifying means 3 (step S1).
2). This is called a title / keyword candidate character rectangle. A title character rectangle is selected (step S13).
For example, a chunk of character rectangle data located at the upper left of the document is defined as a title character rectangle. It is determined whether or not OCR is performed on the title / keyword candidate character string according to an instruction from the user (step S14).

If YES in step S14 (OCR execution), the OCR means 6 performs OCR on the title / keyword candidate character string (step S15). The final title / keyword character string is obtained from the OCR result (step S16). In this case, for example, the OCR means 6
If the CR certainty is obtained, a character string having an OCR certainty below a certain level may be excluded from the final result.
The final title / keyword character string is stored in the title / keyword DB 5 (step S17).

If NO in step S14 (no OCR), the final title / keyword character string is obtained in the state of the image (step S18), and the obtained final title / keyword character string is stored in the title / keyword DB 5 (step S18). S17). Here, an example of a data type when the data is stored in the title / keyword DB 5 is shown in Table 1 below. If there is no OCR, the column of the character string is left empty, and the image is presented to the user.

[0033]

[Table 1]

FIG. 7 is a diagram showing an example of selecting a character area necessary for generating a summary sentence according to the present invention.
Reference numeral 11 denotes an input document. The input document 11 includes six areas in which areas are identified and character colors are specified.
Regions R1 and 11b are regions R3 and 11c are regions R6. A method of generating a summary sentence from a document image at high speed using the result obtained from the character color specifying means 3 will be described below.

FIG. 8 is a flowchart illustrating an example of a process for generating a summary sentence in the document image recognition apparatus to which the present invention is applied. First, character rectangle data is obtained from the area classification means 2 (step S21). Character rectangle data other than the representative character color of the document is obtained from the character color specifying means 3 (step S22). This is called a title / keyword candidate character rectangle. An area including the acquired title / keyword candidate character rectangle is acquired (step S23).
This means that the regions R1, R3, and R6 shown in FIG. 7 are acquired. The acquired regions (R1, R shown in FIG. 7)
OCR is performed on (3, R6). A summary sentence is generated from the OCR result (step S25). Since this is not the point of the present invention, it is realized by using an appropriate conventional technique.

FIG. 9 is a diagram showing an example of generating a summary sentence from a plurality of pages. As shown in FIG. 9, for example, only predetermined areas Ra and Rb are selected from a plurality of pages and OCR is applied to execute the processing. The time can be significantly reduced.

[0037]

As is apparent from the above description, according to the first, seventh and thirteenth aspects, the color document image can be converted to the OC.
The character color can be specified directly without R.

According to the second, eighth and fourteenth aspects of the present invention, by removing the previously specified background color from each character rectangle, it is possible to accurately specify not only black characters but also thin character colors which have been difficult in the past. In particular, white characters can be handled.

According to the third, ninth and fifteenth aspects of the present invention, a character string described in a color other than the representative character color is regarded as an emphasized character string in the document by specifying a representative character color representing the document. Thus, application to claims 5 and 11 and claims 6 and 12 becomes possible, and in particular, there is an effect that an emphasized character string can be specified without performing OCR.

According to the fourth, tenth, and sixteenth aspects, the character color specification result can be presented to the user easily and instantly.

According to the invention of claims 5, 11 and 17, O
The title / keyword candidate character rectangle can be specified without CR.

According to the sixth, twelfth, and eighteenth aspects, it is possible to specify a character area necessary for generating a digest of a document without performing OCR, as in the fifth and eleventh aspects. Therefore, all characters in the document are actually
A summary sentence of a document can be efficiently generated without performing R.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration example of a document image recognition device to which the present invention is applied.

FIG. 2 is a flowchart illustrating an example of a process for specifying a character color in a document image recognition device to which the present invention is applied.

FIG. 3 is a diagram illustrating an example of how a character color of an arbitrary character rectangle to which the present invention is applied is specified;

FIG. 4 is a diagram illustrating an example of a GUI screen for presenting a character color specification result to a user.

FIG. 5 is a diagram illustrating an example of an operation when a mouse pointer is over a character rectangle.

FIG. 6 is a flowchart illustrating an example of processing for specifying a title / keyword in the document image recognition device to which the present invention is applied.

FIG. 7 is a diagram illustrating an example of selecting a character area necessary for generating a summary sentence according to the present invention.

FIG. 8 is a flowchart illustrating an example of a process of generating a summary sentence in the document image recognition device to which the present invention is applied.

FIG. 9 is a diagram illustrating an example of generating a summary sentence from a plurality of pages.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image input means, 2 ... area classification means, 3 ... Character color specification means, 4 ... Title / keyword extraction means, 5 ... Title / keyword DB, 6 ... OCR means, 7 ... Summary sentence creation means, 8 ... Summary sentence DB, 10: character rectangle, 11: input document.

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) H04N 1/40 H04N 1/40 F 5L096 1/60 D 1/46 1/46 Z F term (reference) 5B029 AA02 BB02 CC28 CC29 5B050 BA16 EA06 EA09 5B064 AA01 AA07 BA01 CA08 5C077 MP05 MP06 MP08 PP27 PP31 PP32 SS01 5C079 HA16 LA02 LA31 5L096 AA02 BA18 FA15 FA19 GA38

Claims (18)

[Claims] 1. A document image is input as a color digital image, color information is extracted from the document image, a background color of the document image is specified, and pixels other than the background area are extracted from the document image using the background color. Is extracted as small chunks, the extracted small chunks are integrated to generate connected components, and the connected components are classified into regions such as characters / ruled lines / diagrams / photographs based on their shape characteristics and color characteristics. What is claimed is: 1. A document image recognition method, comprising: identifying a character color of a character region from a color digital image for a character region that is a region classified as the character. 2. The method according to claim 1, wherein when the character color is specified, when a background color is present in the character rectangle classified as the character region, the character color corresponds to a pixel value substantially equal to the background color in the character rectangle. If the pixel to be performed is 0 and the other pixels are 1, then 1
A representative color is determined from a set of pixels having the following values, and the determined representative color is used as the character color of the character rectangle. 3. The method according to claim 1, wherein when the character color is specified, a character color is given to each of the classified character rectangles, and a distribution map for each character color is created. A color satisfying a specific condition as a representative character color of the document. 4. The color of a rectangle representing the character area when presenting an area such as a character / ruled line / figure / photo as a rectangle in a document to the user.
A document color recognition method characterized by using the character color specified by 5. The document image recognition method according to claim 3, wherein if there is a character string described in a color other than the representative character color of the document, the character string is used as a document title or a keyword. 6. The method according to claim 3, wherein if there is a character string described in a color other than the representative character color of the document, the whole area including the character string is OCR and obtained from a plurality of OCR-processed areas. A document image recognition method characterized by creating a summary of a document from a character code. 7. An image input means for inputting a document image as a color digital image, extracting color information from the document image, specifying a background color of the document image, and using the background color to convert a background image from the document image. Extract pixels outside the area as small blocks,
Document image recognition having area classification means for generating a connected component by integrating the extracted small blocks, and classifying the connected component into areas such as characters / ruled lines / diagrams / photographs based on the shape characteristics and color characteristics. A document image recognition apparatus, comprising: a character color specifying unit that specifies a character color of a character region from a color digital image for a character region that is a region classified as the character. 8. The method according to claim 7, wherein when specifying the character color, when a background color is present in the character rectangle classified as the character area, the character color corresponds to a pixel value that is substantially the same as the background color in the character rectangle. If the pixel to be performed is 0 and the other pixels are 1, then 1
A representative color is determined from a set of pixels having the following values, and the determined representative color is used as the character color of the character rectangle. 9. The method according to claim 7, wherein when the character color is specified, a character color is given to each of the classified character rectangles, and a distribution map for each character color is created. A document color recognizing device that sets a color satisfying a specific condition as a representative character color of the document. 10. The method according to claim 7, wherein the character / ruled line / figure /
8. A document image recognition apparatus, wherein when a region such as a photograph is presented to a user as a rectangle in a document, the color of the rectangle representing the character region is the character color specified by claim 7. 11. The method according to claim 9, further comprising, if there is a character string described in a color other than the representative character color of the document, a title / keyword extracting unit that uses the character string as a document title or a keyword. Document image recognition device. 12. The method according to claim 9, wherein if there is a character string described in a color other than the representative character color of the document, an OCR means for OCRing the entire area including the character string, A document image recognition device comprising a summary sentence creating means for creating a summary sentence of a document from the obtained character code. 13. An image inputting step of inputting a document image as a color digital image, extracting color information from the document image, specifying a background color of the document image, and using the background color to extract a background from the document image. Pixels other than the area are extracted as small chunks, and the extracted small chunks are integrated to generate connected components.
A computer-readable recording medium in which a program for causing a computer to execute a document image recognition method including an area classification step of classifying the image into a figure / photograph is recorded. And a computer-readable recording medium storing a program for causing a computer to execute a character color specifying step of specifying a character color of a character area from a color image. 14. A computer-readable recording medium storing a program for causing a computer to execute the document image recognition method according to claim 2, wherein the character is classified as the character area when specifying a character color. When there is a background color in the rectangle, the pixel corresponding to the pixel value substantially the same as the background color in the character rectangle is set to 0, the other pixels are set to 1, and the representative color is set from the set of pixels having the value of 1. Decide,
A computer-readable recording medium on which a program for causing a computer to execute a step of setting the determined representative color to the character rectangle character color is recorded. 15. A computer-readable recording medium on which a program for causing a computer to execute the document image recognition method according to claim 3 is recorded. A computer-readable recording medium that records a program that causes a computer to execute a step of creating a distribution map for each character color based on the distribution map and setting a color satisfying a specific condition as a representative character color of a document based on the distribution map. A readable recording medium. 16. A computer-readable recording medium on which a program for causing a computer to execute the document image recognition method according to claim 4 is recorded.
A program for causing a computer to execute a step of setting the color of a rectangle representing the character area to the character color specified by claim 1 when presenting an area such as a figure / photo as a rectangle in a document to a user. Computer readable recording medium. 17. A computer-readable recording medium storing a program for causing a computer to execute the document image recognition method according to claim 5, wherein a character string described in a color other than the representative character color of the document is provided. If the character string is the title of the document,
A computer-readable recording medium on which a program for causing a computer to execute a keyword extracting step is recorded. 18. A computer-readable recording medium storing a program for causing a computer to execute the document image recognition method according to claim 6, wherein a character string described in a color other than the representative character color of the document is provided. An OCR step for OCR of the entire area including the character string;
A computer-readable recording medium storing a program for causing a computer to execute a summary sentence creating step of creating a summary sentence of a document from character codes obtained from a plurality of R regions.