JP2001034763A - Document image processor, method for extracting its document title and method for attaching document tag information - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a title area and a mark attached by a user from a document image and to utilize the title and the mark as document tag information. SOLUTION: First, a title area extracting means 104 extracts an area of an area average character size being larger than a prescribed extraction decision value as a title area. Thus, a plurality of title areas can be extracted from one document image. Next, a mark extracting means extracts a mark attached to an input image by a user, and a calculating means calculates the characteristic value of the mark. Then, a document tag information attaching means selects document tag information to be attached to the input image among standard tag information on the basis of the characteristic value and the attribute value of the standard tag information. Thus, it is possible to automatically attach the document tag information to the document image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document image processing apparatus and a document image processing method for storing and managing document images as image data, and more particularly to extracting a title area and a mark given by a user from the document image. The present invention relates to the above-described apparatus and method used as document tag information.

[0002]

2. Description of the Related Art Along with a remarkable increase in the capacity of a data storage device, a document image processing device for storing and managing a paper document read from a scanner or the like as a document image as image data is rapidly spreading. .

In such a document image processing apparatus, in order to search for a desired document image from a plurality of document images stored in a data storage device, each document image includes document tag information such as a title and a keyword. Are registered in association with each other.

FIG. 19 conceptually shows the document tag information. As shown in this figure, document tag information, for example, “Top Secret” 191, “Company A” 192, “FY 1999” 1
93, "new car" 194 plays a keyword role for the document image 190. When a plurality of document tag information is assigned to each document image in this way, a necessary document image can be quickly searched by narrowing down the plurality of document tag information.

Conventionally, such document tag information has been manually input by a user when storing a document image. However, it is not realistic for the user to input the document tag information as the number of documents increases the amount of work. Therefore, in recent years, there has emerged an apparatus that performs character recognition on a document image and uses a character string as a result of the recognition as document tag information, so that the document tag information can be added without human intervention. I have.

For example, Japanese Patent Application Laid-Open No. 8-147313 discloses a method using mark sheet paper.
In this method, first, a user marks a check box of document tag information to be added to a document image, which is described on a mark sheet of a predetermined format. Then, by reading the mark sheet by the document image processing apparatus prior to the paper document, the document tag information to be added is designated from among the candidate document tag information registered in advance. According to this method, document tag information can be automatically added to a document image to be registered without using an input device such as a keyboard or a pointing device.

In order to efficiently search for a document image, it is important to provide appropriate document tag information. That is, a general search mode is to specify document tag information corresponding to a desired document image from a plurality of document tag information listed on the display, but such document tag information is quickly specified. To do so, each document tag information must be a content that briefly represents the content of the document.

[0008] In Japanese Patent Application Laid-Open No. H8-202859, after an area to which a title character string belongs (hereinafter referred to as a "title area") is extracted from a document image, character recognition is performed on the title area image, and the recognition result is obtained. Are proposed as a method of using a title character string as document tag information. Since the title character string is a simple description of the content of the document, the document image processing apparatus employing such a title area extraction method can quickly specify document tag information corresponding to a desired document image.

[0009]

The above-mentioned Japanese Patent Application Laid-Open No. 8-202
In the title area extraction method disclosed in Japanese Patent No. 859, from the viewpoint that the title character has the largest size among all the characters belonging to the document image, the document image is divided into a plurality of areas (areas in which adjacent character rectangles are integrated). The average character size in each area is calculated, and the area having the largest average character size is extracted as a title area. Therefore, the number of title areas extracted by such a title area extraction method is naturally one for each document image.

However, when there are a plurality of documents having similar contents, the title usually has similar contents.
When a large number of documents having similar contents exist, there is a problem that document tag information corresponding to a desired document image cannot be quickly specified.

[0011] To avoid the above-mentioned problem, it is possible not to give a title having similar contents at the stage of creating a paper document, but it is not preferable to request the user for such preparation work.

On the other hand, in the method disclosed in Japanese Patent Application Laid-Open No. Hei 8-147313 using a mark sheet, when the image management apparatus is constructed by software, the format and reading of the mark sheet paper in which all the items of the document tag information are described are described. Since it is necessary to define processing and the like, it is very troublesome.
When a new document tag information candidate is additionally registered later, since the items of the document tag information change, it is necessary to recreate the above-described mark sheet paper format, reading process, and the like.

Further, when using mark sheet paper, it is only necessary to always mark the check box using the same paper, so that it is difficult for the user to visually recognize which document tag information is added. There was also a problem that input errors were likely to occur.

The present invention has been proposed based on the above-described circumstances, and is a document image processing apparatus capable of extracting a plurality of title areas and marks provided by a user from one document image and using the extracted mark areas as document tag information. It is an object of the present invention to provide an apparatus, a document title extracting method and a document tag information adding method.

[0015]

The present invention employs the following means to achieve the above object.

First, as shown in FIG. 1, an area dividing means 103 for dividing a document image into a plurality of areas, and after calculating an area average character size for each area divided by the area dividing means 103, The following means is adopted in a document image processing apparatus provided with a title area extracting means 104 for extracting a title area from all areas based on the area average character size.

First, the title area extracting means 104
Calculates a total average character size corresponding to the average height of characters in all regions, and then compares an extraction determination value obtained by multiplying the total average character size by an extraction parameter with the region average character size. An area having an average character size larger than the value is extracted as a title area. In this way, if the area has an average character size larger than the above-mentioned extraction determination value, it is extracted as a title area, so that a plurality of title areas can be extracted from one document image.

Further, the title area extracting means 104
However, the above-described extraction determination values in a plurality of stages may be calculated using the extraction parameters in a plurality of stages. With this configuration, the extraction determination is made based on the extraction determination values in a plurality of stages. Therefore, not only the title area but also the subtitle area (the area to which the subtitle character string composed of characters slightly smaller than the title character belongs) Can also be extracted.

Further, the title area extracting means 104
However, the plurality of stages of extraction parameters may be determined based on a value obtained by dividing the maximum value of the area average character size by the total average character size. A better extraction judgment value can be obtained by calculating the extraction parameter based on the maximum value of the average character size of the area, instead of using a fixed value.

In obtaining the total average character size and the area average character size, accuracy can be improved by using a trim average excluding characters larger than a predetermined ratio and characters smaller than a predetermined ratio.

The extracted character image contained in the title area can be converted by the character recognition means 105 into a title character string which is a character code string. By modifying the title character string by the modifying means 112, the user can appropriately change the title of the document image.

Second, as shown in FIG. 12, in document image processing for reading a paper document to generate and store a document image, first, standard tag information (candidate of document tag information) is A standard tag information storage unit 1215 that stores the attribute value together with the attribute value is provided.

Next, there is provided a mark extracting means 1205 for extracting a specific mark given by a user on a paper document. Here, the mark refers to a general mark, such as a stamp, a sticker, an illustration, or a signature with a certain handwriting, that the user intends to identify a paper document.

Further, a calculating means 120A is provided for calculating a characteristic value representing the characteristic of the mark based on the distribution of the pixels constituting the extracted mark.

Then, a document tag information providing means 1208 for comparing the attribute value with the characteristic value, selecting standard tag information having the highest similarity, and providing the selected tag information to the document image is provided.

As described above, the document tag information can be automatically added to the document image based on the mark that the user uses on a daily basis when organizing the document, and the document management in the office or the like can be easily performed.

[0027]

(Embodiment 1) Embodiments of the present invention will be described below in detail with reference to the drawings. In the first, second, third and fourth embodiments, a document image processing apparatus that extracts a plurality of titles from one paper document will be described.

FIG. 1 is a schematic functional block diagram of a document image processing apparatus to which the present invention is applied. The configuration will be described below together with a document image registration procedure.

First, a document image input means 101 such as a scanner photoelectrically converts a paper document to obtain a document image 108a which is multi-valued image data.
At 1a, processing suitable for storage (for example, compression processing) is performed and registered in the document image area Aa of the storage unit 108. Of course, the multi-valued image data may be registered in the document image area Aa without providing the image processing unit 111a.

The document image from the document image input means 101 is input to the image processing means 111a.
The data is also input to the image processing unit 111b, where it is converted into binary image data and stored in the image memory 107. With the document image stored in the image memory 107 in this manner,
The character rectangle generating means 102 performs the following labeling processing with reference to the document image stored in the image memory 107. This labeling processing means that the black pixel among the pixels adjacent to the black pixel of interest (hereinafter, referred to as “target pixel”) in eight directions adjacent to the upper, upper right, right, lower right, lower, lower left, lower left, and upper left directions. This is a process of giving the same label value (identification information) as that of the pixel. That is, as shown in FIG.
When the eight pixels W3, W4, W6, W7, W8, and W9 are connected to the target pixel W5, the character rectangle generation unit 2 assigns the same label value to the black pixels W2, W3, and W8 as the target pixel W5. give. By performing such labeling processing, the same label value can be given to each black pixel connected component (consecutive black pixels) in the document image.

Next, the character rectangle generating means 102 generates a character rectangle by cutting out the black pixel connected component having the same label value as described above, and passes this character rectangle to the area dividing means 3. Here, the character rectangle means a circumscribed rectangle of the black pixel connected component. Note that some characters may not be composed of one black pixel connected component. In consideration of this, a process of expanding a black pixel region in a document image should be performed before performing the labeling process. Can also. In other words, this is a process of converting eight pixels adjacent to the black pixel of interest into black pixels. By performing this process an appropriate number of times (usually two or three times), the area of the black pixels is enlarged to 1 Black pixel connected components separated in one character can be combined into one. By performing the labeling process after performing such a process, it is possible to correctly generate the character rectangle for each character.

When the processing of the character rectangle generating means 102 is completed, the area dividing means 103 examines the neighborhood of each character rectangle and divides the area of the document image by integrating the character rectangles adjacent to each other. For example, the region dividing means 103 receiving the character rectangles C1 to C12 shown in FIG. 8 divides the document image into regions E1, E2, and E3 by integrating the character rectangles C1 to C4, C5 to C9, and C10 to C12, respectively. . By performing such area division processing, the area of the document image can be divided for each character string. Whether the character rectangles are adjacent to each other or between lines is determined by using an appropriate threshold value between characters and lines.

As a result, information such as the total character size (to be described later) in the document image, the number of divided areas, and the number of character rectangles in each area can be obtained. In the present invention, a serial number starting from 1 is assigned to each divided area, and a serial number starting from 1 is assigned to a character rectangle belonging to each area. The number is represented by NumChar _n and the m-th character size in the n-th area is represented by SizeChar _n , _m .

As shown in FIG. 9, the widths W1 to W4 and the areas A1 to A4 of the character rectangle vary greatly depending on the type of the character even when the character font having the same number of points is used. The height H of the character rectangle
Such fluctuations are small in 1 to H4. Therefore, in the present invention, "the height of the character rectangle" in which the point number of the character font is relatively accurately reflected is adopted as the character size.

Here, the title area extracting means 104
Only a predetermined area among all the areas divided as described above is extracted as a title area. Hereinafter, the title region extraction processing will be described with reference to the flowchart shown in FIG.

First, the title area extracting means 104 calculates an area average character size for each area (FIG. 2, step 1). This region average character size is the average value of all character sizes belonging to one region, and the region average character size SizeReg _n in the n-th region is the sum of all character sizes SizeChar _n and _m belonging to the region. This is a value obtained by dividing by the number of characters in the area NumChar _n . This relationship is shown in the following equation.

[0037]

(Equation 1)

Next, the area average character size SizeReg _{n of} each area calculated as described above and the number of characters in the area NumChar
_{From n} , the total average character size SizeAll in the document image is calculated by the following formula (FIG. 2, step 2).

[0039]

(Equation 2)

[0040] The area average character size SizeReg _n and the method of calculating the total average character size SizeAll is not limited to the method described above, for example, a predetermined proportion of the trimmed mean (minimum value side and the maximum value side, which will be described later For example, 1
Method of calculating the average value after excluding 0% data)
Can also be adopted.

Here, the title area extracting means 104
The extraction determination of the title area is performed based on whether or not the following extraction determination formula is satisfied.

[0042]

(Equation 3)

That is, the value (extraction judgment value) obtained by multiplying the total average character size SizeAll calculated as described above by the extraction parameter α is compared with the region average character size SizeReg _{n of} each region, and this extraction judgment formula is established. Only the area to be extracted is extracted as a title area (FIG. 2, steps 3 → 4 → 5).
Note that the extraction parameter α is a constant larger than 1.0, and 1.
A value of about 2 is preferred.

When the above procedure is repeated and extraction determination is performed for all the regions (FIG.
O "), the title area extraction processing is terminated, and each title area image 108b extracted here is registered in the title area Ab of the storage means 108.

Next, the character recognizing means 105 cuts out the title area image extracted as described above from the document image, and performs character recognition on the title area image to obtain a title character string as a character code string. . The title character string obtained here is passed to the display control means 110 via the correction means 112, displayed in a list on a display (not shown) together with the title area image, and presented to the user (see FIG. 10 (I)).

The user checks the displayed title area image and the title character string, and if the user wants to register the title character string as displayed, the instruction input means 109
Indicates registration. Then, the title character string is passed from the character recognition means 105 to the document registration means 106.

On the other hand, when the user wants to change or modify the title character string, the user double-clicks the displayed title character string by using the pointing device of the instruction input means 109, for example. The correction means 112, based on this double click,
For 0, for example, an instruction is made to blink the title character string on the display and to display a cursor in the character string. Then, the user operates the keyboard of the instruction input means 109 to input the correction character string to the correction means 112 and replace the character string after the cursor with the correction character string. The character string corrected in this way is input from the correction means 112 to the character recognition means 105, and the title character string is corrected. Then, similarly to the above, when the user instructs registration using the instruction input unit 109, the title character string after the correction is transferred from the character recognition unit 105 to the document registration unit 106.

If the confirmation and correction processing is not provided, the contents recognized by the character recognition means 105 are passed to the document registration means 106 without being displayed on the display.

Upon receipt of the title character string, the document registration means 106 stores the document image 108a in the storage means 108, the storage pointer of the title area image 108b, the title character string, and the position of the title area in the document image. Storage means 1 for storing registration information comprising
08 in the registration information management table 108c (see FIG. 5) formed in the table area Ac. Here, the storage pointer of the document image 108a is
8 is obtained from the document image area Aa, the storage pointer of the title image 108b is obtained from the title area Ab of the storage means 108, and the position and size of the title area are obtained from the character recognition means 105.

As described above, the registration information management table 108c
Is generated, thereafter, when an instruction to search for a document image is input from the instruction input unit 109 including a keyboard, a pointing device, and the like, the display control unit 110 transmits the title area image and the title stored as described above. A list of character strings is displayed on the display (FIG. 10 (I)).
).

When the user selects a desired title (title area image or title character string) from the list display by the instruction input means 109, the display control means 110 displays a document image corresponding to the title on the display. I do. At this time, as shown in FIG. 10 (II), it is preferable to clearly indicate the title area in the document image by surrounding it with a rectangular frame F or the like. Such a rectangular frame F can be generated based on the position and size of the title area registered in the registration information management table 108c.

In addition to the method of selecting any one from the list displayed on the display as described above,
By inputting specific document tag information from the instruction input means 109,
When a title corresponding to the document tag information is registered in the registration information management table 108c, it goes without saying that a corresponding document image may be displayed.

As described above, according to the present embodiment, if a region having an average character size larger than the extraction determination value is extracted as a title region, a plurality of title regions are extracted from one document image. it can. Therefore, even when there are many documents having similar contents, the document tag information (title) corresponding to the desired document image can be quickly specified.

Although the above description does not refer to the procedure when there is no area where the extraction judgment formula is satisfied in the title area extraction processing, in such a case, the title area is extracted. Is displayed on the display and the user is requested to input a character string serving as document tag information. When the user inputs the character string in response to this request, the character string is changed to the title character string of the document image. Is used as

(Embodiment 2) In Embodiment 1 described above,
If the area has a region average character size larger than the extraction determination value, the region is similarly extracted as a title region without distinguishing the size of the region average character size. Therefore, it is not possible to perform an appropriate process based on the size of the area average character size, such as a process of displaying only a title character string in a list without displaying a subtitle character string composed of characters slightly smaller than the title character. In the present embodiment, the above is described by adopting a configuration in which a plurality of levels of extraction determination values are calculated using a plurality of levels of extraction parameters, and a title area is extracted in association with a level attribute (information indicating the level of the extracted level). Since the problem has been solved, only the configuration different from that of the first embodiment will be described below.

The area average character size SizeReg _n and the total average character size SizeAll are obtained in the same procedure as in the first embodiment.
The title region extracting means 104 that has calculated the above-mentioned formulas performs a multi-stage extraction judgment based on whether or not the following multi-stage extraction judgment formula holds.

[0057]

(Equation 4)

Α _p in the above equation is a p-step (level p)
The value is set so as to satisfy the condition of [Equation 5]. For example, when performing five-stage extraction determination, α ₁ = 1.5, α ₂ = 1.3, α ₃ = 1.2, α ₄ = 1.15,
α ₅ is preferably about 1.1.

[0059]

(Equation 5)

Referring to the flowchart shown in FIG. 3, the title area extracting means 104 performs extraction determination of all levels in order from level 1 (FIG. 3, step 14 → 1).
5 → 14), if the extraction determination formula does not hold at all levels, this area is not extracted as a title area, and extraction determination is performed for the next area (FIG. 3, steps 14 → 13 → 14 → 15). ). On the other hand, if the extraction determination formula is satisfied at any level, this area is extracted as the title area of the level (in association with the level attribute), and then the extraction determination is performed for the next area (FIG. 3). Steps 15 → 16 → 13 → 14 → 15).

When the above procedure is repeated to determine the extraction for all the regions (FIG.
O "), the title area extraction processing ends.

It is to be noted that the character string input by the user is used as the title character string when there is no area where the extraction judgment formula holds, as in the first embodiment, and the level attribute of this title character string is Level 1 and the total number of levels are also 1.

Also, the point that the extracted title character string can be changed / corrected is the same as in the first embodiment.

FIG. 6 is an explanatory diagram of the registration information management table 108c according to the present embodiment.
(Fields 501 to 505), a "level attribute" field 601 and a "total number of levels" field 602 are added. For example, when there is an area extracted at level 1 in the five-stage extraction determination, the document registration unit 106 sets “5” in the “total level number” field 602 and “level attribute” field 601 corresponding to this area. Is registered as "1".

FIG. 11 is a diagram showing the contents displayed on the display at the time of retrieval according to the present embodiment.
09 can be specified. Then, the display control unit 110 refers to the “level attribute” field 601 and the “total number of levels” field 602 of the registration information management table 108c to list only the titles within the range specified as described above on the display. indicate.

As described above, according to the present embodiment, a plurality of levels of extraction determination values are calculated using a plurality of levels of extraction parameters, and the title area is extracted in association with the level attribute. Different processing can be performed based on the size of the area average character size, such as processing for displaying only the title character string in a list without displaying the character string in a list.

(Embodiment 3) In Embodiment 2 described above,
Although the configuration is such that the extraction parameters in a plurality of stages are set in advance (fixed values), it is preferable that such extraction parameters are determined according to the characteristics of the input document image. In the present embodiment, a plurality of levels of extraction parameters are determined based on a value obtained by dividing the maximum value of the area average character size by the total average character size (see step 23 in FIG. 4). Only the points different from the second embodiment will be described.

The area average character size SizeReg _n and the total average character size SizeAl are obtained in the same procedure as in the second embodiment.
Title area extracting means 4 to calculate the l first calculates the maximum value max value alpha ₁ for the {SizeReg _n} divided by the total average character size SizeAll area average character size by the following equation.

[0069]

(Equation 6)

Next, the title area extracting means 4 calculates α ₁ calculated as described above and the total number of levels P (P
> = 1), the extraction parameter α _p of each level is determined by the following equation.

[0071]

(Equation 7)

For example, when α ₁ is 1.5 and five levels of extraction judgment are performed, the extraction parameters α _{1 to} α ₅ of each level are as follows.

[0073]

(Equation 8)

As described above, according to [Equation 7], the extraction parameter α _p of each level can be determined so as to be equally spaced between α ₁ and 1.0 calculated as described above.

The subsequent procedure is the same as that of the second embodiment except that the extraction determination is performed using the extraction parameters determined as described above, and therefore the description is omitted.

[0076] However, in the method described above, if there is no title area in the document image, alpha _1, for example 1.03, such as 1.0
There is a problem that a text region is erroneously extracted as a title region because the value is in the vicinity. Therefore, in the present invention, an extraction parameter having a predetermined value or less, such as 1.05, is not adopted.

If the difference between the extraction parameters at each level is equal to or less than a predetermined value, for example, 0.03, it is not possible to make a good extraction judgment. Therefore, the extraction parameter is set so that the difference between the extraction parameters becomes the predetermined value (0.03). The settings are modified. That is, in the above case, α ₁
The value subtracted by 0.03 is set as the extraction parameter of each level.

As a result, the total number of levels P may decrease. In such a case, the actual number of levels (the value obtained by subtracting the number of reduced levels from the total number of levels P) is registered as the total number of levels P. This is set in the “all levels” field 602 of the information management table 108c.

As described above, according to the present embodiment, the extraction parameters are determined not according to the fixed values but according to the characteristics of the input document image. it can.

(Embodiment 4) In each of the above embodiments, the characters in the title area having a relatively large size are included in the calculation of the total average character size, and commas, periods, and punctuation marks having small sizes are also included. Accuracy tends to be lower. Therefore, from all the characters in the document image,
A character having a size larger than a predetermined ratio (for example, 90%);
A so-called trim average is used in which a total average character size is calculated from characters excluding characters smaller than a predetermined ratio (for example, 10%). Further, the same problem occurs when calculating the area average character size. Therefore, the trim average can also be used for calculating the area average character size.

As a result, the character size excluding the period, comma, and punctuation can be obtained for both the total average character size and the area average character size, and a value with higher precision can be obtained.

Here, in each of the above embodiments, the total average character size is calculated from the region average character size. However, if the same method is applied to the case where the trim average is used, a character having a large size for each region is used. Since small characters are excluded, it is not possible to exclude all characters included in the title area in calculating the total average character size. Therefore, here, when calculating the total average character size, processing is performed again for all characters in the document image.

However, even if the method using this trim averaging is used, any of the predetermined values of the first embodiment or the step values of the second and third embodiments may be used as the extraction parameters. Of course.

Although the description of each of the above embodiments does not refer to the number of documents serving as document images,
The number of paper documents is not particularly limited. That is, the same effect can be obtained for one page or a plurality of pages as long as the same extraction parameter is used for each page. In particular,
In the second and third embodiments, by using the same extraction parameter for a plurality of pages, a title and a subtitle can be correctly extracted from a single document that covers a plurality of pages, such as paper data.

In the above description, the height of the character rectangle is adopted as the character size. However, the width and area of the character rectangle may be adopted as the character size.

In the description of FIG. 1, the storage unit 108
Image processing means 111
a and 111b, and the document image for title extraction is 2
Compressed images or multi-valued image data can be used as document image data registered in the document image area Aa of the storage unit 108 using the value image data. As a result, various display methods are possible, such as displaying a document image obtained as a result of the search processing based on the title extracted as described above in color.

(Embodiment 5) Hereinafter, Embodiments 5 and 6 will be described with respect to a document image processing apparatus which automatically adds a mark given to a paper document by a user as document tag information.

First, a mark made up of a title, a keyword, or the like is added to one of the pages constituting the paper document by the user. Here, marks are stamps, stickers,
It refers to general marks, such as illustrations and signatures with certain handwriting, that the user attaches with the intention of identifying paper documents.

When storing a paper document including a large number of pages in the document image processing apparatus of the present invention, it is necessary to determine which page of the paper document has the mark. At this time, a method of detecting the mark by searching all the pages of the paper document is conceivable, but there is a problem that the detection process takes time.

As a method for solving such a problem,
For example, the document image processing apparatus may be set in advance so that the mark is detected only on the first page.

In the embodiment of the present invention, FIG.
As shown in (b), for the pages (hereinafter referred to as “document tag information designation pages”) 21 and 24 to which the above-mentioned mark is attached, a specific two-dimensional code image 26
Is described, the document tag information designation page is determined.

FIG. 1 is a block diagram of a document image processing apparatus according to a fifth embodiment of the present invention. Hereinafter, a procedure of processing performed by the document image processing apparatus will be described.

First, the image input unit 1201 digitizes a paper document by using a photoelectric conversion device such as a scanner or a digital multifunction peripheral, and inputs it as a document image. Here, as shown in FIG. 13, document tag information of “secret”, “Company A”, and “1999” attached to the document tag information designation page 21 is added to the input images 22 and 23, and the document tag information is added to the input image 25. The document tag information of “top secret” and “company B” attached to the designated page 24 is added. Then, the image input means 120
1 includes a document tag information designation page 21, an input image 22,
23, a document tag information designation page 24, and an input image 25 in this order.

The document image input here is temporarily stored in the image memory 1202, further subjected to data compression by the image data compression processing unit 1203, and then stored in the image storage area 1211 of the storage unit 1210. . At this time, an image ID is assigned to each document image so that each stored document image can be specified, and this image ID is assigned to “Image I” in the registration management table 1212 shown in FIG.
D "field 121. The storage means 12
The pointer information to the image data stored in the ten image storage areas 1211 is stored in the “pointer to image data” field 122 of the registered image management table 1212.

The document image stored in the image memory 1202 is converted into a binary image by
After being digitized, it is also sent to the mark extracting means 1205.
The mark extracting means 1205 first determines whether or not a specific two-dimensional code image exists at a predetermined position at the lower right of the image, so that each input document image is displayed on the document tag information designation page. Determine if there is.

At this time, with respect to the document image determined to be the document tag information designated page, “1” is set in the “document tag information designated page flag” field 123 of the registered image management table 1212, and otherwise “0”. Is stored. This flag is used to identify that the document image is a document tag information designation page to which only a mark is added and does not include the contents of a paper document as a document. For example, after document tag information is added to a document image by a method described below, based on this flag,
If the document image corresponding to the document tag information designation page is deleted, memory resources can be saved.

The same mark management group number is assigned to all the input document images from the time when a certain document tag information designation page is input to the time when the next document tag information designation page is input. Further, this mark management group number is stored in the “mark management group number” field 125 of the registered image management table 1212. Here, it means that the same document tag information is assigned to the document images to which the same mark management group number is assigned.

Next, the mark extracting unit 1 extracts the document image determined as the document tag information designated page by the above processing.
The process of extracting a mark by 205 will be described.

First, the labeling process described in the first embodiment is performed on all of the document tag information designated pages except for the region to which the two-dimensional code is added. Then, among a plurality of black pixel connected components obtained by the labeling process, components having a mutual distance smaller than a specific threshold are integrated into one region. As shown in FIG. 16, the areas obtained in this manner correspond to the areas 41 to 43 of the respective marks, and by extracting these areas, each mark image can be obtained.

Here, the number of marks extracted from each document tag information designation page is determined by the registration image management table 1 described above.
212 is stored in the “number of marks” field 124.

In order to manage the information of each extracted mark image, a mark ID is assigned to each mark image and stored in a “mark ID” field 131 of a mark management table 1213 as shown in FIG. . Further, the mark management group number of the document tag information designated page to which each mark is attached is stored in the “mark management group number” field 132 of the mark management table 1213. The position and size (width, height) of the mark image extracted from each document tag information designation page in the document tag information designation page are stored in the “position” field 134 of the mark management table 1213, respectively. "size"
It is stored in the field 135.

In this embodiment, the same mark management group number is assigned to a document image input between the first document tag information designation page and the next document tag information designation page, and It is managed as a series of document images attached to the first document tag information designation page. In addition, a management method in which a mark management group number is assigned only to a specific document image input next to the document tag information designation page and a mark management group number is not assigned to other document images is also conceivable. This is a management method used when, for example, it is desired to add a table of contents to the specific document image.

Next, the characteristic amount calculating means 1206 of the calculating means 120A calculates a numerical value representing the characteristic of each mark image extracted by the mark extracting means 1205. Here, a feature amount in a known technique, Moment Invariants, is used as the numerical value. Hereinafter, this moment invariant will be briefly described.

It is assumed that i and j are pixel coordinates, and I (i, j) is a function having the pixel value, that is, I = 1 for a black pixel and I = 0 for a white pixel. Then, _mpq defined by [Equation 9] is called a (p + q) _-order moment.

[0105]

(Equation 9)

Here, when this m _pq is used, the center of gravity (x, y) of the two-dimensional image is represented by [Equation 10].

[0107]

(Equation 10)

Based on the center of gravity calculated in this manner, μ _pq defined by [Equation 11] is called a center moment.

[0109]

[Equation 11]

Based on this central moment, numerical values M1 to M6 calculated as
It is defined as a feature amount (in the moment invariant) of the two-dimensional image.

[0111]

(Equation 12)

Since these feature amounts do not change even when the two-dimensional image is rotated or translated, the user manually attaches a specific mark on the sheet as in the embodiment of the present invention. In such a case, the numerical value is effective for characterizing this mark.

The feature quantity calculated by the feature quantity calculation means 1206 is passed to the similarity calculation means 1207 of the calculation means 120A, and the similarity between this feature quantity and the attribute value of each standard tag information is calculated. You. In order to explain this method, first, a method of managing each standard tag information and a method of calculating an attribute value of each standard tag information will be described.

The above-mentioned standard tag information is specifically data associated with a mark (hereinafter, referred to as a “standard mark”) that is expected to be used by the user, and plays a role of a keyword for an input image. This is a candidate for document tag information such as a character string. This standard tag information is stored in the “standard tag information” field 141 of the standard tag information management table 1214 as shown in FIG. The image data of the standard mark is stored in the standard tag information storage unit 1215, and a pointer to the image data is stored in the “pointer to standard mark” field 142 of the standard tag information management table 1214. ing. Further, the characteristic amount calculating means 1206 calculates six characteristic amounts in the moment invariant of these standard marks, and stores these characteristic amounts in the standard tag information management table 121.
4 in the "attribute value (M1 to M6)" field. That is, this feature amount becomes the attribute value of each standard mark.

The distance between the attribute value of each standard mark calculated in this way and the feature value of the mark image extracted from the input image in the moment invariant is calculated by [Equation 13] (least square method). I do.

[0116]

(Equation 13)

Here, M1 to M6 represent attribute values of the standard mark, and m1 to m6 represent feature amounts of the extracted mark image. As the value of the distance L calculated by the above equation is smaller,
This indicates that the similarity between the extracted mark image and the standard tag information is high.

Next, the document tag information determining means 1208
The standard mark having the maximum similarity calculated by the similarity calculating means 1207 is specified, the standard tag information of this standard mark is selected as the document tag information of the input document image, and the document image is given to this document image. Further, the document tag information is stored in the “document tag information” field 133 of the mark management table 1213.

By applying the above processing, document tag information can be automatically added to each input document image. Using the information of each table obtained here, an image can be searched according to the following procedure.

First, when the user specifies the document tag information to be used for the search, the mark management group number associated with the document tag information is set to the mark management table 121.
3 can be specified. Furthermore, the image ID of the document image to which the mark management group number has been assigned and the information of the pointer to the document image data can be specified from the registered image management table 1212. The document image specified here is an image associated with the document tag information specified by the user. Also, by specifying a plurality of document tag information, it is possible to narrow down image data to be searched.

Next, regarding the document tag information having the maximum similarity calculated by the similarity calculating means 1207,
If the distance L from the extracted mark image is greater than a predetermined threshold, it is determined that there is no existing document tag information to be associated with this mark image and that a new standard mark has been input. I do. In this case, the “position” field 134 and the “size” field 135 of the mark management table 1213 and the registered image management table 12
A mark image is displayed on the basis of the information in the "pointer to image data" field 122, and the user is prompted to register document tag information for associating the new standard mark.

The input document tag information is newly added to the “standard tag information” in the standard tag information management table 1214.
It is stored in the field 141. Further, the image data of the newly input standard mark is stored in the standard tag information storage unit 1215 for use in subsequent search processing, and pointer information to the mark image data is stored in the standard tag information management table 1214. The “pointer to the mark image” field 142 is stored. Further, the feature amount of the new standard mark in the moment invariant is calculated, and the “attribute value (M1
To M6) ”field 143.

As described above, the user can register new standard tag information only by inputting a new mark image and document tag information.

In the above description, FIGS.
In 5 (a), the standard tag information associated with the standard mark is a character string used for the standard mark, but these need not necessarily be limited to character strings. That is, in the standard tag information management table 1214, it is possible to associate arbitrary standard tag information with each standard mark.

For example, instead of the standard tag information using a character string as described above, a reduced image of each standard mark is associated with each standard mark as standard tag information, and this reduced image is printed on a search sheet. deep. By scanning the reduced image of the search sheet with a scanner, a desired document image can be searched.

Furthermore, in the explanatory diagrams of FIGS. 13 and 16,
Although a two-dimensional code is used to specify a document tag information designation page from all input images, a one-dimensional code or the like may be used. In addition, as a means for specifying the document tag information designation page, a method of using a specific mark instead of a two-dimensional code image, a method of using paper of a specific color, or a specific shape or size. The same effect can be expected by using a method using paper.

When the same document tag information is added to all input images, a document image processing apparatus is constructed by defining only the first image to be input as a document tag information designation page image. It is also possible. In this case, since it is known that only the first image input is the document tag information designation page, the process of specifying the document tag designation image using a two-dimensional code image or the like is not necessary, and the entire process is omitted. It can be simplified.

Of course, it is also possible to search all pages of a paper document and extract marks without using a two-dimensional code image or the like. At this time, apart from the mark added by the user, a character such as “top secret” in the paper document may be extracted as the mark. In such a case, the character may be added to the registered image management table 1213 as one of the marks.

In the above description, the mark image and the standard tag information are associated with each other by using the feature amount in the moment invariant. However, the two images are overlapped to compare the ratio of the matching black pixels. The same effect can be expected even if the association is performed by using.

Further, a plurality of standard marks can be associated with one standard tag information. This can be realized by registering a plurality of the same standard tag information in the standard tag information management table 1214 and associating each with a different standard mark. In this case, a paper document with a different mark can be input, and the same document tag information can be added to the input document image.

Conversely, one standard mark may be associated with a plurality of standard tag information. This can be realized by associating the same standard mark with different standard tag information in the standard tag information management table 1214. In this case, a paper document with one mark can be input, and a plurality of document tag information can be added to the input document image.

(Embodiment 6) In this embodiment, a document image is given document tag information by extracting a mark pressed in a margin of a paper document to be registered. Hereinafter, only differences from the fifth embodiment will be described with reference to FIG.

First, as in the fifth embodiment, the image input unit 1201 digitizes a paper document input by the user to obtain a document image. Here, as shown in FIG.
The document images 31 and 32 contain document tag information of “secret”, “company A”, and “1999”, and the document image 33 contains “secret”, “B”
Document tag information of “company”. For this reason, a mark associated with the document tag information to be added is attached to the margin of each image.

The image data obtained here is temporarily stored in the image memory 1202, further compressed in the image data compression processing unit 1203, and then stored in the image storage area 1211 of the storage unit 1210. As information of the image data stored here, as shown in FIG. 17A, “Image I” in the registered image management table 1212 ′ is used.
The storage of necessary information in the "D" field 121 'and the "pointer to image data" field 122' is the same as in the fifth embodiment.

The image in the image memory 1202 is binarized by the image binarization processing unit 1204, and then sent to the mark extraction unit 1205 '. In the present embodiment,
In order to reliably extract the area of the mark image, a mark with a frame as shown in FIG. 17B is used, and the mark extraction means 1205 'extracts each mark by the following processing.

First, the above-described labeling process is performed on the black pixels of each binary image, and the size of the circumscribed rectangle is calculated for each black pixel connected component to which the same label value has been added. At this time, the circumscribed rectangle size of the black pixel connected component corresponding to the frame portion of the mark is sufficiently larger than the size of each character in the input image, but the mark needs to be pushed so as to fit in the margin of the document. Therefore, the size does not become extremely large. By utilizing this property, only the area where the size of the area occupied by the circumscribed rectangle falls within the two specified threshold values is extracted from the black pixel connected components obtained by the labeling processing. That is, the height and the width are each determined by a certain threshold [the size of the margin (height,
Width) is usually larger than
In addition, by extracting only the area of the black pixel connected component that is smaller than another certain threshold value (the maximum size that is generally considered as a margin size), the area of each mark image can be extracted.

By the above processing, the number of marks extracted from each document image is stored in the “number of marks” field 124 ′ of the registered image management table 1212 ′. Also, a mark ID is given to each extracted mark image, and this mark ID is assigned to a “mark ID” field 131 ′ of a mark management table 1213 ′ as shown in FIG.
To be stored. Also, the image ID of the input image to which each mark is attached, the information about the attached mark and the size of the mark are stored in the mark management table 121, respectively.
3 'are stored in the fields 132', 134 'and 135' of "image ID", "position" and "size".

In this embodiment, the document tag information is added only to the marked image. In addition, the images input between the image with the first mark and the image with the next mark are converted into a series of document images attached to the image with the first mark. If management is desired, a method of assigning and managing a mark management group number as in the fifth embodiment can be adopted.

The calculation means 120A (feature value calculation means 1206, similarity calculation means 1207) and the document tag information determination means 1208 are similar to the fifth embodiment, and each of them is a feature quantity of a moment invariant which is a known technique. , The document tag information associated with each mark image is specified. Then, the specified document tag information is stored in the “document tag information” field 133 ′ of the mark management table 1213 ′.

By using the above processing, a mark can be pressed in the margin of a paper document to be registered and input.
It is possible to automatically detect and add document tag information. In this case, the document tag information designation page used in the fifth embodiment is unnecessary, and only the document to be registered is input. As described above, the registered image management table 1212 ′ and the mark management table 1213 ′ have a simpler configuration than the registered image management table 1212 and the mark management table 1213 in the fifth embodiment.

Of course, in this embodiment, as in the fifth embodiment, a method of adding a two-dimensional code or the like to a marked page may be adopted in order to speed up the mark extraction process. it can.

In the present embodiment, the mark is pressed in the blank portion of the surface on which the content of the paper document is described, but the input is made by using a scanner capable of reading both sides. The same effect can be expected when inputting by pressing the mark on.

Further, the above-mentioned mark has a frame, but this frame is not essential. Even if there is no frame,
Since a normal mark is considered to be composed of black pixel connected components larger than characters in a paper document body, this embodiment can be applied.

[0144]

As described above, first, according to the present invention, an area having an average character size larger than the extraction determination value is extracted as a title area.
A plurality of title areas can be extracted from one document image.
In addition, a plurality of stages of extraction determination can be performed based on a plurality of stages of extraction parameters, and the plurality of stages of extraction parameters can be determined according to characteristics of the input document image. Further, by using the trim average, which is calculated by excluding characters belonging to the larger predetermined ratio and the smaller predetermined ratio, in the calculation of the total average character size or the calculation of the region average character size, accuracy can be further improved. it can.

Furthermore, secondly, according to the present invention, only by inputting a marked document to a document image processing apparatus without using a keyboard or a pointing device, document tag information is automatically added to an input image. Can be provided. Since the document image can be searched by using the document tag information given here, the document image processing apparatus can be efficiently managed and operated.

[Brief description of the drawings]

FIG. 1 is a schematic functional block diagram of a document image processing device according to a first embodiment of the present invention.

FIG. 2 is a flowchart of a title area extraction process according to the first embodiment.

FIG. 3 is a flowchart of a title area extraction process according to the second embodiment.

FIG. 4 is a flowchart of a title area extraction process according to the third embodiment.

FIG. 5 is an explanatory diagram of a registration information management table according to the first embodiment.

FIG. 6 is an explanatory diagram of a registration information management table according to the second embodiment.

FIG. 7 is an explanatory diagram of a labeling process.

FIG. 8 is an explanatory diagram of a region dividing process.

FIG. 9 is a diagram illustrating a relationship among height, width, and area of a character rectangle.

FIG. 10 is a diagram showing contents displayed on a display at the time of search according to the first embodiment.

FIG. 11 is a diagram showing contents displayed on a display at the time of a search according to the second embodiment.

FIG. 12 is a schematic functional block diagram of a document image processing device according to Embodiments 5 and 6 of the present invention.

FIG. 13 is an explanatory diagram of a registered image management table according to the fifth embodiment of the present invention.

FIG. 14 is an explanatory diagram of a mark management table according to the fifth embodiment of the present invention.

FIG. 15 is an explanatory diagram of a document tag information management table.

FIG. 16 is an explanatory diagram related to a result of extracting a mark image.

FIG. 17 is an explanatory diagram of a registered image management table according to the sixth embodiment of the present invention.

FIG. 18 is an explanatory diagram of a mark management table according to the sixth embodiment of the present invention.

FIG. 19 is an explanatory diagram showing the concept of document tag information.

[Explanation of symbols]

 101 image input means 102 character rectangle generation means 103 area division means 104 title area extraction means 105 character recognition means 106 document registration means 107 image memory 108 storage means 108a document image 108b title area image 108c registration information management table 109 instruction input means 110 display Control means 111a, 111b Image processing means 112 Correction means 1201 Image input means 1202 Image memory 1203 Image data compression processing means 1204 Image binarization processing means 1205, 1205 'Mark extraction means 120A Calculation means 1206 Feature amount calculation means 1207 Similarity calculation Means 1208 Document tag information adding means 1210 Storage means 1211 Image storage area 1212, 1212 'Registered image management table 1213, 1213' Mark management table 214 standard tag information management table 1215 standard tag information storing means

Continued on the front page (72) Inventor Akito Umebayashi 1006 Kazuma Kadoma, Kadoma City, Osaka Prefecture F-term in Matsushita Electric Industrial Co., Ltd. FA59 FA64 GA15 GA34 HA08 JA03 JA11

Claims (29)

[Claims] An image input unit configured to read a paper document to generate a document image; an area dividing unit configured to divide the document image into a plurality of areas; and an average size of a character in each area divided by the area dividing unit. Calculating a region average character size corresponding to the character region, and then extracting a title region from all regions based on the region average character size. After calculating a total average character size corresponding to the size, an extraction determination value obtained by multiplying the total average character size by an extraction parameter is compared with the region average character size. A document image processing apparatus comprising the above-described title region extracting means for extracting a title as a title region. 2. The document image processing apparatus according to claim 1, further comprising: the title area extracting unit that calculates the area average character size and the total average character size based on an average character height. 3. The document image processing apparatus according to claim 1, further comprising: the title area extracting unit that calculates the area average character size and the total average character size based on an average character width. 4. The document image processing apparatus according to claim 1, further comprising the title area extracting means for calculating the area average character size and the total average character size based on an average area of characters. 5. The document image processing apparatus according to claim 1, wherein the title area extracting means calculates the extraction determination value in a plurality of stages using an extraction parameter in a plurality of stages. 6. The title region extracting means calculates a plurality of stages of the extraction determination value using a plurality of stages of extraction parameters, and extracts a title region in association with a level attribute indicating the extracted stage. 2. The document image processing device according to 1. 7. The document image according to claim 2, wherein the title region extracting means determines the plurality of stages of extraction parameters based on a value obtained by dividing a maximum value of the region average character size by a total average character size. Processing equipment. 8. The title area extracting means uses a trim average for calculating the total average character size and the area average character size from characters excluding characters larger than a predetermined ratio and characters smaller than a predetermined ratio. A document image processing apparatus according to claim 1. 9. The document image processing apparatus according to claim 1, further comprising a correction unit that corrects the extracted character string of the title area. 10. The document image processing apparatus according to claim 1, wherein said document image is a document image of a plurality of pages. 11. An image input process for reading a paper document to generate a document image, a dividing process for dividing the document image into a plurality of regions, and calculating an average character size of each region corresponding to an average character size. In the document title extracting method of the document image processing device, the calculating method comprises: a calculating process of extracting the title region from the entire region based on the average character size of the region. The above-described calculation processing for calculating the total average character size, a comparison processing for comparing the extraction determination value obtained by multiplying the total average character size by an extraction parameter with the area average character size, and an area average character size larger than the extraction determination value Extracting the document title as a title area. Method. 12. The document title extracting method according to claim 11, further comprising the calculation processing for calculating the area average character size and the total average character size based on an average character height. 13. The document title extracting method according to claim 11, further comprising the calculation processing for calculating the area average character size and the total average character size based on an average character width. 14. The method for extracting document tag information according to claim 11, further comprising the calculation processing of calculating the area average character size and the total average character size based on the average area of the character. 15. The document title extracting method of the document image processing apparatus according to claim 14, wherein the title extracting process calculates the extraction determination value in a plurality of stages using an extraction parameter in a plurality of stages. 16. The method according to claim 1, further comprising calculating the extraction determination values in a plurality of stages using the extraction parameters in a plurality of stages, and extracting a title area in association with a level attribute indicating the extracted stage. 14. The document title extracting method of the document image processing device according to 14. 17. The title extraction processing for determining the plurality of stages of extraction parameters based on a value obtained by dividing the maximum value of the area average character size by the total average character size,
A document title extracting method for the document image processing apparatus according to claim 15. 18. The title extraction process for calculating a total average character size and an area average character size by using a trim average for calculating an average value of characters excluding a character larger than a predetermined ratio and a character smaller than the predetermined ratio. A document title extracting method for the document image processing apparatus according to claim 11. 19. The document title extracting method for a document image processing apparatus according to claim 11, further comprising a correction process for correcting the extracted character string in the title area. 20. The method according to claim 11, wherein the document image is a multi-page document image. 21. A document image generated by reading a paper document is divided into a plurality of regions, and a region average character size corresponding to an average character size and a total average character corresponding to an average character size of all regions are obtained for each region. A character size is calculated, and an extraction determination value obtained by multiplying the total average character size by an extraction parameter is compared with the region average character size, and a region having a region average character size larger than the extraction determination value is extracted as a title region. A recording medium on which a program to be recorded is recorded. 22. A document image processing apparatus comprising image input means for reading a paper document to generate a document image and storage means for storing the document image, wherein standard tag information is stored together with attribute values of the standard tag information. Standard tag information accumulating means, mark extracting means for extracting a specific mark added by a user on the paper document, and characteristic representing the characteristic of the mark based on the distribution of pixels constituting the specific mark Document image processing comprising: calculating means for calculating a value; and document tag information providing means for selecting specific standard tag information based on the attribute value and the characteristic value and providing the selected standard tag information to the document image. apparatus. 23. The document image processing apparatus according to claim 22, wherein said mark extracting means extracts said specific mark on a specific sheet. 24. The document image processing apparatus according to claim 23, wherein said mark extracting means determines a sheet with a two-dimensional code as said specific sheet. 25. The document image processing apparatus according to claim 22, wherein said mark extracting means extracts said specific mark on a margin of said paper document. 26. The document image processing apparatus according to claim 22, wherein said paper document is composed of a plurality of pages. 27. A document tag information assigning method of a document image processing apparatus for assigning document tag information of a document image generated by reading a paper document, wherein a standard tag information is stored together with an attribute value of the standard tag information. Tag information accumulation processing, mark extraction processing for extracting a specific mark added by a user on the paper document, calculation processing for calculating a numerical value representing the characteristic of the mark based on the distribution of pixels of the mark, A document tag information providing method for a document image processing apparatus, comprising: a process of selecting specific standard tag information based on an attribute value and a characteristic value and providing the selected standard tag information to the document image. 28. The method according to claim 27, wherein said paper document comprises a plurality of pages. 29. When a paper document is read to generate a document image, a specific mark added by a user on the paper document is extracted, and the characteristics of the mark based on the pixel distribution of the mark are represented. A recording medium in which a program for calculating a numerical value and selecting document tag information to be added to the document image from among the document tag information candidates based on the numerical value is recorded.