JP3526821B2 - Document search device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document search device and a recording medium for searching a keyword from a recognition result obtained by performing character recognition on a document image.

[0002]

2. Description of the Related Art Generally, when a paper-form document is stored in a document database, a paper-form document is read as image data, and the image data is subjected to character recognition to collect a set of character codes (character recognition). Result).
The document is stored in the document database as a set of character codes. When searching for a keyword from the document database, it is determined whether the keyword is included in the character recognition result. In commonly used character recognition, characters written in an original document (document in paper form) may not be correctly converted into a character code. Thus, when there is an error in character recognition, the character represented by the character code may be different from the character written in the original document. Therefore, a search omission may occur when searching for a keyword from a set of character codes accumulated in the document database. Missing search means that even if the original document has a character string that matches the keyword, when the keyword is searched from the character recognition results stored in the document database, the character string that matches the keyword is not detected. Say that.

As a conventional technique for preventing such a search omission, for example, the technique disclosed in Japanese Patent Laid-Open No. 7-152774 is known.

According to the conventional technique disclosed in JP-A-7-152774, at the time of search, using a list of similar characters in which a plurality of candidates are listed in advance for characters that are likely to cause character recognition errors among the characters included in the keyword, The expansion string is generated. A character whose character recognition is likely to be erroneous is, for example, a character whose shape is similar to that of the character.

A conventional technique disclosed in Japanese Patent Application Laid-Open No. 7-152774 will be described with reference to FIGS. 24A and 24B.

In FIG. 24A, the characters "book" and "mouth" contained in the original document are converted into character codes corresponding to the characters "tree" and "ku" having similar shapes due to an error in character recognition. An example of conversion is shown. The character recognition result is a set of character codes.
For the sake of explanation, in A, the character code is shown as a character corresponding to the character code. Although the original document contains the keyword "Japan",
When the keyword "Japan" is searched from the character recognition result, a search omission occurs.

FIG. 24B shows an example of a list of similar characters. In line 99-1, the character "book" is the character "tree",
It shows that it is apt to be mistakenly recognized as "large", "thick" and "aged". The line 99-2 indicates that the character "mouth" is apt to be erroneously recognized as the characters "□ (rectangle of symbol)", "times", "circle" and "every".

According to the conventional technique disclosed in Japanese Patent Laid-Open No. 7-152774, when the keyword "Japan" is searched, the search shown in FIG.
Using the list of similar characters shown in B, the expanded character strings “Hiki”, “Nichidai”, “Hita”, and “Nichizai” are generated. When searching for the keyword “Japan” from the character recognition result, each of the expanded character strings “Hiki”, “Nichidai”, “Hita”, and “Nichizai” is also used as a keyword.
As a result, in the character code, a portion of "Hiki" in which "Japan" is erroneously recognized as a character can be searched.

[0009]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
According to the related art disclosed in No. 4, if a character included in a document is erroneously recognized as a character not included in the list of similar characters, the problem of omission of search cannot be avoided.
For example, assume that the keyword “population” is searched from the character recognition result shown in FIG. 24A. The character "ku" in which the character "mouth" is erroneously recognized as a character is the line 99- in FIG. 24B.
It is not included in the list of similar characters for the character "mouth" shown in 2. Therefore, a search using the keyword "personal ward" is not performed, and a search omission occurs.

If the number of characters included in the list of similar characters is increased in order to reduce the possibility of omission of the search, the number of expanded character strings increases, and the cost (time and calculation amount) for the search increases. To do.

The present invention has been made in view of the above problems, and the cost (time and amount of calculation) required for retrieval is low, and the omission of retrieval due to character recognition error can be reduced. An object is to provide a document search device.

[0012]

A document search device of the present invention is a document search device for searching a keyword from a recognition result obtained by performing character recognition on an image of a document, wherein the keyword is at least A character code and a character image are assigned to each of the at least one first character, and the recognition result includes at least one second character; A character code and a partial area of the image of the document are assigned to each one of the second characters, and the document search device, based on the comparison of the character codes, determines that at least one of the second characters matches the keyword.
Determining whether or not one first matching portion is present in the recognition result, and if there is, first matching portion identifying means for identifying the at least one first matching portion;
It is determined whether or not at least one first portion satisfying the condition exists in a portion excluding the specified at least one first matching portion from the recognition result, and if there is, the at least one first portion. a first part specifying means for specifying a first portion, wherein the feature amount of the image of the partial area allocated to the second characters in the first portion, the character image of the first character included in the keyword At least one second matching portion that matches the keyword based on the comparison with the feature amount of
And a second matching portion identifying means for identifying the at least one second matching portion when it is present, and the predetermined first condition is:
A condition that there is the first portion in the vicinity of the specific second character having a width less than the predetermined width, thereby the objective described above being achieved. Other document retrieval device of the present invention
Is obtained by performing character recognition on the image of the document.
Document retrieval device for retrieving keywords from recognized recognition results
And the keyword is at least one first sentence
A character and that of said at least one first character
A character code and character image are assigned to each.
And the recognition result includes at least one second character.
And each of the at least one second character
The character code, the partial area of the image of the document, and the character
Indicates the certainty of character recognition obtained when recognition is performed.
The reliability is assigned, and the document retrieval device
Based on the comparison of the character codes,
At least one first matching part that matches the recognition result
It is judged whether or not it exists, and if it exists, the
First matching part identification for identifying at least one first matching part
Means and at least one first condition satisfying a predetermined first condition
At least one part of which is identified from the recognition result
Determine whether it exists in the part excluding the first matching part of
And, if present, the at least one first portion
And a first part specifying means for specifying
Of the image of the partial area assigned to the second character
Before the feature amount and the first character included in the keyword
Based on the comparison with the feature amount of the character image,
The at least one second matching part that matches the
The presence of at least one first part
And if present, the at least one second
A second matching portion identifying means for identifying a matching portion,
The predetermined first condition is that the assigned reliability is predetermined.
The first part in the vicinity of a particular second character smaller than a threshold
The purpose is to achieve the above objective.
Be done. The document retrieval device determines the image quality of the image of the document.
Means for determining the image quality based on the image quality of the determined image.
And a means for determining a predetermined threshold value.
Yes. The second matching portion identifying means is included in the first portion.
The character code of the second character is included in the keyword.
Whether it matches the character code of the specific first character
First determining means for determining, and the first part included in the first part
The character code of the second character is a special character included in the keyword.
If the character code of the fixed first character does not match,
At least the second character included in the first portion is included.
1 having the width closest to the width of the specific first character.
Or 2 or more consecutive second characters are identified as non-matching characters
Non-matching character specifying means and an image of the specified first character
And the 1 or 2 included in the non-matching character
1 or 2 or more assigned to the above consecutive second characters
The distance from the image feature of the area including the upper partial area is
If the value is less than the specified value, the specific first sentence
Second determining means for determining that a character matches the non-matching character
And may be provided. The document search device is
Calculate a predetermined criterion value from at least one first match
Based on the calculation means and the judgment reference value,
Of the at least one second matching part, the predetermined second condition is satisfied.
And a detection means for detecting the second matching portion that satisfies the condition.
May be. The calculating means includes the at least one
The at least one second character included in the first matching portion
Image features of at least one subregion assigned to
The determination reference value is calculated based on
Is included in the at least one second matching portion
At least one assigned to at least one second character
Of the feature amount of the image of one partial region and the determination reference value
Includes the condition that the distance is less than a predetermined value
But it's okay.

[0013]

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION First, a document filing system 210 for storing and retrieving documents will be described.

FIG. 1 illustrates a document filing system 210.
Shows the configuration of. The document filing system 210 includes an image input device 201 and an OCR (optical cha).
(racker reader) device 202, document database 203, document search device 204, and display device 2
Including 05 and.

The image input device 201 converts an original document (for example, a document in the form of paper) into document image data D _i . The image input device 201 is, for example, a scanner or a digital camera.

The OCR device 202 uses the document image data D _i
Perform character recognition on. The OCR device 202 includes
Known OCR techniques can be used. The result of character recognition by the OCR device 202 is OCR as the character recognition result D _c.
It is output from the device 202.

[0027] in the document database 203, the document data D _d is stored. The document data D _d includes the character recognition result D _c and the document image data D _i for the same document.

The document retrieval device 204 retrieves the keyword K _w from the character recognition result D _c contained in the document data D _d stored in the document database 203. The document search device 204 of the present invention uses the keyword K _w from the character recognition result D _c.
When searching for, the document image data D _i included in the document data D _d is used.

The document retrieval device 204 receives the character recognition result D _c.
When the keyword K _w is detected from the search result R
D _t is output to the display device 205.

The display device 205 displays the search result based on the search result RD _t . For example, the display device 205 is
The document image data D _i stored in the document database is displayed on the display, and the area corresponding to the keyword K _w in the area of the document image data D _i displayed on the display is highlighted. The highlighted display can be, for example, a colored display or a reverse display. The area corresponding to the keyword K _w is determined based on the search result RD _t .

Next, the data structure of the document image data D _i and the character recognition result D _c will be described.

FIG. 2A shows an example of the document image data D _i . The document image data D _i is, for example, bitmap image data.

FIG. 2B shows a data structure of a character recognition result D _c which is a result of performing character recognition on the document image data D _i shown in FIG. 2A. The character recognition result D _c is obtained as a set of characters D _c [j] (0 ≦ j ≦ N _d −1).
Here, N _d is the number of characters included in the character recognition result D _c . In the present specification, the numbers in [] indicate subscripts (indexes). The character code C _c is included in the character D _c [j].
[J], character coordinates (x ₁ [j], y ₁ [j]), (x
₂ [j], y ₂ [j]) and the reliability C _r [j] are assigned.

The character code C _c [j] corresponds to the OCR device 20.
It is the code determined by 2. Letter code C
_c [j] is a code represented by 2 bytes, for example. However, in the example shown in FIG. 2B, for the sake of explanation,
Instead of the character code, the character corresponding to the character code is shown.

Character coordinates are set to 1 by the OCR device 202.
A partial area in the document image data D _i recognized as one character is shown. This partial area is represented by, for example, a rectangle. The character coordinate (x ₁ [j], y ₁ [j]) is the coordinate of the upper left vertex of the rectangle, and the character coordinate (x ₂ [j], y ₂
[J]) is the coordinates of the lower right vertex of the rectangle. Any coordinate system may be used as the coordinate system of the character coordinates.

The reliability C _r [j] can be defined, for example, as the likelihood, the accuracy, or the certainty when the OCR device 202 executes the character recognition.

The reliability C _r [j] indicates whether the result of character recognition is likely to be correct or low. In the example shown in FIG. 2B, the reliability is represented by a number between 0 and 1, and the closer the reliability is to 1, the more likely the result of character recognition is.

Character recognition by the OCR device 202
And the characters written in the original document
Do not necessarily correspond one-to-one. With the OCR device 202
There is a possibility that a cutout error may occur during character recognition.
Because there is. The cut-out error is, for example, original
Recognizes one character in multiple documents as multiple characters
Or vice versa, multiple characters in the original document
Is recognized as one character. Figure 2A
And in the example shown in FIG. 2B, in the original document
The character "Lake" is cut out by the OCR device 202.
For three characters, "three", "old" and "month"
Has been recognized. Therefore, the original document
The letter "Lake" is a character recognition result D_cLetter D in_c[2]
~ Letter D _cIt corresponds to the three characters [4]. This
Character recognition result D_cThe letters inside are the original document
Characters that correspond to one character written in
Corresponding to a fragment of a character or a combination of multiple character fragments
In some cases, it may correspond.

The arrangement order of the characters in the character recognition result D _c is
It is the same as the arrangement order of the characters in the original document (for example, the order from the character located on the left side to the character located on the right side).

In the present specification, "character" includes not only characters of a particular language such as Chinese characters and alphabets but also all symbols to which a character code is assigned such as numbers and symbols.

[0041] "2 or more consecutive characters" is 2 or more characters D subscript j are continuous in character recognition result D _c
c [j] is meant.

Further, the "original document" is not limited to a document in paper form. The "original document" can be any written object.

The document retrieval device 204 of the present invention retrieves the keyword K _w from the character recognition result D _c having the data structure shown in FIG. 2B.

FIG. 2C shows the data structure of the keyword K _w . In the example shown in FIG. 2C, the keyword K _w is the character string “Lake Biwako”, and the keyword K _w consists of 4 characters. However, the number of characters of the keyword K _w is not limited to 4. The keyword K _w may include any number of letters greater than or equal to one. Each of the characters included in the keyword K _w is called a keyword character. The keyword character is represented as _Kw [i] using the subscript i. Here, 0 ≦ i ≦ 3. Generally, assuming that the number of characters included in the keyword K _w is N _k , the keyword K _w is K _w [0] to K.
_It is represented as a character string consisting of N _k keyword characters of _w [N _k -1].

Keyword characters K _w [0] to K _w [N _k −
A character code is assigned to each of [1]. For example, the character code “0x487c” (JIS code) is assigned to the keyword character K _w [0] (= “biwa”).

The search result by the document search device 204 is output to the display device 205 as the search result RD _t .

FIG. 2D shows the data structure of the search result RD _t . The search result RD _t shown in FIG. 2D shows the result of searching the keyword K _w (= “Biwako shore”) shown in FIG. 2C from the character recognition result D _c shown in FIG. 2B. The search result RD _t is N _r pieces of search location data RD _t [t] (0
≦ t ≦ N _r −1). Search location data RD
_t [t] is the character recognition result D _c (Fig.
The part (matching part) of B) that matches the keyword K _w is shown. N _r indicates the number of matching parts.

The search location data RD _t [0] has list elements 2241, list elements 2242, and list elements 2243.
And a list element 2244. The length of the list is equal to the length of the keyword K _w (the number of characters (keyword characters) included in the keyword K _w , in this case, 4). Each of the list elements 2241 to 2244 is
Four keyword characters K included in the keyword K _w
corresponding to _{w [0] ~K w [3} ], 1 or 2 or more consecutive characters in the character recognition result D _c. For example, in the list element 2241, the character D _c [0] is the keyword character K _w.
Indicates that it corresponds to [0] (“Biwa”). List element 2
In 242, the character D _c [1] is the keyword character K _w [1].
Indicates that it corresponds to ("Biwa"). List element 2243
Indicates that three consecutive characters D _c [2] to D _c [4] correspond to the keyword character K _w [2] (“lake”). List element 2244 indicates that the letter D _c [5] corresponds to the keyword K _w [3] (“bank”). The character code of each of the characters D _c [2] to D _c [4] does not match the character code of the keyword character K _w [2] (“lake”). Because the characters D _c [2] to D _c [4] are
This is because the OCR device 202 recognizes the characters “three”, “old”, and “month”. In the example shown in FIG. 2D, the characters “three” and “old” are used by the document search device 204.
Three consecutive characters D _c [2] recognized as "month"
.. D _c [4] are combined into one group, and the one group is determined to correspond to the keyword character K _w [2] (“lake”).

Search location data RD _t [t] (1≤t≤N _r
-1) is also a list having a length of N _k , similar to RD _t [0] described above. Here, N _k is the length of the keyword K _w .

The OCR device 202 executes character recognition for each processing unit. The processing unit is, for example, 1 of a document.
It may be a page or a paragraph.
Such a processing unit is called a "character block".

Usually, the character font and the character size are often constant in the character block. Therefore, it is preferable to execute the search with the character block as one unit from the viewpoint of improving the search accuracy.

Character D _c [j] included in character recognition result D _c
May be grouped by character block.
Such a group is referred to as character block data D _t and represents the result of performing character recognition within one character block.

FIG. 3 shows the structure of the character block data D _t .

The character block data D _t includes character block coordinates 2201, the number of characters 2202, and direction information 2203.
And a set of characters D _c [j] (D _c [0] to D _c [8] in the example shown in FIG. 3).

The character block coordinates 2201 indicate coordinate values in the rectangular document image data D _i circumscribing the character block.

The number of characters 2202 is the character block data D
Indicates the number of characters contained in _t .

The direction information 2203 indicates in which direction the characters are written in the character block (whether vertical writing or horizontal writing). For example, the value of the direction information 2203 is 1
Indicates vertical writing, and the value of the direction information 2203 is 0.
Indicates horizontal writing.

The character block data D _t may further include information on the font used in the character block.

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) FIG. 4 shows the configuration of a document retrieval apparatus 1 according to Embodiment 1 of the present invention. Document retrieval device 1
Is a text search means 101 and a character identification means 102.
Character shape search means 103 and character width estimation means 104
Including and

The document search device 1 can be used as the document search device 204 shown in FIG. In this case, the document search device 1 searches the document data D _d stored in the document database 203 for the keyword K _w . Document data D _d
Includes the document image data D _i and the character recognition result D _c .

In the following description, it is assumed that the document is written horizontally (that is, the characters in the document are arranged from left to right). The procedure of the document search process described below can be applied to a vertically written document by replacing the character width and height with the character height and width, respectively. Whether the document is written vertically or horizontally can be determined, for example, by referring to the direction information 2203 (FIG. 3) included in the character block data D _t .

The keyword K _w is input from the outside of the document search apparatus 1 to the document search apparatus 1, and an instruction to search the specific document data D _d for the keyword K _w is given. The input of the keyword K _w and the instruction of the search are performed by an input means (not shown) such as a keyboard.

The text search means 101 uses the keyword K.
_The character code assigned to the keyword character (first character) included in _w is compared with the character code assigned to the character (second character) included in the character recognition result D _c , and if these character codes match. For example, it is determined that the second character matches the first character. Let N _{k be} the length of the keyword K _w , and let each of the N _k consecutive characters in the character recognition result D _c be
If it matches a corresponding one of the N _k keyword characters in the keyword K _w , the text search means 101
N _k consecutive characters in the character recognition result D _c are specified as a matching part (first matching part) that matches the keyword K _w . Such a matching portion (first matching portion) may exist in any number of 1 or more in the character recognition result D _c . The text search means 101 outputs one or more matching parts (first matching part) as the first search result RD _t1 . The number of matching parts (first matching part) included in the first search result RD _t1 is N _r1 . The first search result RD _t1 includes detection point data RD _t1 [t]. Here, 0 ≦ t ≦ N _r1 −1. The detection location data RD _t1 [t] represents one of the one or more first matching parts identified by the text search means 101. The detection point data RD _t1 [t] is
It is a list including the same number of list elements as the number of keyword characters included in the keyword K _w . Each list element is
Represents a character having a character code that matches the character code of the corresponding keyword character.

As described above, the text search means 101
Based on the character code comparison, the number of matching keywords
At least one matching part (first matching part) is the character recognition result.
Fruit D _cIs present in the
A first match identifying the at least one first match
It functions as a part specifying means.

If there is no error in the character recognition by the OCR device 202 (FIG. 1), the text search means 101 is sufficient for searching the keyword from the recognition result. However, as already mentioned, the OCR device 20
If there is an error in the character recognition in 2 (FIG. 1), there is a possibility that a search omission may occur only by the text search means 101 based on the comparison of the character codes. That is, there is a possibility that there is a portion that matches the keyword K _{w in} the portion obtained by removing the matching portion (first matching portion) specified by the text search means 101 from the character recognition result Dc.

The character specifying means 102 determines the character recognition result D _c.
It is determined whether or not there are one or more characters satisfying a predetermined condition in the portion excluding the one or more matching portions (first matching portion) specified by the text search means 101 from.
If there is, the character identifying means 102 selects one or more characters satisfying the predetermined condition as the candidate portion SD _c (first portion).
Specify as. Here, the “predetermined condition” includes, for example, the condition (1) below.

[0068] (1) "is the character in the character recognition result D _c, it is present in the vicinity of the particular character which has a width less than the predetermined width" or "predetermined conditions"
Instead of the condition (1), or in addition to the condition (1), the condition (2) may be included.

[0069] (2) included in "is the character in the character recognition result D _c, present in the vicinity of the particular character with less confidence than a predetermined confidence" Generally, in the character recognition result D _c The smaller the width of the character, the higher the possibility that a character cutting error will occur in character recognition.
Further, the lower the reliability of the character included in the character recognition result D _c , the higher the possibility that a character recognition error will occur in character recognition. Therefore, these particular characters (ie
Characters with small width or characters with low reliability)
There is a high possibility that a character recognition error has occurred in the vicinity of.
The specific meaning of "nearby" will be described later with reference to FIG.

When the first matching portion does not exist in the character recognition result D _c , the character identifying means 102 determines whether or not one or more characters satisfying a predetermined condition exist in the character recognition result D _c. To determine. When one or more characters satisfying the predetermined condition are present in the character recognition result D _c , the character identifying means 102 sets the one or more characters satisfying the predetermined condition as the candidate portion SD _c (first portion). Identify.

As described above, the character specifying means 102 determines that at least one candidate portion SD _c (first
Part) is the text search means 10 from the character recognition result D _c.
It determines whether or not it exists in a portion other than the at least one first matching portion specified by 1, and if it exists, it functions as a first portion specifying means for specifying the at least one first portion. .

The character shape search means 103 has at least 1
When one candidate portion SD _c (first portion) is specified, the feature amount of the image of the partial area assigned to the character (second character) included in the candidate portion SD _c and the keyword K _w.
It is determined whether or not there is a matching portion (second matching portion) that matches the keyword K _w in the candidate portion SD _c , based on a comparison between the keyword character (first character) included in the image feature amount. . The detailed configuration and operation of the character shape search means 103 will be described later with reference to FIG. 7.

Such a matching portion (second matching portion) is
There can be any number greater than or equal to 1 in the candidate portion SD _c . The character shape search unit 103 outputs one or more matching parts (second matching part) as the second search result RD _t2 .
The number of matching parts (second matching part) included in the second search result RD _t2 is N _r2 . The second search result RD _t2 includes detection point data RD _t2 [t]. Where 0 ≦ t ≦ N _r2 −
It is 1. The detection location data RD _t2 [t] represents one of the one or more second matching parts specified by the character shape searching unit 103. Detection point data RD
_t2 [t] is a list including the same number of list elements as the number of keyword characters included in the keyword K _w . Each list element is one or more consecutive characters having an image feature amount whose distance from the image feature amount of the corresponding keyword character is a predetermined threshold Thd ₁ or less.

The character width estimation means 104 uses the keyword K _w.
Width of each keyword character included in K _ww [i]
Estimate (0 ≦ i ≦ N _k −1). The width K _ww [i] of the keyword character is used by the character shape searching unit 103. The character width estimation means 104 also estimates the width K _ww of the keyword K _w . The width K _ww of the keyword K _w is used by the character specifying unit 102.

Search result R by the text search means 101
D _t1 and the search result RD _t2 by the character shape search means 103.
Is output from the document search device 1 as a result of searching the keyword K _w from the character recognition result D _c .

In FIG. 5, the character width estimating means 104 causes the character width K _ww [i] of each character K _w [i] included in the keyword K _w .
An example of estimating

In the example shown in FIG. 5, it is assumed that the keyword K _w is "declining birthrate". In this case, an example will be described in which the character width of the character “” (= K _w [2]) included in the keyword K _w is estimated.

The character width estimating means 104 determines the character recognition result D.
Among the characters included in _c , the character having the maximum height is obtained, and the height of this character is set as the estimated value a of the character height. The height of the character included in the character recognition result D _c can be calculated from the character coordinates assigned to the character.

Alternatively, the height of the character having the maximum height among the characters in the character block data D _t may be used as the estimated value a of the character height.

Alternatively, the character height estimation value a may be a mode value, an average value, or a median value instead of the maximum character height value. The unit of the character width and the character height is, for example, pixel.

The character width estimating means 104 uses the standard height b and the standard width c of the character "ka" to convert the character "ka" (=
The width K _ww [2] of K _w [2]) is calculated by ( _Equation 1).

[0082]

[ _Equation 1] K _ww [2] = a · c / b The standard height b and standard width c of the character “ka” are the characters “ka” using the font of the document retrieval apparatus 1. Image (character image KC
_i [2]) and obtain the height and width of the character image KC _i [2]. Alternatively, the ratio b / c of the standard height b of the character and the standard width c may be determined in advance for all the characters. Character block data D
_{If t} has information of the font used in the character block, the same font as that font may be used when generating the image of the character “”.

FIG. 6 shows a candidate portion SD satisfying a predetermined condition.
An example of a process for identifying _c will be shown. This processing is executed by the character identifying means 102.

[0084] keyword K _w is "low birth rate", a character string in the character recognition result D _c "... declining birthrate torquecontrol of the problem ...
・ " In FIG. 6, for the sake of explanation, the characters in the character recognition result D _c are represented as the characters corresponding to the assigned character code. Further, here, the “predetermined condition” is assumed to be the condition (2) already described.

Further, of the reliability assigned to each character in the character recognition result D _c , the reliability of the characters “a” and “hi” is lower than a predetermined threshold value Thr, and the other values are threshold values. It is assumed to be higher than Thr.

The character width estimation means 104 determines the width K _ww of the keyword K _w . The width K _ww of the keyword K _w is
It is calculated as the sum of the widths of the respective characters included in the keyword K _w .

When the character specifying means 102 specifies the candidate portion SD _c , first, the character "a" whose reliability is lower than the threshold value is used.
Was an end point (right end), and a width of K _ww equal range A (the part of "low birth rate I"), the "I" and the starting point (the left end), a width of K _ww
And a range B (part of "question of Ihi") equal to. As described above, the vicinity of a specific character means a range in which the width set to the left and right of the specific character (in this case, the character “a”) is equal to K _ww .

[0088] If there is a range more reliability is lower than the threshold Thr characters in B ( "human"), the start point of the character "human", width seek range C equal to K _ww. Range A, B
The character included in any one of C and C is a candidate portion SD _c
Specified as.

FIG. 7 shows a detailed structure of the character shape search means 103.

The character shape search means 103 includes a character image extraction means 301, a character image table 302, a shape matching means 303, and a matching control means 304.

The character image extracting means 301 uses the keyword K.
1 or 2 with a width that is closest to the width of a particular character in _w
The above consecutive characters are specified as non-matching characters, and the area of the document image (that is, the character image) corresponding to the non-matching characters is extracted.

The character image table 302 stores in advance images of characters included in the keyword K _w . The image of this character is, for example, a bitmap font.

The shape matching means 303 compares the feature amount of the image of the area extracted by the character image extracting means 301 and the feature amount of the image of the character included in the keyword K _w stored in the character image table 302. The two images are compared to determine whether or not the two images are similar.

The collation control means 304 controls the operation of the character shape search means 103.

The operation of the character shape retrieval means 103 is shown in steps S101 to S103 below. This processing is executed by the collation control means 304. Here, the candidate portion S
Let D _{c be} the set of characters SD _c [j]. Each character SD _c [j] has the same data structure as the character described with reference to FIG. 2B. Note that step S101
In S103, the variable j represents the subscript of the character SD _c [j], and the variable i represents the subscript of the keyword character K _w [i].

Character recognition result D_cSome candidate parts
SD_cCan exist. Character recognition result D _cMultiple candidates in
Partial SD_cIs present, each candidate part S
D_cProcessing of steps S101 to S103 below
Is done.

Step S101: Variable j is variable star
t_j is substituted, and the value “0” is substituted for the variable i. The variable start_j is a subscript of a character located at the start end (left end) of the candidate portion SD _c . Further, in step S101, a list having a length equal to the length N _k of the keyword K _w is prepared as detection point data.

Step S102: One or more consecutive characters that should be associated with the keyword character K _w [i] are identified as “mismatched characters” from the candidate portion SD _c , and the image C _i is extracted. This processing is executed by the character image extracting means 301. Details of this processing will be described later with reference to FIG. The image C _i is an image of a partial area assigned to one or more consecutive characters specified as a non-matching character. Also, the keyword character K _w
The image KC _i of [i] is obtained from the character image table 302.

Step S103: Keyword letter K
_It is determined whether _w [i] and the non-matching character specified in step S102 match. In order to make this determination, first, the shape matching means 303 is used to match the image C _i with the image KC _i . The image matching is performed by comparing the feature amounts of the images. Image C _i
The Euclidean distance between the feature amount of the image and the feature amount of the image KC _i is smaller than the predetermined threshold Thd ₁ , which indicates that the image C _i and the image KC _i are similar to each other. If the image C _i and the image KC _i are similar, it is determined that the unmatched character specified in step S102 matches the keyword character K _w [i].

The keyword character K _w [i] and step S
When it is determined that the non-matching character specified in 102 matches, the character S is added to the i-th position of the detection location data.
D _c [j] is registered, the variable i is incremented by 1, the variable next_j is substituted for the variable j, and the process returns to step 102. This is the next character in the keyword, the candidate part SD
Means to search from the part to the right of the matched character in _c . The variable next_j is a candidate part SD _c.
Indicates the subscript of the character adjacent to the right of the character in which the matching is completed. The value of the variable next_j is determined by the character image extracting means 301 in step S102.

The keyword character K _w [i] and step S
When it is determined that the non-matching character specified in 102 does not match, the value “0” is assigned to the variable i, the variable start_j + 1 is assigned to the variable j, and the variable start_
The variable j is substituted for j, and the process returns to step 102. This means that the character of interest in the candidate portion SD _c is shifted to the right by one character and the first keyword character K _w [0] is searched again.

If the image KC _i of all the keyword characters included in the keyword K _w is similar to the corresponding image C _i by the processing shown in steps S101 to S103 above, the keyword characters included in the keyword K _w become The character in the corresponding candidate portion SDc is specified as the matching portion (second matching portion).

As described above, the character shape search means 103
Comparison of the feature amount of the image of the partial area assigned to the character (second character) included in the candidate portion SD _c (first portion) and the image feature amount of the keyword character (first character) included in the keyword Based on the above, at least one second matching part that matches the keyword K _w is the character identifying means 102.
At least one candidate part SD identified by
_It determines whether or not it exists in _c (first part), and if it exists, it functions as a second matching part specifying means for specifying the at least one second matching part.

FIG. 8 shows a candidate portion SD _c corresponding to a specific character in the keyword K _w in the character image extracting means 301.
An example in which the characters inside are specified is shown.

Keyword K_wIs "declining birthrate"
Image data D_iWrites, "... the problem of declining birthrate ..."
Suppose there is a part. Now, let's shine until "the little child"
If the match is over, then the keyword K_wSentence in
Character "ka" (= K_w[2]) is collated. Document image
The “” in the text indicates that the character recognition in the OCR device 202 is off.
Due to a mistaken protrusion, the character fragment "a" and the character fragment
It is divided into "hi". Into each character fragment
Is the candidate part SD_cCharacter SD in_c[J] and the character SD _c
It corresponds to [j + 1]. Rectangle 1310, 1311
And 1312 are the characters SD_c[J], SD
_c[J + 1] and SD_cSentence assigned to [j + 2]
Calligraphy image data D_iIt is a partial area inside.

The width of the rectangle 1310 is w _1, and the width of the rectangle 131
The width of a region (rectangle 1313) including 0 and a rectangle 1311 is w _2, and the width of a region including rectangles 1310 and 1312 is w ₃ .

[0107] keyword K When performing the collation of the character "of" in the _w, character "of" (= K _w [2]) and the character width estimated value K _ww [2] for, width w ₁ ~ width w ₃ Are compared with each. Character width estimated value K _ww [2] of width w ₁ to width w ₃
If the one having the value closest to is the width w ₂ ,
The character SD _c [j] and the character SD _c [j + 1] are identified as non-matching characters.

Further, from the document image data D _i , the rectangle 13
The image in 13 is extracted as the character image C _i . In addition, the two characters (character SD
The number “j + 2” of the character (the character on the right) next to the character _c [j] and the character SD _c [j + 1]) is determined as the value of the variable next_j.

FIG. 9 shows an example of a method for obtaining the characteristic amount (vector amount) from the character image C _i .

The character image C _i of the character "A" is divided into 16 blocks B [i] (0≤i≤15). Each block is numbered from upper left to lower right. i
= 0 to 15, the black pixel density of the block B [i] (the number of black pixels in the block divided by the area of the block) is calculated, and 16 values are obtained. A 16-dimensional vector amount having these 16 values as components is set as a feature amount.
The feature amount thus obtained indicates the feature of the shape.

As described above, according to the document search apparatus 1 of the present invention, after the search based on the character code by the text search means 101, there is a high possibility that a search omission occurs in the search by the text search means 101. The search is further performed based on the image feature amount. This can reduce the omission of search.

Further, since the target of the search based on the comparison of the image feature amounts is limited to the part where the search omission is likely to occur, the cost (time and calculation amount) required for the search can be low.

In the above description, in the search based on the character code in the text search means 101, the character code of the character in the character recognition result D _c and the character code of the keyword character in the keyword K _w are simply compared. Was there. However, each of the characters in the character recognition result D _c holds a plurality of candidate characters in the character recognition processing by the OCR device 202, and any one of the character codes of the plurality of candidate characters and the specific keyword in the keyword Kw are stored. if there is a match and the character code may be determined that a specific keyword character and in keyword K _w in the character recognition result Dc match. Alternatively, when a specific character in the keyword K _w is expanded into a plurality of similar characters and one of the character codes of the plurality of similar characters matches the character code of the character in the character recognition result D _c , It may be determined that the character in the character recognition result D _c matches the specific keyword character in the keyword K _w .

As the search by the text search means 101, an ambiguous search using a synonym dictionary such as a thesaurus may be used, or another known search method may be used.

The document data D _d may be in the index format.

The character image table 302 can use data in which character codes and character images are paired like font data. However, character images are collected for each character code from actual document image data. Alternatively, the average value of the feature amount for each character code may be used. Further, when there is a search result with a high degree of certainty with the same keyword (for example, the search result RD _t1 of the text search means 101), a pair of a character code and a character image is generated from the search result, and a character image table 302 is created. If the search result does not exist, the font data may be used.

The character width estimating means 104 can be omitted. When the character width estimation means 104 is omitted, the same value (for example, the character height estimation value a) may be used as the character width estimation value regardless of the character.

After the processing by the character specifying means 102, the processing of integrating / re-dividing the characters included in the candidate portion SD _c may be performed. In order to realize this process, a character subdivision unit may be added to the document search device 1. The character subdivision means is a candidate part SD _c identified by the character identification means 102.
The characters inside are integrated, and the image corresponding to the integrated character is extracted from the document data D _d . The extracted image is divided into as fine pieces as possible. When the document is written horizontally, for example, this division is performed in a portion where the projection histogram of black pixels in the vertical direction is obtained and the projection histogram is smaller than a predetermined threshold value. The portion thus divided into as much as possible is called a subdivision element. The subdivision element is
Generally, it is an element whose size is equal to or smaller than the character in the candidate portion SD _c . When determining the character width of the non-matching character in the character image extraction means 301, the candidate portion SD
_{Although the} character width is variable for each character in _c , the character width may be variable for each subdivision element. By using the subdivision element in this way, even when the character cutout position is incorrect at the time of character recognition in the OCR device 202, it is possible to specify an appropriate cutout position based on the character width of the keyword character. Become.

Hereinafter, some variations of the document search device 1 according to the first embodiment of the present invention will be described with reference to the drawings.

FIG. 10 shows the configuration of a character shape search means 103a as a variation of the character shape search means 103.

The character shape searching means 103a can be used in place of the character shape searching means 103 shown in FIG.
10, the same components as those shown in FIG. 7 are designated by the same reference numerals, and the description thereof will be omitted.

The character shape searching means 103a includes a matching control means 606 and a similar character matching means 602.

The collation control means 606 controls the operation of the entire character shape search means 103a.

The similar character collating means 602 determines the candidate portion SD.
It is determined whether the character code C _c [j] of the character included in _c matches the character code of the keyword character K _w [i]. Alternatively, the character code C _c [j] has a similar character list (see FIG. 24) for the keyword character K _w [i].
It may be determined whether or not it matches the character code of any of the characters included in B). Such a similar character list is prepared in advance for all characters and is a list of characters that are likely to be erroneously recognized.

The operation of the character shape retrieval means 103a will be described below.
This is shown in steps S301 to S303. This process is a collation
It is executed by the control means 606. Where the candidate part
SD _cIs the character SD_cLet it be a set of [j]. That's it
This character SD_c[J] is the sentence described with reference to FIG. 2B
It has the same data structure as a character. Note that S301 to S3
In 03, the variable j is the character SD_cShow the subscript of [j]
The variable i is the keyword letter K_wShow subscript of [i]
You

Character recognition result D_cSome candidate parts
SD_cCan exist. Character recognition result D _cMultiple candidates in
Partial SD_cIs present, each candidate part S
D_cThe following steps S301 to S303
Is done.

Step S301: Variable j is variable star
t_j is substituted, and the value “0” is substituted for the variable i. The variable start_j is a subscript of a character located at the start end (left end) of the candidate portion SD _c . Further, in step S301, a list having a length equal to the length N _k of the keyword K _w is prepared as detection point data.

Step S302: The character SD _c [j] is
Whether or not it matches the keyword character K _w [i] is determined by the similar character collating means 602 based on the comparison of the character codes.

If they match, the character SD _c [j] is registered at the i-th position of the detected position data, both the variable i and the variable j are incremented by 1, and step S302 is repeated. This means that the next character in the candidate portion SD _c will be matched with the next keyword character.

If they do not match, the process proceeds to step S303.

As described above, the similar character collating means 602 causes the character (second character) included in the candidate portion SD _c (first portion).
Function as a first determination unit that determines whether or not the character code of No. 1 matches the character code of the specific keyword character K _w [i] (first character) included in the keyword.

Step S303: One or more consecutive characters that should be associated with the keyword character K _w [i] are identified as “mismatched characters” from the candidate portion SD _c , and the image C _i thereof is extracted. . This processing is performed by the character image extracting means 301 and has already been described with reference to FIG. Further, the image KC _{i of} the keyword character K _w [i] is obtained from the character image table 302.

Next, the shape matching means 303 determines whether or not the keyword character K _w [i] matches the non-matching character. In order to make this determination, first, the shape matching means 3
03 is used to collate the image C _i with the image KC _i . The image matching is performed by comparing the feature amounts of the images. The Euclidean distance between the feature amount of the image C _{i and} the feature amount of the image KC _i being smaller than the predetermined threshold Thd ₁ indicates that the image C _i and the image KC _i are similar to each other. If the image C _i and the image KC _i are similar, it is determined that the non-matching character specified in step S303 matches the keyword character K _w [i].

When it is determined that the keyword character K _w [i] matches the non-matching character, i of the detected position data is detected.
1 or 2 identified as the non-matching character in the th position
The above consecutive characters are registered, the variable i is incremented by 1, the variable next_j is substituted for the variable j, and the process returns to step S302. This means that the next character in the keyword is searched for in the part adjacent to the right of the character in the candidate part SD _c that has been matched. Variable next_j
Indicates the subscript of the character adjacent to the right of the character in the candidate portion SDc that has been collated. The value of the variable next_j is determined by the character image extracting means 301 in step S303.

When it is determined that the keyword character K _w [i] does not match the unmatched character, the variable i has the value “0”.
Is assigned, the variable start_j + 1 is assigned to the variable j, the variable j is assigned to the variable start_j, and the process returns to step 302. This means that the character of interest in the candidate portion SD _c is shifted to the right by one character and the first keyword character K _w [0] is searched again.

As described above, the character image extracting means 301
Characters SD _c [j] included in the candidate part SD _c (first part)
If the character code of (second character) does not match the character code of the specific keyword character K _w [i] included in the keyword K _w , the character SD included in the candidate portion SDc (first portion) _It functions as a non-matching character specifying means that specifies at least one continuous character having at least the width of the keyword character K _w [i] and including _c [j] (second character) as a non-matching character.

Further, the shape matching means 303 detects the feature quantity of the image of the specific keyword character K _w [i] (first character),
The distance from the feature amount of the image of the area including one or more partial areas assigned to one or more consecutive characters (second character) included in the non-matching character is a predetermined threshold T.
a particular keyword character K if less than hd _1
w [i] functions as a second determination unit that determines that the mismatched character matches.

Steps S301 to S303 described above
Compared with the processing procedure shown in steps S101 to S103, the processing procedure shown in (1) is performed before the matching processing based on the comparison of the image feature amounts (step S303), and based on the comparison of the character code (step S302). ) Has been added. By the matching process based on the comparison of the character codes, the character SD _c [j] in the candidate portion SD _c becomes the keyword character K.
If it is determined that the matching _w [i], the character SD _c in the candidate portion SD _c [j], the matching processing based on the comparison of the feature amount of the image with the keyword character K _w [i] is not performed. In general, the matching process based on the comparison of character codes can be performed faster than the matching process based on the comparison of image feature amounts.
By performing the processing procedure shown in steps S301 to S303, it becomes possible to improve the processing speed of the search in the document search device 1.

If the reliability C _r [j] of the character SD _c [j] in the candidate portion SD _c is higher than a predetermined threshold value in step S303, the shape matching means 303 omits the matching. Good. Because the reliability C _r [j] of the character SD _c [j] is higher than the predetermined threshold, it is OCR.
This is because there is a high possibility that the character recognition in the device 202 has been performed correctly, and there is less need to compare the feature amounts of the images.

FIG. 11 shows the configuration of a document search device 701 as a variation of the document search device 1. The document search device 701 is, for example, the document search device 1 shown in FIG.
Can be used instead of.

In FIG. 11, the same components as those shown in FIG. 4 are designated by the same reference numerals, and the description thereof will be omitted.

The document search device 701 is provided with the character specifying means 10
The search precision control means 705 which produces | generates the threshold value Thr used in 2 is included.

The document data D _d input to the document search device 701 has quality information. Quality information is
It is a value related to the image quality of the document image data D _i , and represents, for example, the resolution of the document image data D _i , the degree of character blurring, and the degree of character collapse. The quality information is represented by, for example, a value between 0 and 1, and the larger the value, the better the image quality of the document image data D _i .

The search accuracy control means 705 outputs the threshold value Thr according to the quality information of the document data D _d based on a table in which the relationship between the quality information and the threshold value Thr is determined in advance. As the relationship between the predetermined quality information and the threshold Thr, for example, the relationship that the quality information is equal to the threshold Thr can be used.

Generally, when the image quality of the document image data D _i is good, it is expected that the character recognition in the OCR device 202 will be performed correctly. Therefore, the document image data D
By adjusting the threshold Thr according to the image quality of _i , it is possible to adjust the number of candidate portions SD _c specified by the character specifying means 102. As a result, good quality document data D
_With respect to _d, it is possible to prevent the number of processing targets (the number of candidate portions SD _c ) of the character shape search unit 103 from increasing more than necessary,
The processing time can be shortened and excessive detection can be suppressed. Also,
For document data D _d of poor quality, the OCR device 2 is increased by increasing the processing targets of the character shape search means 103.
It is possible to reduce the omission of search due to a clipping error or a recognition error in character recognition in 02.

FIG. 12 shows the configuration of a document search device 801 as a variation of the document search device 701. The document search device 801 can be used, for example, instead of the document search device 701 shown in FIG.

In FIG. 12, the same components as those shown in FIG. 11 are designated by the same reference numerals, and the description thereof will be omitted.

The document search device 801 receives the character recognition result D _c.
Document data D _d from the reliability assigned to the middle character
Quality information extraction means 805 for obtaining quality information about

Reliability C_rCharacter recognition is correct for [j]
It reflects the probability, and documents with high quality information are written
It is considered that the probability of correct recognition is high. Therefore trust
Degree C _rThe quality information can be obtained from [j].

The quality information extraction means 805 can obtain the quality information as an average value of the reliability C _r [j] of all the characters included in the character recognition result D _c of the document data D _d .

According to the configuration shown in FIG. 12, the quality information can be objectively obtained from the reliability of the characters included in the document data D _d .

FIG. 13 shows the configuration of a document search device 901 as a variation of the document search device 701. The document search device 901 can be used instead of the document search device 701 shown in FIG. 11, for example.

In FIG. 13, the same components as those shown in FIG. 11 are designated by the same reference numerals, and the description thereof will be omitted.

The document retrieving apparatus 901 includes the character specifying means 10
2 includes a search precision designating unit 905 that allows the user to designate the threshold Thr used in 2.

The search precision designating means 905 allows the user to designate the threshold Thr according to the purpose. When the user wants to know as many correct detection points as possible without being willing to increase the number of excessive detection points, the threshold Thr may be increased. If the user thinks that it is sufficient to know one correct detection point and does not want to increase the number of excessive detection points, the threshold Thr may be reduced. Excessive detection points are the points that do not match the keywords in the original document, but are detected as matching parts by the document search device.

As described above, according to the document search device 901, it is possible to search according to the intention of the user.

FIG. 14 shows a configuration of a document search device 1151 as a variation of the document search device 1. The document search device 1151 can be used instead of the document search device 1 shown in FIG.

In FIG. 14, the same components as those shown in FIG. 4 are designated by the same reference numerals, and the description thereof will be omitted.

The document retrieval device 1151 includes a similar character string averaging means 1102 for obtaining an average value of image feature amounts from the detection point data included in the first matching portion set RD _t1 and a similar character string averaging means 1102. And a character string re-detection means 1103 for refining the detection point data included in the set RD _t2 of the second matching parts by using the obtained average value of the feature amount of the image.

It is considered that the number of excessive detection points included in the search result RD _t1 in the text search means 101 is smaller than the number of excessive detection points included in the search result RD _t2 in the character shape search means 103. . Therefore,
It is highly possible that the detection location data included in the search result RD _t1 truly matches the keyword.

The similar character string averaging means 1102 refers to the detected position data included in the search result RD _t1 and extracts the character image C _i of the partial area of the document image data D _i corresponding to each keyword character. The character image C _i shows how the keyword character is written in the original document. Similar character string averaging means 1102
Further calculates the feature amount of the extracted character image C _i ,
The feature amounts are averaged for the same keyword character. This averaged feature amount is used as the character string re-detection means 11
In 03, it is used as a criterion value.

As described above, the similar character string averaging means 110
2 functions as a calculation unit that calculates a predetermined determination reference value from the search result RD _t1 (at least one first matching portion).

There is a possibility that the detection position data included in RD _t2 , which is the search result by the character shape searching means 103, includes an excessive detection position. This is because there is a possibility that the character shape searching means 103 may be searching using a character image different from the font used in the original document.

The character string re-detection means 1103 carries out a process of sieving such detection points by using the averaged feature amount (judgment reference value) calculated by the similar character string averaging means 1102.

The character string rediscovery means 1103 retrieves the search result R.
The character image C _i of the partial area of the document image data D _i corresponding to each keyword character is extracted by referring to the detection point data included in D _t2 . Next, the distance between the feature amount of the extracted character image C _i and the determination reference value is a predetermined threshold Thd ₂
The detection point data that is smaller than the above is detected.

In this way, the character string re-detection means 1103
Functions as a detection unit that detects a second matching portion satisfying a second predetermined condition in the search result RD _t2 (at least one second matching portion) based on the determination reference value.

The detection location data detected by the character string re-detection means 1103 as described above is output as a new set of detection location data.

It should be noted that, using the averaged feature amount (judgment reference value) calculated by the similar character string averaging means 1102,
If the document data D _d is searched again, the accuracy of the search can be improved.

As described above, according to the document retrieval apparatus 1151, among the detected location data included in the relatively unreliable retrieval result RDt ₂ , the detected location data satisfying a predetermined condition based on a certain criterion value. By detecting, it is possible to suppress the number of excessive detection points.

Further, the multi-stage search means indicated by reference numeral 1101 in FIG. 14 is assumed to be the document search apparatus 1 shown in FIG. 4 in the above-mentioned example. However, the multi-stage search means 1
The configuration of the document search device that can be adopted as 101 is not limited to this. As the multi-stage search means 1101, an arbitrary document search device capable of controlling the number of excessive detection points included in the search result stepwise can be adopted.

The search results obtained by the multi-stage search means 1101 are search results RD _t1 , search results RD _t2 , ...
, Search result RD _tn , and of these, the search result with the smallest number of excessive detection points included is the search result R
If it is D _t1 , the similar character string averaging means 1102 calculates a judgment reference value based on the search result RD _t1 .

Alternatively, instead of the multi-step search means 1101, any document search device that does not control the number of excessive detection points included in the search results stepwise may be adopted. Such an example is shown in FIG.

FIG. 15 shows the structure of a document search device 1051 as a variation of the document search device 1151.
The document search device 1051 can be used instead of the document search device 1151 shown in FIG.

In FIG. 15, the same components as those shown in FIG. 14 are designated by the same reference numerals, and the description thereof will be omitted.

The document retrieving apparatus 1051 is comprised of retrieving means 100.
1 and a similar character string averaging means 1002.

As the search means 1001, an arbitrary document search device can be adopted. The search result in the search means 1001 is output as RD _t .

The similar character string averaging means 1002 refers to at least one detection point data similar to the keyword in the search result RD _t , and refers to the character of the partial area of the document image data corresponding to each keyword character. Image C
Extract _i . The character image C _i shows how the keyword character is written in the original document. The similar character string averaging means 1002 further calculates the feature amount of the extracted character image C _i and averages the feature amount for the same keyword character. The averaged feature amount is used as a determination reference value in the character string redetection unit 1103.

The processing for the similar character string averaging means 1002 to obtain at least one detection point data similar to the keyword in the search result RD _t is performed as follows.

First, each keyword character contained in the keyword K _w is imaged to obtain a character image KC _i . When the character block data D _t has information on the font used in the character block, the same font as that font can be used when generating the image of the keyword character.

Character image KC_iThe feature quantity is calculated from
It Here, as the feature amount, for example, in character recognition,
Features that are used by the user and the features that were described with reference to FIG.
Can be adopted. Next, the key word in the detected location data
De K_wCharacter recognition result D corresponding to each character of_cCharacter position in
Document image data D by referring to the placement information_iFrom character image C _i
Is extracted. The character image C_iFrom the feature amount,
Be done. Character image KC_iCharacteristic amount obtained from
Image C_iEuclidean distance from the feature value obtained from
Is calculated. This Euclidean distance is the keyword K_w
Add all the characters included in and add the sum to the key
K_wThe value divided by the number of characters included in
Keyword K_wIs defined as the distance from. Detection point data
Keyword is K_wIs similar to
Data and keyword K_wMeans a small distance from
It

As described above, the distance from the keyword K _w for each detection point data of the search result RD _t is obtained, and a predetermined number of detection point data are selected in order from the smallest distance. In this way, at least one detection position similar to the keyword is obtained.

(Second Embodiment) FIG. 16 shows the configuration of a document search device 451 according to the second embodiment of the present invention.

The document search device 451 includes a wild card search means 401, a character shape search means 402, a character width estimation means 403, and a character image table 404.

The document search device 451 can be used as the document search device 204 shown in FIG. In this case, the document search device 451 searches for the keyword K _w from the document data D _d stored in the document database 203. The document data D _d includes the document image data D _i and the character recognition result D _c .

The wildcard search means 401 determines the character code assigned to the character (first character) included in the keyword K _w and the character code assigned to the character (second character) included in the character recognition result D _c. Are compared, the part where at least one character matches is specified, and this is searched result RD
Output as _t1 . The number of matching parts included in the search result RD _t1 is N _r1 . The search result RD _t1 includes detection location data RD _t [t]. Here, 0 ≦ t ≦ N _r1 −1.
In the detection point data RD _t [t], at least one of the keyword characters is the character recognition result D in the comparison of the character codes.
Indicates a match that matches the characters contained in _c .

The character shape searching means 402 specifies a keyword character (wildcard character) which does not match the character included in the character recognition result D _c in the comparison of the character codes,
Based on the comparison of the image feature amounts, it is determined whether the wildcard character matches the character included in the character recognition result D _c .

The character width estimating means 403 calculates the estimated value of the width of the keyword character. This estimated value is used in the wildcard search means 401 and the character shape search means 402.

The character image table 404 is similar to the character image table 302 shown in FIG. 7, and the description thereof is omitted here.

FIG. 17 shows the wildcard search means 401.
The processing procedure by is shown.

The processing procedure shown in FIG. 17 will be described below step by step.

Step S1701: The value "0" is assigned to the variable i. The variable i is a subscript indicating the position of the keyword character in the keyword. Substituting the value “0” for the variable i means that the processing is performed in order from the first character K _w [0] of the keyword.

Step S1702: It is determined whether the variable i is equal to the variable N _k . The variable N _k represents the number of characters of the keyword K _w . If the determination in step S1702 is "Yes", the process ends. This is because when the variable i is equal to the variable N _k , it means that the processing has been completed up to the last character of the keyword K _w . If the determination in step S1702 is “No”, the process proceeds to step S1703.

Step S1703: The value "0" is set in the variable j.
Is substituted. The variable j is a subscript indicating the position of the character in the character recognition result D _c . Substituting the value “0” into the variable j means that the processing is sequentially performed from the beginning of the character recognition result D _c .

Step S1704: It is determined whether the variable j is equal to the variable N _d . The variable N _d represents the number of characters in the character recognition result D _c . If the determination in step S1704 is “Yes”, the process is step S170.
Go to 8. This is because if the variable j and the variable N _d are equal, it means that the processing has been completed up to the last character in the character recognition result D _c . If the determination in step S1704 is “No”, the process proceeds to step S1705.

Step S1705: K _w [i] and C
_It is determined whether or not _c [j] is equal. K _w [i] is
Represents the character code of the i-th character of the keyword, C
_c [j] represents the character code of the jth character in the character recognition result D _c . The determination in step S1705 is “Yes
If “s”, the process proceeds to step S1706. If the determination in step S1705 is “No”, the process proceeds to step S1707.

As described above, the wildcard search means 40
In step S1705, at least a part of the keyword K _w is the recognition result D _c by comparing the character codes in step S1705.
Functioning as a first determining unit that determines whether or not at least part of

Step S1706: i of detected location data
The character D _c [j] is registered at the th position. The detection point data has a length of N as described above with reference to FIG. 2D.
Here is a list of _k . The character D _c [j] is registered as the i-th list element of this list. Step S170
The detailed processing procedure of 6 will be described later with reference to FIG.

Step S1707: The variable j is incremented by 1. This means that the subsequent processing is performed for the next character in the recognition result D _c .

Step S1708: The variable i is incremented by 1.
To be made. This is the keyword K _wNext Kiwa in
It means that the following processing is performed for the character.

FIG. 18 shows the search result RD _t1 searched by the processing procedure shown in FIG. In the detection location data RD _t [1], there is a character in the recognition result D _c whose character code matches all the keyword characters included in the keyword Kw (= “Lake Biwa”). In the detection location data RD _t [0], there is a character in the recognition result D _c in which the character codes of some characters of the keyword K _w match. The list element 1861 indicated by “*” in the detection point data RD _t [0] indicates that a character in the recognition result D _c whose character code matches the keyword character K _w [2] (= “lake”) exists. No, keyword character K _w [2]
It means that the character in the recognition result D _c corresponding to is undecided. Such list elements are called "wildcards".

FIG. 19 shows step S1 shown in FIG.
The detailed processing procedure of 706 is shown.

The processing procedure shown in FIG. 19 will be described below step by step.

Step S1901: The value "0" is assigned to the variable t. The variable t is a subscript of the detection point data.

Step S1902: It is judged whether or not the variable t and the variable N _r are equal. The variable N _r indicates the number of detection point data detected so far. If this determination is “Yes”, the process proceeds to step S1907.
If this determination is “Yes”, it means that the detection point data capable of registering D _c [j] does not exist in the detection point data detected up to the present time. If the determination in step S1902 is “No”, the process proceeds to step S1903.

Step S1903: Detected location data RD
_The last registered character D _c [k] at _t [t] is obtained.

Step S1904: The subscript m of the keyword character K _w [m] corresponding to the character D _c [k] is acquired. The subscript m of the keyword character K _w [m] is the character D _c [k] at which position in the list of the detection location data RD _t [t].
It is obtained by checking if is registered.

Step S1905: Character D _c [k],
Between keyword D _c [j] and keyword character K _w [m + 1]
~K _w [i-1] is whether fits just enough is determined. This determination is made by, for example, character D _c [k] and character D _c
The space between _c [j] and the keyword character K _w [m +
Is performed by determining whether or not within the range of 1 × to 1.2 times the width of _{1] ~K w [i-1} ]. This range may be variable depending on the width of the space between characters in the document.

The determination in step S1905 is “Yes
If “s”, the process proceeds to step S1909. The determination being “Yes” means that the character D _c [j] can be registered in the detection location data RD _t [t].

[0209] The judgment in step S1905 is "N".
If “o”, the process proceeds to step S1906.

Step S1906: The value of the variable t is incremented by 1. That is, the character D _c [j] is not registered in the detection location data RD _t [t], and the process proceeds to the next detection location data.

Step S1907: The list RD _t [N _r ] for the new detection point data is secured, and the character D _c [j] is registered at the i-th position. List elements other than the i-th position of the secured list RD _t [N _r ] are wild cards.

Step S1908: The value of the variable N _r is incremented by 1.

Step S1909: Detection location data RD
_The character D _c [j] is registered at the i-th position of _t [t].

Hereinafter, referring to FIGS. 20A and 20B, the character recognition result D according to the processing procedure shown in FIG.
An example of searching the keyword K _w from _c will be described. here,
It is assumed that the character recognition result D _c has the data structure shown in FIG. 2B, and the keyword K _w is “shore of Lake Biwa”.

Now, of the keywords K _w “Biwako shore”, the processing for “Biwako” has been completed, and the processing for the keyword character “shore” (K _w [3], i = 3) is performed next. Be seen.

FIG. 20A shows the keyword K _w "shore of Lake Biwa".
Of these, the state of the detection point data RD _t [0] at the time when the matching process for “Lake Biwa” is completed is shown. Among the “Biwako” that have already been processed, there are characters (D _c [0] and D _c [1]) in the recognition result D _c with the same character code for “Biwa” and “Biwa”, respectively. There is. For the keyword character “lake”, there is no character in the recognition result D _c having a matching character code, so the correspondence relationship is undetermined. For the keyword character “Kan”, the correspondence relationship is undecided because it has not been determined whether there is a character in the recognition result D _c having a matching character code.
This determination is made in step S1705 (FIG. 17).

Now, assume that the variable j = 5. Step S
Character code C _c of the character D _c [5] In 1705 [5]
And the character code of the keyword character "Kan" ( _Kw [3]) are determined to be equal. Therefore, the process is step S17.
Proceed to 06. The detailed processing procedure of step S1706 is shown in FIG.

Now, assume that the variable t = 0. Step S
In 1903, the character D _c [k] last registered in the detection location data RD _t [0] is indicated by reference numeral 2 in FIG. 20A.
It is the character D _c [1] indicated by 062. Therefore, k =
It becomes 1.

In step S1904, the keyword character corresponding to the character D _c [1] of the detection location data RD _t [0] is the keyword character K _w [1]. Therefore, m =
It becomes 1.

In step S1905, the character D
_c [1 (= k)] and in the space between the characters _{D c [5 (= j)} ], keyword character _{K w [m + 1] ~K} w [i-1]
(In this case, K _w [2]) is determined whether it fits in exactly. Character D _c [1 (= k)] and character D _c [5 (=
j)] is 131 (determined as the difference between the x coordinate 318 of the lower right corner of D _c [1] and the x coordinate 449 of the upper left corner of D _c [5] shown in FIG. 2B. ). The character width of the keyword character K _w [2] is the character width estimation means 40.
3 is obtained by a procedure similar to that described with reference to FIG. It is assumed that the character width of the keyword character K _w [2] is 125. 125 <131 <125 × 1.
Since 2 (= 150) is established, the keyword character K is placed in the space between the character D _c [1] and the character D _c [5].
_w It is determined that [2] fits in exactly. Therefore, the process proceeds to step S1909.

At step S1909, RDt
The character D at the 3 (= i) th position of [0] _c[5 (=
j)] is registered.

FIG. 20B shows the state of the detection point data RD _t [0] at the time when the character D _c [5] is registered.
The detection location data RD _t [0] shown in FIG.
Contains wildcards. The wild card search means 401 outputs the search result RD _t1 including the detection location data RD _t [t] that may have a wild card in this way.

When the detected location data includes a wild card, the character shape search means 402 collates the wild card.

FIG. 21 shows the processing procedure of wildcard matching performed by the character shape searching means 402.

The processing procedure shown in FIG. 21 will be described below step by step.

Step S2101: The value "0" is assigned to the variable t. The variable t is a subscript of the detection point data.

Step S2102: It is determined whether or not the variable t is equal to the variable N _r . The variable N _r indicates the number of detection point data included in the search result RDt ₁ in the wildcard search means 401. If the determination in step S2102 is "Yes", the process is step S21.
Proceed to 07. If this determination is “Yes”, it means that the processing has been completed for the detection point data included in the search result RD _t1 .

The judgment in step S2102 is "N".
If “o”, the process proceeds to step S2103.

Step S2103: Detection location data RD
_{At t} [t], it is determined whether or not a wild card exists. If the determination in step S2103 is "Yes", the process proceeds to step S2104.

The judgment in step S2103 is "N".
If “o”, the process proceeds to step S2106.

Step S2104: The keyword character corresponding to the wild card and the character in the recognition result D _c are specified.

Step S2105: Step S2104
The shape of the keyword character specified in step 3 is collated with the character in the recognition result D _c . If this shape matching the keyword character and the character recognition result in the D _c are matched, the list element is a wildcard of the detection point data RD _t [t], registers the character recognition result in the D _c To do. If they do not match, the list element, which is a wild card, of the detection location data RD _t [t] is left as a wild card.

Step S2106: The variable t is incremented by 1.

Step S2107: The detection point data including the wild card is deleted. Of the detection point data included in the search result RD _t1 , the remaining detection point data obtained by deleting the detection point data including the wild card in step S2107 is output as the search result RD _t .

Now, one detection point data R included in the search result RDt ₁ input to the character shape search means 402.
Suppose D _t [0] is in the state shown in FIG. 20B. An example in which the processing procedure shown in FIG. 21 is specifically applied to the detection location data RD _t [0 (= t)] will be described below.

Since the wild card is included in the detection location data RD _t [0], the determination in step S2103 is “Yes”.

[0237] In step S2104, the keyword character corresponding to the wild card is the detection location data RD.
It is specified by checking the position of the wild card in _t [0]. As a result, the keyword character corresponding to the wild card is specified as K _w [2].

As described above, the character shape searching means 402 is
In step S2104, the first non-matching that identifies the first character that does not match the character recognition result among the at least one keyword character (first character) included in the keyword as the first non-matching character (K _w [2] in this case). Functions as a character identification means.

In step S2104, the process of specifying the character in the recognition result D _c corresponding to the wild card is performed as follows. Character recognition result in D _c for each wild card is obtained by combining the one or more consecutive characters in the recognition result D _c, it is referred to as a second mismatch characters.

In the detection location data RD _t [0], since the character D _c [1] exists to the left of the wild card,
The second non-matching character is considered to include the character D _c [2] as the leftmost character. However, at this point, it is unclear which of the rightmost characters of the second non-matching character is D _c [2] to D _c [4]. That is, whether the second mismatch character consists only of characters Dc [2], letter D _c [2] and letter D _c
Character D _c [2] and character D _c [3]
It is unknown whether or not it consists of the character D _c [4]. It is not necessary to consider the possibility that the rightmost character of the second non-matching character is D _c [5]. This is because the character D _c [5] has already been registered at the third position of the detection location data RD _t [0].

In step S2104, the second non-matching character having the width closest to the width of the first non-matching character (K _w [2] in this case) is specified. The width of the second non-matching character is defined as the area width of the area including the partial area assigned to each character included in the second non-matching character. Thus, in step S2104, the character shape search unit 402 also continuously has at least one or more second characters included in the character recognition result and having one or more consecutive widths that are closest to the width of the first non-matching character. It functions as a second non-matching character specifying means for specifying the second character as the second non-matching character.

FIG. 22 shows the relationship between the characters to be combined and the area width.

The area width of the character D _c [2] is 35, the area width when the character D _c [2] and the character D _c [3] are combined is 80, and the area width of the character D _c [2] is Character D _c [3] and character D
The area width when _{c and} [4] are combined is 125. Such a region width is obtained from the character coordinates shown in FIG. 2B.

On the other hand, the character width estimating means 403 obtains the width K _ww [2] of the keyword character K _w [2]. K _ww
If [2] = 125, then the second with the width closest to the width of the first non-matching character (K _w [2] in this case).
As the non-matching character, a combination of the character D _c [2], the character D _c [3], and the character D _c [4] is specified. Letter D
_A list (D _c [2], D _c [3], D _c [4]) obtained by combining _c [2] and the character D _c [3] and the character D _c [4].
Enter.

At step S2105 (FIG. 21), the keyword character K _w [2] and the characters (D _c [2], D _c [3],
D _c [4]) is shape-matched. In this shape matching, the feature amount of a character image (obtained from the document image D _i ) enclosed by a rectangle including the partial areas assigned to the characters D _c [2] to D _c [4] and the keyword character K _w. It is performed by comparing with the feature amount of the character image of [2] (obtained from the character image table 404). Each feature amount is a vector amount, and if the Euclidean distance between them is smaller than a predetermined threshold Thd ₁ , the list (D
_c [2], D _c [3], D _c [4]) is detected point data R
Register at the second position of D _t [0]. Alternatively, FIG.
The character indicated by reference numeral 2262 in 2 may be newly generated, and the character 2262 may be registered in the second position of the detection location data RDt [0]. Newly generated character 226
2 indicates a character in which the characters D _c [2], D _c [3], and D _c [4] are combined.

In this way, the character shape retrieval means 402
In step S2105, the keyword character K
_w [2] (first non-matching character) image feature amount and characters D _c [2], D _c [3], D included in the second non-matching character
_c [4] (1 or 2 or more consecutive second characters) is compared with a feature amount of an image of an area including 1 or 2 or more partial areas assigned to the first non-matching character 2nd judgment of whether or not it matches 2 non-matching characters
It functions as a judgment means.

Further, in another example in which the keyword is "Lake Biwa", even when the wild card is at the end of the detection point data as in the detection point data RD _t [0] shown in FIG. 18, the above-mentioned processing procedure is performed. The second non-matching character can be specified by the same processing procedure as.

In the detection location data RD _t [0] (FIG. 18), the character D _c [1] is added to the list element to the left of the wild card.
Is registered, it is considered that the leftmost character of the second mismatch character is D _c [2]. With the character D _c [2] as a reference, the number of characters to be combined is variable, and the keyword character K
The second non-matching character is specified by repeating the comparison between the width of _w [2] and the region width (the width of the combined at least one character, that is, the width of the second non-matching character). In this comparison, the allowable value of the width of the second non-matching character is calculated according to the width of the keyword character K _w [2], and under the condition that the width of the second non-matching character is smaller than this allowable width, The second non-matching character may be specified. For example, if the allowable width is 1.2 times the keyword character K _w [2], and the number of characters to be combined is 4, (D _c [2] + D _c [3] + D _c [4] +
The area width of D _c [5]) is 22 in FIG.
As shown in 61, it becomes 253, and the allowable width (125
X1.2 = 150) is exceeded. Therefore, when the number of characters to be combined is 4 or more, it is not necessary to consider, and the character D _c
[2], D _c [3], and D _c [4] are specified as the second mismatch character.

In another situation, when the wildcards are consecutively present, it is possible to specify the second non-matching characters one by one by the processing procedure described above.

In the above example, when the second non-matching character is specified, the leftmost character (D _c [2]) of the second non-matching character is fixed, and the number of characters to be joined to the right side of the second non-matching character is variable. On the contrary, when specifying the second non-matching character, the process of fixing the rightmost character of the second non-matching character and varying the number of characters to the left of the second non-matching character is similar to the above-described processing procedure. Can be done.

When wildcards are continuous or when wildcard characters are present at the end of the detected position data, the character that may be specified as the second non-matching character is divided into the smallest possible elements. However, the processing of steps S2104 and S2105 may be performed on this element. This division is performed to prevent omission of search even when adjacent characters are combined and recognized by the OCR device 202. When the document is written horizontally, for example, this division is performed in a portion where the projection histogram of black pixels in the vertical direction is obtained and the projection histogram is smaller than a predetermined threshold value.

The wildcard search means 401 is
The detection location data in which at least one character in the keyword matches the character recognition result is output as the search result RD _t1 , but the detection location data in which the number of characters in the keyword matches the character recognition result in a predetermined number or more It may be output as RD _t1 . For example, the wildcard search means 401 may output detection location data in which more than half of the characters in the keyword match the character recognition result.

Further, in the wild card search means 401, the character code of the keyword character and the character recognition result D _c
It was determined whether or not the character codes of the middle characters match, but it is determined whether or not the character codes of the similar characters of the keyword character and the character codes of the character recognition result D _c match. Good. The similar characters are, for example, ("kata" in katakana and "evening" in kanji), ("dog", "large" and "thick").
It means a character with a similar shape, such as.

As described above, according to the document retrieval apparatus 451 of the second embodiment of the present invention, even if one of the keyword characters included in the keyword is compared with the character in the character recognition result D _c in the character code comparison. If they match, the matching is performed based on the comparison of the image feature amounts in the vicinity of the character recognition result D _c . In comparing the character codes, it is not necessary that all the characters of the keyword match the characters in the character recognition result D _c . Therefore, it is possible to reduce the omission of search due to an error in character recognition. Also,
Since the target of the search based on the comparison of the image feature amounts is limited to the detection position where even one of the characters of the keyword in the character code matches the character in the character recognition result D _c , the cost of the search ( Time and calculation amount) are low.

(Third Embodiment) FIG. 23 shows the configuration of a document search system 1561 according to a third embodiment of the present invention. The document search system 1561 is an example of a usage pattern of the document search device described in the first and second embodiments.

The document search system 1561 includes a first communication means 1501, a center 1502, an image registration server 1503, an image search server 1504, an image database 1505, a second communication means 1506, and a terminal 15.
07 and.

[0257] The first communication means 1501 and the second communication means 1506 communicate with each other via the communication line 1509.
The communication line 1509 can be, for example, a telephone line (including PHS and mobile phone) or the Internet (wireless or wired).

The image registration server 1503 has a function of character recognition by OCR.

The terminal 1507 is equipped with a scanner, for example, and can obtain document image data from an original document. Alternatively, the terminal 1507 can capture document image data captured by a digital camera.

The image search server 1504 includes, for example, the document search device described in the first and second embodiments.

The user inputs the document image data obtained by a scanner, a digital camera or the like into the terminal 1507. The terminal 1507 sends this document image data to the center 1
Send to 502. The center 1502 receives the document image data and sends it to the image registration server 1503. The image registration server 1503 performs character recognition by OCR on the document image data, and stores the character recognition result and the document image data in the image database 1505.

The user can retrieve the document stored in the image database 1505 from any terminal capable of communicating with the center 1502. You can also use services such as browsing, printing, and circulation. The document stored in the image database 1505 is browsed through image browsing software. As image browsing software, for example,
HTML (Hyper Text Markup La
A browser that browses documents in the ngage) format may be used.

The center 1502 has a personal identification means, and it is possible to dedicate the image database 1505 to each user and to charge the use of the service for each user.

As a personal identification means, a fingerprint collation system or password according to a known technique can be used.

As described above, according to the document search system 1561 of the third embodiment of the present invention, the user can browse and search the document owned by the user anytime, anywhere.

The document search process described in the first and second embodiments can be recorded in the recording medium in the form of a program. As the recording medium, any type of computer-readable recording medium such as a floppy (registered trademark) disk or a CD-ROM can be used. By installing the document search processing program read from the recording medium into the computer, the computer can be made to function as the document search device.

In the above-described first and second embodiments, a Japanese document has been described as an example. However, the application of the present invention is not limited to Japanese documents. It is also possible to apply the present invention to documents in any other language (for example, Chinese documents, English documents, Korean documents).

Further, in the above-described first and second embodiments, the threshold value used when the shape matching means compares the feature amount of the character image KC _i of the keyword character with the feature amount of the character image C _i in the document image data. Thd ₁ is assumed to be a predetermined value. The threshold Thd ₁ may be changed according to the character code of the keyword character. For example, by using a character image table in advance, the probability distribution of the distance between the feature amount of the character image of the font used and the feature amount of the character image in the document image data is obtained, and an arbitrary probability is set. Threshold T
You can decide hd ₁ . Further, by controlling the threshold value Thd ₁ , it becomes possible to freely control the accuracy of the search.

[0269]

As described above, according to the present invention, it is possible to provide a document retrieval device and a recording medium capable of reducing omission of retrieval.

According to the present invention, when searching a keyword from a character recognition result, first, a search is performed based on comparison of character codes. Next, of the portions that do not match the keyword by the search based on the comparison of the character codes, the portion that satisfies the predetermined condition is searched based on the comparison of the image feature amounts. Thus, a search omission that may occur in the search based on the comparison of character codes is covered by the search based on the comparison of image feature amounts. Therefore,
It is possible to reduce search omissions due to character recognition errors. In addition, the target of the search based on the comparison of the image feature amount is
Since it is limited to the part that satisfies the predetermined condition, the cost (time and calculation amount) required for the search can be low.

According to the present invention, in the comparison of character codes, if any one of the characters of the keyword matches with the character in the character recognition result, the collation based on the comparison of the image feature amount is performed in the vicinity thereof. Be seen. In comparing character codes, it is not necessary that all the characters of the keyword match the characters in the character recognition result. Therefore, it is possible to reduce the omission of search due to an error in character recognition. Also,
The search target based on the comparison of the image feature amounts is limited to the detection position where even one of the characters of the keyword matches the character in the character recognition result in the comparison of the character codes. The amount of calculation) is low.

[Brief description of drawings]

FIG. 1 is a diagram showing the configuration of a document filing system 210.

FIG. 2A is a diagram showing an example of document image data D _i .

FIG. 2B is a diagram showing a data structure of a character recognition result D _c which is a result of performing character recognition on document image data D _i .

FIG. 2C is a diagram showing a data structure of a keyword K _w .

FIG. 2D is a diagram showing a data structure of a search result RD _t .

FIG. 3 is a diagram showing a structure of character block data D _t .

FIG. 4 is a block diagram showing a configuration of a document search device 1 according to the first embodiment of the present invention.

FIG. 5 is a diagram showing an example in which a character width estimation unit 104 estimates a character width of each character K _w [i] included in a keyword K _w .

FIG. 6 is a diagram showing an example of processing for identifying a candidate portion SD _c that satisfies a predetermined condition.

FIG. 7 is a block diagram showing a detailed configuration of a character shape search unit 103.

FIG. 8 shows a keyword K in the character image extracting means 301.
shows an example in which the character is identified in the corresponding candidate portions in SD _c with a particular character in _w

FIG. 9 is a diagram showing an example of a method for obtaining a feature amount (vector amount) from a character image C _i .

FIG. 10 is a block diagram showing the configuration of a character shape search unit 103a as a variation of the character shape search unit 103.

FIG. 11 is a block diagram showing the configuration of a document search device 701 as a variation of the document search device 1.

FIG. 12 is a block diagram showing the configuration of a document search device 801 as a variation of the document search device 701.

FIG. 13 is a block diagram showing the configuration of a document search device 901 as a variation of the document search device 701.

FIG. 14 is a block diagram showing the configuration of a document search device 1151 as a variation of the document search device 1.

FIG. 15 is a block diagram showing the configuration of a document search device 1051 as a variation of the document search device 1151.

FIG. 16 is a document search device 451 according to the second embodiment of the present invention.
Block diagram showing the configuration of

FIG. 17 is a flowchart showing a processing procedure in the wildcard search means 401.

FIG. 18 is a diagram showing a search result RD _t1 searched by the processing procedure shown in FIG. 17;

FIG. 19 is a flowchart showing a detailed processing procedure of step S1706 shown in FIG.

FIG. 20A is a diagram showing a state of detection point data RD _t [0] at the time when the matching process for “Lake Biwa” in the keyword K _w “Biwako shore” is completed.

FIG. 20B is a diagram showing a state of detection point data RD _t [0] when the character D _c [5] is registered.

FIG. 21 is a flowchart showing a wild card matching process procedure in the character shape searching unit 402.

FIG. 22 is a diagram showing a relationship between a combined character and a region width.

FIG. 23 is a document search system 1 according to the third embodiment of the present invention.
Diagram showing the configuration of 561

FIG. 24A is a character “book” included in an original document.
And "mouth" are converted into character codes corresponding to the characters "tree" and "ku", which have similar shapes, due to an error in character recognition.

FIG. 24B is a diagram showing an example of a list of similar characters.

[Explanation of symbols]

1, 451, 701, 801, 901, 1051, 11
51 document search device 101 text search means 102 character specification means 103, 402 character shape search means 104, 403 character width estimation means 201 image input device 202 OCR device 203 document database 204 document search device 205 display device 210 document filing system 301 character image Extracting means 302, 404 Character image table 303 Shape matching means 304, 606 Matching control means 401 Wildcard searching means 602 Similar character matching means 705 Search accuracy control means 805 Quality information extracting means 905 Search accuracy specifying means 1002, 1102 Similar character string average Characterizing means 1103 character string re-detecting means

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI G06K 9/00 G06K 9/00 S 9/62 620 9/62 620D (72) Inventor Koji Kaji, Osaka Prefecture Kadoma City Daimon Kadoma 1006 Address: Matsushita Electric Industrial Co., Ltd. (56) References JP 2001-337993 (JP, A) Matsukawa Yoshihiko, Imagawa Taro, Kondo Kenji, Meguro Koji, improved document image search performance using shape feature search, electronic information IEICE Technical Report, Japan, The Institute of Electronics, Information and Communication Engineers, September 16, 1999, Volume 99, No. 305, PRMU99-74, 77-83 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06F 17/30 310-419 G06K 9/00-9/72

Claims (6)

(57) [Claims] 1. A document search device for searching a keyword from a recognition result obtained by performing character recognition on an image of a document, wherein the keyword includes at least one first character, A character code and a character image are assigned to each one of the first characters, the recognition result includes at least one second character, and each of the at least one second character includes a character code and a character code. , A partial area of the image of the document is assigned, and the document search device determines whether at least one first matching portion matching the keyword is present in the recognition result based on the comparison of the character codes. It is determined whether or not there is, and if there is, a first matching part specifying means for specifying the at least one first matching part, and a predetermined first condition. It is determined whether or not at least one first portion to be added exists in a portion excluding the specified at least one first matching portion from the recognition result, and if there is, at least one first portion. a first part specifying means for specifying a part, the feature amount of the image of the said partial area assigned to the second character included in the first part, of the character image of the first character included in the keyword Based on the comparison with the feature amount, it is determined whether at least one second matching portion that matches the keyword is present in the specified at least one first portion. A second matching part specifying unit for specifying one second matching part, wherein the predetermined first condition is a width smaller than a predetermined width.
The document search device having a condition that the first portion is present in the vicinity of a specific second character having a . 2. A document search device for searching a keyword from a recognition result obtained by performing character recognition on an image of a document, wherein the keyword includes at least one first character, A character code and a character image are assigned to each one of the first characters, the recognition result includes at least one second character, and each of the at least one second character includes a character code and a character code. , The partial area of the image of the document and the character recognition
Trust that indicates the certainty of character recognition obtained when performing
And the document search device determines whether or not at least one first matching portion that matches the keyword exists in the recognition result based on the comparison of the character codes, and In the case of performing, at least one first matching part specifying means for specifying the at least one first matching part, and at least one first part satisfying a predetermined first condition are at least one first specified part from the recognition result. The first part specifying means for judging whether or not the first part is present in the part excluding the one matching part, and if the part is present, the second part included in the first part and the feature quantity of the image of the partial area allocated to the character, based on a comparison of the feature quantity of the character image of the first character included in the keyword, the less matches said keyword Also determines whether one second matching portion exists in the specified at least one first portion, and if so, specifies a second matching portion that specifies the at least one second matching portion. and means, the predetermined first condition, wherein the confidence assigned to the vicinity of the small specific second character than a predetermined threshold value first
A document retrieval device, provided that there is a part . 3. The document search device according to claim 2, further comprising: a unit that determines the image quality of the image of the document; and a unit that determines the predetermined threshold value based on the determined image quality of the image. . 4. The second matching portion identifying means is the first matching portion identifying means.
First determining means for determining whether or not the character code of the second character included in the portion matches the character code of the specific first character included in the keyword; and the first determining means included in the first portion. The two-letter character code is
When the character code of the specific first character included in the keyword does not match, the width that includes at least the second character included in the first portion and is closest to the width of the specific first character is set. A non-matching character specifying unit for specifying one or more continuous second characters as a non-matching character, a feature amount of the image of the specific first character, and one or more continuous two or more characters included in the non-matching character. The specific first character matches the unmatched character when the distance from the image feature amount of the area including one or more partial areas assigned to the second character is smaller than a predetermined value. The document search device according to claim 1, further comprising a second determination unit that determines to do so. 5. The document search device comprises the at least one
Calculating means for calculating a predetermined judgment reference value from one of the first matching portions, and the at least one second judgment portion based on the judgment reference value.
The document search device according to claim 1, further comprising: a detection unit that detects a second matching portion satisfying a predetermined second condition among the matching portions. 6. The calculating means determines the determination reference value based on a characteristic amount of an image of at least one partial region assigned to the at least one second character included in the at least one first matching portion. The second condition is calculated as a distance between a feature amount of an image of at least one partial region assigned to the at least one second character included in the at least one second matching portion and the determination reference value. The document search device according to claim 5, including a condition that is smaller than a predetermined value.