JPH0689330A - Image filing system - Google Patents

Abstract

PURPOSE:To easily give an instruction about a desired character string area in an input image displayed on a screen and obtain the character string direction, to efficiently input a keyword used to retrieve an input image, and to lighten the burden of operation on an operator. CONSTITUTION:When the position coordinates of one optional point are given from an operation part 1 in the input image displayed on the screen of a display part 3, a recognition object character string extraction part 7 sets a proper area based on the given coordinates and also decides the array direction of the character string components in the area at the same time, and expands the area in the array direction of the character string components according to the result to extract the character string components in the extended area. A character recognition part 8 recognizes the character string components and further codes them, and the coded character string is filed on an optical disk 11 while made to correspond to the input image as a key word.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image filing system for recording / retrieving image information.

[0002]

2. Description of the Related Art In an image filing system for recording and retrieving image information, various methods have been considered for easily inputting a title used for retrieving image information.

Conventionally, for example, the one disclosed in Japanese Unexamined Patent Publication No. 60-136877 is known as a device that enables the input of a title used for searching such image information. This is to cut out a desired character string area on the screen displaying an input image by pointing with a pointing device such as a mouse in the process of filing an image and recognizing the content of this pointing area as a keyword. It is coded and input as a title to be used for subsequent image retrieval.

Further, the character string included in the image is recognized,
In order to create a name or keyword to be given to the image by this recognized character code, a format definition language such as that disclosed in "Study D, J71-D, No.10, pp2050-2058" A method of describing a character string area is also known.

[0005]

However, according to the first idea, when a character string area is designated by a mouse or the like,
At least two points in the character string area need to be designated, and at the same time, the operator must set the direction of the character string. Further, since the instruction work is required for each input image, the work load on the operator is reduced. There was a problem that it became large and difficult to use.

Further, even in the later thought, since the document structure and the character string position are described by using physical quantities such as absolute coordinates in the image, even when processing a document having the same structure, a large position There is a problem that the character string area cannot be extracted accurately when inputting a gap, and the practical description that the burden on the operator is large because the description of the character string area is complicated. There was a problem that it would occur.

In the case of creating a keyword by designating a keyword image from an input image, if a plurality of recognition candidates are obtained by character recognition, one of them should be selected and input as a title. Therefore, for example, when the recognition result is unclear and it is not possible to narrow down to one candidate, an incorrect recognition result may be input as a title, which makes it difficult to perform accurate image search. there were. In order to avoid this problem, if the operator checks and corrects the character recognition result of the keyword image at the time of image input, the operator's burden will be increased.

The present invention has been made in view of the above circumstances, and a desired character string region on an input image displayed on a screen can be easily specified, and at the same time, a character string direction can be obtained, so that the input image An object of the present invention is to provide an image filing system capable of efficiently inputting a keyword or the like used for a search and reducing the work load on an operator.

Further, according to the present invention, even if the number of characters or lines of a document of the same type varies or the position shifts, a character string component to be a keyword can be accurately detected, and the operator can be detected. An object is to provide an image filing system that can reduce the burden.

A further object of the present invention is to provide an image filing system capable of performing image search without the operator having to confirm the recognition result regarding the keyword image in the input image.

[0011]

According to the present invention, there is provided a display means for displaying an input image on a screen, a means for designating a position coordinate of an arbitrary point on the screen of the display means, and an instruction made by this means. Means for setting an appropriate area from the position coordinates, means for determining the arrangement direction of the character string components from the input image in the area set by this means, areas along the arrangement direction of the character string components determined by this means And a recognition target character string extracting means having a means for extracting a character string component in the area expanded by this means, and a character recognition and code for the character string component extracted by the recognition target character string extracting means. And filing the coded character string as a keyword of the input image in association with the input image.

Further, according to the present invention, means for extracting a character string component from an input image, a block in which the character string components extracted by this means are integrated in a predetermined relationship, and the physical position of these blocks are detected. And means for analyzing the size of the blocks to obtain the semantic connection between the blocks, means for assigning attribute information to the blocks from the semantic connection between the blocks, and a predetermined character string. Means for storing information regarding the attribute information, means for extracting the character string component of the block to which the attribute information is assigned by referring to the information regarding the predetermined character string stored in this means, and the character string extracted by this means Means for character recognition and coding of components, and filing by correlating the coded character string with the input image as a keyword of the input image And it is made of.

Further, according to the present invention, means for designating a keyword area for an input image, means for cutting out an image of the keyword area designated by this means from the input image as a keyword image, and a keyword cut out by this means A means for recognizing characters in the image, a means for storing the result of character recognition by this means including character recognition candidates, and a similarity by judging the similarity of each character recognition candidate with respect to a keyword given at the time of retrieval. It is configured by means for preferentially presenting a high image.

[0014]

As a result, according to the present invention, when the operator designates an arbitrary point on the input image displayed on the screen, an appropriate area is set first and the input image within the area is set. The arrangement direction of the character string components of is determined, the region is expanded along the arrangement direction of the character string components based on this judgment, and character recognition and coding are executed for the character string components of this expanded region, and encoding is performed. Filing of the input image is performed using the created character string as a keyword. With this, the operator can character-recognize and code a keyword such as title information to be given to the input image and input the keyword by only pointing an arbitrary point on the input image.

Further, according to the present invention, the physical position and size of a block in which the character string components extracted from the input image are integrated in a predetermined relationship are analyzed to obtain the semantic connection relationship between the blocks. Further, by adding attribute information to each block, it becomes possible to accurately detect a character string component described at an arbitrary position corresponding to the title or keyword of the input image based on the semantic connection relationship and the attribute information. This makes it possible to accurately detect a character string component that is to be a keyword even if the desired character string in the same type of document has a variation in the number of characters or the number of lines and a positional shift. .

Further, according to the present invention, character recognition is performed on the keyword image cut out from the keyword area designated for the input image, the character recognition result is stored including the character recognition candidates, and the image is searched. An image that is similar to the character recognition candidate is preferentially presented rather than the similarity between the character recognition candidate and the character recognition candidate determined by the keyword given at some time.
As a result, even if the recognition of the keyword image is unclear, the correct recognition result is left in the plurality of recognition candidates,
At the time of search, the one having a high similarity to the keyword is selected from the recognition candidates and the corresponding image is presented, so that the operator can obtain the desired image. That is, an appropriate image search can be performed without confirming the recognition result of the keyword image. Further, when the image is presented, the operator determines that the desired image is obtained and gives a confirmation instruction so that the recognition result of the keyword of the image is uniquely determined as the keyword used for the search. Good.

[0017]

【Example】

(First Embodiment) FIG. 1 shows a schematic structure of the first embodiment. In the figure, reference numeral 1 denotes an operation unit, which is composed of a pointing device such as a keyboard or a mouse used for inputting instruction information regarding the operation of the apparatus. Then, the input data from the operation unit 1 is sent to the control unit 2.

The control unit 2 controls the operation of the entire apparatus, and includes a display unit 3, an image input unit 4, an output unit 5, an input image compression unit 6, a recognition target character string extraction unit 7, and a character recognition unit 8. A control instruction is given to the registration unit 10 and the recording unit 12.

Here, the display unit 3 is composed of a CRT display or the like for presenting various information from this apparatus to the operator. The image input unit 4 is for inputting, as image information, for example, by optically scanning a document in which characters, figures, photographs, tables, graphs, etc. are described.

The image filing processing by the present system can be interactively proceeded by the operation of the operation unit 1 and the information presentation by the display unit 3 for the image input as the image information from the image input unit 4. In this case, the document (input image) input from the image input unit 4 is the input image compression unit 6
Is given to the recognition target character string extraction unit 7. The input image compression unit 6 compresses the input image in a predetermined compression format and stores it in the storage unit 12. The recognition target character string extraction unit 7 extracts, from the input image, a “character string corresponding to the title or keyword of the input document described in an unspecified position of the input image” that the user instructs using the pointing device. Like In this case, conventionally, two points in the area had to be designated in order to acquire the character string area, but in the present invention, only one point which can be regarded as the center of the character string area to be acquired is designated and the desired character is designated. The row area can be acquired. At this time, the result of extracting the character recognition target character string may be displayed on the display unit 3.

The character recognizing unit 8 performs the recognizing process by cutting out the characters included in the character recognizing target character string one by one and comparing them with the standard character patterns of the recognizing target category stored in the dictionary unit 9 in advance. , Each character is encoded and supplied to the registration unit 10. At this time, the character recognition result may be displayed on the display unit 3. Then, the user interactively corrects the incorrect character recognition result displayed on the display unit 3 using the keyboard of the operation unit 1, and then the character string encoded by the registration unit 10 is changed to In association with the compressed input image stored in the storage unit 12, it is registered in the optical disc 11 as the title or keyword of the input image.

The output unit 5 comprises a laser printer or the like that outputs the image stored on the optical disk 11.
The storage unit 12 also stores a compressed input image, intermediate results and final results in each process, information about the state of the device, and the like. Next, the operation of the embodiment configured as described above will be described.

In this case, the image filing process in this embodiment is composed of the three processes shown in FIG.

(A) The input image is compressed (shown in FIG. 13).

(B) An area including "a character string to be a title or a keyword to be added to an input document" interactively instructed by the operator is extracted from the input image (Fig. 14), and the characters included in the area are extracted. The pattern is coded by character recognition processing (15 in the figure).

(C) The compressed input image and the coded character string are associated with each other and registered in the optical disc (FIG. 16).

The process (a) here is carried out by the image compression unit 6, the process (b) is carried out by the recognition target character string extraction unit 7 and the character recognition unit 8, and the process (c) is carried out. It is implemented in the registration unit 10.

The process (b) described above, which is an essential part of the present invention, will be described below with reference to the input image example shown in FIG.

Before the image filing work, the operator selects the item "image input and automatic addition of keywords" from the menu displayed on the display unit 3 using the keyboard or pointing device.

Then, the input image is subjected to binarization processing, noise removal processing, etc., and the image 17 subjected to the binarization processing, noise removal processing, etc. is displayed on the display unit 3,
Further, a message “Please specify the character recognition target character string” is displayed.

From this state, the operator points on the image 17 displayed on the display unit 3 to obtain a character string area corresponding to the title or keyword of the input document described in the input document. Use to give instructions. In this case, as shown in FIG. 4, one point that can be regarded as the central portion 19 of the title character string 18 corresponding to the title or keyword of the input document is designated.

As a result, the coordinate value (x, y) on the image corresponding to the position designated by the pointing device is obtained. Then, using this coordinate value (x, y), the recognition target character string is extracted by the processing procedure described below.

In this case, the image 1 containing the coordinate values (x, y)
7 In the limited range above, the space between the characters in the horizontal direction and the space between the characters in the vertical direction are obtained, and the character string direction is estimated by comparing them, and the character (actually, the black connected component rectangle) is in that direction. The character string area to be recognized can be extracted by putting together.

First, the following equation 1, equation 2, equation 3 and equation 4 are applied to the coordinate value (x, y) to obtain the coordinate value (lx, ly) and the coordinate value (rx as shown in FIG. , Ry), and the application range of the process for extracting the recognition target character string region 20
To limit.

[0035] Here, W is the width of the input image, H is the height of the input image, and α is the threshold value.

Then, the processing applicable range 20 (that is, lx≤x≤rx and ly≤y≤ry) obtained from these equations is satisfied.
In a range that satisfies both), perform a labeling process to extract the black connected region by examining the connection relationship between each black pixel,
The circumscribed rectangle of each black pixel area is extracted (FIG. 6). further,
As shown in FIGS. 7A and 7B, black pixel connected components 21 that are close to each other to some extent or black pixel connected components 2 that have overlapping areas.
2 is made into one character candidate rectangle 23 by putting them together.

Next, the character string direction of the character string to be recognized is estimated from the character candidate rectangle group 23. For that purpose, as shown in FIG. 8, first, the black connected component rectangle C closest to the coordinates (x, y) designated by the pointing device is found. The black connected component rectangle C has the smallest distance from the coordinates (x, y) among the centers of gravity of all black connected component rectangles.

Then, paying attention to the black connected component rectangle C, from the whole black connected component rectangle, the following equations 5 and 6 are obtained.
, A set CS with other black connected component rectangles that both satisfy is extracted.

Min (cx2, x2) -max (cx1, x1)> 0 Equation 5 min (cy2, y2) -max (cy1, y1)> 0 Equation 6 Here, (cx1, cy1) is black-connected. The upper left corner of the component rectangle 24, (cx2, cy2) is the lower right corner, and (x1, y
Let 1) be the upper left corner of another black connected component rectangle, and (x2, y2) be the lower right corner.

Here, the number of rectangles satisfying the expression (1) 5 (that is, vertically aligned with the black connected component rectangle C) is counted (this count value is VC).

(2) The number of rectangles satisfying the expression 6 (that is, aligned with the black connected component rectangle C in the horizontal direction) is counted (this count value is referred to as HC).

If VC + th (threshold value) <HC, the direction of the character string to be recognized is horizontal, and H
If C + th <VC, the direction of the character string to be recognized is the vertical direction. In addition, in the cases other than the above, the following processing is further performed.

│HV-VC│≤th In this case, the average value of the distances between the black connected component rectangles that are horizontally adjacent to each other is obtained from the above-mentioned CS, and this is defined as the horizontal character space HS, and is vertically adjacent. The average value of the distances between the black connected component rectangles is calculated, and this is taken as the vertical space between characters.

Then, the HS is estimated by, for example, obtaining the average value of the distances between adjacent character candidate rectangles that satisfy the following expression: min (xi2, xj2) -max (xi1, xj1)> 0 in CS described above. can do. Similarly, VS is estimated by obtaining the average value of the distances between adjacent character candidate rectangles that satisfy the following expression: min (yi2, yj2) -max (yi1, yj1)> 0 ... You can

Here, the coordinate value of the upper left end of the black connected component rectangle i is (xi1, yi1), and the coordinate value of the lower right end is (xi2, yi2).
And the coordinate value of the upper left corner of the black connected component rectangle j is (xj1, y
j1) and the coordinate value of the lower right corner is (xj2, yj2).

Generally, a printed document has a property that the distance between characters is smaller than the distance between lines.
The direction in which the space between characters is small can be estimated as the direction of the character string. That is, when HS <VS ... Equation 9 holds, the character string direction is horizontal, and when HS> VS ... Equation 10 holds, the character string direction is vertical. Further, if HS = VS ... Equation 11 holds, it is impossible to estimate in the character string direction.

When the character string to be recognized is obtained in this way, the processing applicable range 20 is further expanded in the character string direction as follows.

First, when the character string direction is the horizontal (vertical) direction, the processing applicable range 20 is expanded as shown in FIG. In this case, 1x ′ = 0 (1x ′ = 1x) Equation 12 1y ′ = 1y (1y ′ = 0) Equation 13 rx ′ = W (rx ′ = rx) Equation 14 ry ′ = ry (ry ′ = H) ... Equation 15 is obtained, and the new processing application range 201 is “1x ′ ≦ x ≦
rx ′ and 1y ′ ≦ y ≦ ry ′ are both satisfied ”.

Then, the character string region to be recognized is estimated in this new processing application range 201. In this case, first, a labeling process is performed in the process application range 201 to extract the above-described black connected component rectangle C, and the recognition target character string region is estimated as follows.

When the character string direction is the horizontal (vertical) direction, first, it is on the left (upper) side as viewed from the rectangle C (that is, the x (y) coordinate of the lower right corner of the rectangle other than the rectangle C <cx1 ( c
Equation 6 (Equation 5) is applied to the rectangle (which is y1) to extract a rectangle group CSH1 (CSV1) satisfying this condition. Here, a rectangle C is included in CSH1 (CSV1).

Next, x (y) at the upper left corner of each rectangle
The appearance order of the rectangles is rearranged in CSH1 (CSV1) in descending order of coordinates. This is CSH1 '(CSV
1 '). As a result, CSH1 (CSV1) is rearranged from the rectangle closest to the rectangle C to the farthest rectangle.

Next, in CSH1 '(CSV1'), the following conditions are sequentially applied from the beginning, and the rectangle CLH1 (CLV1) farthest from the rectangle C satisfying the conditions is extracted. The conditions here are Equation 6 (Equation 5) and Equation 16 (Equation 1)
It is to satisfy both of 7). │xi1-xj2│ ≦
λ ... Equation 16 (| yi1−yj2 | ≦ λ ... Equation 17) However, the appearance order of the rectangle j is later than that of the rectangle i. Further, λ is the height of the rectangle C.

Then, among the coordinates of the upper left corner of the rectangle up to CLH1 (CLV1), the minimum x coordinate xmin and y coordinate y are obtained.
Find min.

Next, the right (lower) side of the rectangle C (that is, the x (y) coordinate of the upper left corner of the rectangle other than the rectangle C>
Equation 6 (Equation 5) is applied to a rectangle (cx2 (cy2)), and a rectangle group CSH2 (CSV2) satisfying this condition is satisfied.
To extract. Here, the rectangle C is included in CSH2 (CSV2).

Next, x (y) at the upper left corner of each rectangle
The order of appearance of rectangles is rearranged in CSH2 (CSV2) in ascending order of coordinates. This is CSH2 '(CSV
2 '). As a result, CSH2 (CSV2) is rearranged from the rectangle closest to the rectangle C to the farthest rectangle.

Next, in CSH2 '(CSV2'), the following conditions are sequentially applied from the beginning, and the rectangle CLH2 (CLV2) farthest from the rectangle C satisfying the conditions is extracted. The conditions here are Equation 6 (Equation 5) and Equation 16 (Equation 1)
7) is to satisfy both. Furthermore, CLH2 (C
Among the coordinates of the lower right corner of the rectangle up to LV2), the maximum x coordinate: xmax and y coordinate: ymax are obtained.

Then, (xmin, ymin) and (xm
A rectangle specified by the coordinates of two points (ax, ymax) is set as the recognition target character string area (that is, xmin ≦ x ≦ xma).
x and ymin ≦ y ≦ ymax are both satisfied).

Here, the image 1 displayed on the display unit 3 is displayed.
The character recognition target area may be surrounded by a frame on the display 7, the color of the corresponding portion may be changed and displayed, and the character portion and the background portion may be filled in with a color different from them.

When the character recognition target area is not correctly extracted by the one-point instruction by the above-described method, the pointing device is used to select the upper leftmost end and the uppermost end of the desired character string on the image 17 displayed on the display unit 3. 2 at the bottom right
The conventional “2
By using "point designation", the recognition target character string area can be designated. At this time, the designated area is vertically long (w <
In the case of h), the recognition target character string is considered to be vertical writing, and in the case of landscape orientation (w ≧ h), the recognition target character string is considered to be horizontal writing. However, w: horizontal width of the recognition target character string area, and h: vertical width of the recognition target character string area.

Next, when the mouse shown in FIG. 10, for example, is used as the pointing device, the instruction of the position coordinates will be described.

In the figure, reference numeral 24 denotes a mouse body, a left button 25 provided on the body 24 is used when a character recognition target area is designated by one-point designation, and a middle button 26
Is used when character recognition is performed on a designated area and an image is automatically coded, and the right button 27 is used when a character recognition target area is designated by a two-point instruction. The logic regarding the control signal generated by pressing each button of the mouse is as follows.

(1) Press the "left button 25" and then press the "middle button 26". The character recognition target area is designated by the one-point designation, and the pattern inside the target area is recognized and coded.

(2) Press and hold the "left button 25". The area determined by the last one-point instruction is set as the character recognition target area.

(3) Press the "left button 25" and then the "right button 27". The 1-point instruction mode is released and the 2-point instruction mode is set.

(4) Press the "right button 27" and then the "left button 25". The 2-point instruction mode is released and the 1-point instruction mode is entered.

(5) Continue to press the "right button 27". The area determined by the last two-point instruction is the character recognition target area.

(6) Press the "middle button 26". It is invalid unless the "left button 25" has been pressed once or more in advance and the "right button 27" has been pressed twice or more in advance.

In addition to this, the "left button 25" of the above (2)
When is continuously pressed, for example, the character string region candidates may be extracted by the number of times the “left button 25” is pressed and displayed on the display unit 3. For example, in the character string as shown in FIG. 11A, the left button 27 is placed at the position P1 in the figure.
When the button is pressed once, the character string area is displayed as shown in FIG. 11B, and when it is pressed twice, the character string area is displayed as shown in FIG. 11C as the character string area candidate. You may go. At this time, the character recognition process is performed by pressing the middle button 26 when the character string region candidates that are sequentially extracted match the desired character string region.

In this manner, the character recognition processing target character string is extracted, and the character recognition unit 8 is activated in a state where a signal for operating the character recognition function by the pointing device is generated. In this case, in the recognition target character string area,
Since the character candidate rectangles have been generated, the character pattern for each rectangle is supplied to the character recognition unit 8. At this time, the pattern integration of the character candidate rectangles on the left and right is basically not performed. As a result, a character such as "movement" is extracted as two patterns such as "heavy" and "force". Since the two patterns extracted in this way have the possibility of being one character and the possibility of being two characters, it is necessary to consider each of these two possibilities. Therefore, in such a case, for example, character recognition may be performed on all of the plurality of cutout candidates, and finally the contextual consistency may be evaluated to determine one.
Conversely, there are also character patterns in which a plurality of characters are in contact. Such a character pattern, for example, FIG.
For the character pattern shown in (a), the minimum point of the peripheral distribution (projection component) f in the direction perpendicular to the character string of black pixels forming the character pattern is taken as shown in FIG. The position is determined to be the boundary position of the plurality of contacting character patterns. After that, by examining the pattern shape in the vicinity of the boundary position, it is determined that the recessed portion near the boundary position is the cutting position of the character string pattern, and the character string pattern is cut at the cutting position. Cut out individual string patterns.

When individual character patterns are cut out, each character pattern is then associated with a category of characters to which the pattern may belong. This association is performed, for example, by calculating the degree of coincidence by superimposing the statistically created standard pattern and the unknown pattern, as in the composite similarity method. As a result, the first to Mth character categories associated with the unknown pattern are listed together with the code of the character category and the above-mentioned matching degree in order from the one with the highest matching degree, and are output. The list is displayed on the display unit 3 and the operator corrects the erroneous recognition result while operating the operation unit 1.

When the title or keyword of the input document is determined by such processing, the character string code is associated with the compressed input image already stored in the storage unit 12 and stored in the optical disc 11. The file is stored and the filing process is ended.

Here, the data stored in the optical disk 11 need not be the input image itself, but may be a part of it or processed data such as a recognition result.

Therefore, in this way, by designating a point on the desired character string described at an arbitrary position on the input image, the appropriate area is set and the character of the input image within the area is set. The arrangement direction of the column components is determined, the region is expanded along the arrangement direction of the character string components based on this determination, character recognition and coding are executed for the character string components of the enlarged region, and this coding is performed. Since the input image is filed using the character string as a keyword, keywords such as title information used for searching the input image can be efficiently input, and the work load on the operator can be reduced.

(Second Embodiment) In the first embodiment described above, the user designates the "character string area which can be a keyword or title of the input document" for each input document with the pointing device. This work can be cumbersome when filing a large number of input documents.

By the way, when the input document has almost the same format, it is expected that the "character string area which can be a keyword or title of the input document" is at a substantially fixed position in the document, and one document If you know the area of, you can use it in other documents.

Therefore, in the second embodiment, when the "character string corresponding to a keyword or a title" is located at almost the same position in a large number of documents, the character string to be recognized is designated by the pointing device for the first image. Only by doing so, the character string to be recognized can be correctly extracted in other documents.

In this case, when one document is selected as a representative, the coordinate value (xmin, xmin, x) of the upper left corner of the rectangular area obtained when the desired character string area is designated on the input image using the pointing device. ymin) and the coordinate value (xmax, ymax) of the lower right corner are respectively expressed by the following equation 1
8, equation 19, equation 20, and equation 21 are used.

[0078] As a result, the recognition target area is, as shown in FIG. 13, an enlarged character string area 31 represented by coordinates 29 at the upper left corner and coordinates 30 at the lower right corner with respect to the character string area 28 obtained by the first instruction. It is possible to accurately extract a desired character string even if the input document has a format change or a slight positional deviation.

With respect to other input documents, a desired character string can be extracted by extracting an image included in the enlarged character string area 31. At this time, as in the first embodiment, a black connected component rectangle is extracted in the character string region 31 and the distribution thereof is examined to determine a more accurate character string region, and the image inside thereof is determined. You may extract as a desired character string pattern.

In addition to the above, for example, the character string region 28 obtained by the first instruction may be adaptively changed for each input image to determine the desired character string region. For example, if the desired character string pattern is outside the rectangle because the input image is tilted, the area 2
The area 28 may be gradually enlarged so that the image corresponding to the coordinate values on each side of 8 is all white pixels (that is, the background portion), and a rectangle enclosing a desired character string component may be generated. If all the images corresponding to the coordinate values on each side of the area 28 are already white pixels, the area is gradually reduced to generate the smallest rectangle that circumscribes the desired character string component. May be.

Therefore, in this way, when the title area in the input image is standardized, if the area is set in advance, the title information is automatically coded and input in a large number of documents. Therefore, the input cost can be significantly reduced.

(Third Embodiment) In the above-described first embodiment, during the image filing operation, the user directly uses the operation unit 1 on the input document image displayed on the display unit 3 to “add to the input document”. A character string to be used as a title or a keyword (that is, a character string to be recognized) has been designated or specified. However, here, before the image filing work, the user previously desired a character string (character string to be recognized). ) May exist (for example, absolute position coordinates in the document image) is defined by, for example, format control.

In this case, the "region in which a desired character string can exist" is defined in advance by the user as a rectangular region S based on, for example, coordinate information of two points. When a document is input during the image filing operation, the information in the rectangular area S is used to operate as follows.

First, layout analysis processing or character string extraction processing is performed on an input document to extract character patterns in character string units or word units, and represent them in rectangles (the extracted character string rectangle group is referred to as SS. Next, the "rectangular area S" and the character string rectangular group SS are overlapped with each other to extract a character string rectangle S'of the character string rectangular group SS that overlaps with the "rectangular area S". At this time, the rectangle S ′ may be explicitly displayed as compared with other rectangles. Of course, it is also possible to display all the character string rectangle groups SS, or it is effective to display them by superimposing them on the original image. Further, the recognition result can be displayed near them.

Next, the operator sequentially designates a rectangle corresponding to a desired character string in the "character string rectangle S'with overlapping" using the operation unit 1 (the designated rectangle is referred to as PR). Then, in the "instructed rectangle PR", the internal pattern is recognized by the character recognition unit 8 and coded. The recognition results are stored in the order in which they are instructed (pointing). At this time, the recognition results are sequentially displayed on the display unit 3 so that the operator can correct the erroneous recognition results while operating the operation unit 1. You can

Next, the "correspondence between the coordinates obtained by pointing with the pointing device and the rectangle PR" will be described. This correspondence relationship is that the coordinate P designated by the pointing device corresponds to the smallest rectangle among the rectangles including it. However, if another rectangle R'is included in the rectangle (provided that R'does not include P), the rectangle R'is not considered. For example, as shown in FIG. 14, when an arbitrary point (for example, 33) inside the rectangle 32 is designated, only the pattern inside the rectangle 32 is recognized. Also, the rectangle 34
If one point (eg, 35) inside the
Only the internal pattern of 4 is recognized. And
When an arbitrary point (for example, 37) inside the rectangle 32 and inside the rectangle 36 which is not included in the rectangle 34 is designated,
All patterns inside the rectangle 36 are recognized. At this time, for example, the rectangle 32 and the rectangle 34 (that is, the rectangles included in other rectangles) may be displayed on the display unit 3 to be slightly smaller than they actually are so that the user can easily identify the rectangle. .

In these processes, it is not necessary to determine the rectangular area S in advance. That is, the layout analysis may be performed on the entire input image or a part of the input image, and the result may be displayed on the screen so as to be superimposed on the original image so that the operator can instruct the character string to be recognized.

When the title or keyword of the input document is determined by such processing, the character string code is associated with the compressed input image already stored in the storage unit 12 and stored in the optical disc 11. Store and finish the filing process.

(Fourth Embodiment) FIG. 15 shows a schematic structure of the same embodiment. The same parts as those in FIG. 1 are designated by the same reference numerals.

In this case, the document (input image) input from the image input unit 4 is the input image compression unit 6 and the document structure analysis unit 4.
Given to 2.

The document structure analysis unit 42 performs binarization processing on the input image, further analyzes the document structure of the input document image, and extracts paragraphs, character strings, and other graphic components as rectangular information. I have to. Then, in the document structure data creation unit 43, the various pieces of rectangular information extracted by the document structure analysis unit 42 are associated with each other and systematized as data expressing the document structure. At this time, the creation result of the document structure data may be displayed on the display unit 3.

The processes in the document structure analysis unit 42 and the document structure data creation unit 43 are stored in the knowledge base 45 which is interactively created by the user in advance by the operation of the operation unit 1 and the information presentation by the display unit 3. It is designed to be executed by referring to the knowledge specific to the input document.

Then, in the recognition target character string extraction unit 44, based on the knowledge peculiar to the input document stored in the knowledge base 45, the character string (that is, the character string) corresponding to the title or keyword of the input document is extracted from the document structure data. (Recognition target character string) is extracted. At this time, the image extracted as the character recognition target character string may be displayed on the display unit 3.

Reference numeral 41 denotes a knowledge acquisition support unit for supporting the work of describing and revising the "knowledge about the input document" and the "information about the character string information of the character recognition target" to be stored in the knowledge base 45.

Since the other structure is the same as that of FIG. 1 described above, the description thereof is omitted here.

Next, the processing operation of the fourth embodiment will be described.

In the present apparatus, the "character string as a keyword or title" described in the image is automatically extracted from the image input by the operator, the pattern in the character string area is recognized, and the character string is automatically recognized. It is encoded so that it can be associated with the input image for filing.

Therefore, it is not necessary for the operator to directly input the "keyword, title or the like" by a key, so that the filing work can be saved. First, the input document image is analyzed to extract character line components, word components, block (corresponding to paragraphs) components, etc., and represent them with rectangles. Then, the layout understanding process examines the logical relationship between the rectangles to assign attributes of “title, abstract, body, header, footer, caption, etc.” to each rectangle. As a result, the user can specify a desired character string, that is, a “character string that is a keyword or a title” by its attribute or logical relationship (relative position).

As a result, the flexibility is higher than the conventional method of designating the character string by the absolute position coordinates in the image,
Since it is possible to cope with a slight positional deviation or format change in the input image, there is an advantage that it is not necessary to sequentially specify the character string area when filing a large number of documents of the same format. In addition, since it is possible to specify a desired character string region using an interface that is easy to operate, it is possible to reduce the total work time as compared with the conventional case.

The image filing process in this embodiment is composed of the three processes shown in FIG.

(A) The input image is compressed (46 in the figure).

(B) An area containing a character string to be given as a title or a keyword to be added to the input document is automatically detected from the input image (47), and the character pattern included in the area is coded by the character recognition processing. (Fig. 48).

(C) The compressed input image and the coded title or keyword are associated with each other and registered in the optical disc (49 in the figure).

The process (a) here is carried out by the image compression unit 6, and the process (b) is carried out by the sentence structure analysis unit 4.
2. The document structure data creation unit 43, the recognition target character string extraction unit 44, and the character recognition unit 8 perform the process (c) in the registration unit 10.

The process (b) which is the main part of the present invention will be described below in accordance with the process procedure shown in FIG.

In this case, in the processing procedure of FIG.
From step a to step h, the document structure analysis unit 42
And the step i is the document structure data creation unit 43.
Then, steps j and k are the recognition target character string extraction unit 4
In step 4, steps 1 and m are performed by the character recognition unit 8, respectively.

Now, when an image is input from the image input section 4 using an image input means such as an optical scanner, this input image is binarized (step a), and then the binarized image The minute isolated points of are removed as noise (step b).

Next, for example, by checking the connection relationship between each black pixel from the binarized image, the labeling process for extracting the black connection region is performed, and the circumscribed rectangle of each black connection region is extracted (step c). At this time, the coordinate value at the upper left end and the height and width (or the coordinate value at the lower right end) of each black connected component rectangle are extracted as data. FIG. 18A shows an example of extracting a black connected component rectangle from the input image data.

Next, for the detected black connected component rectangle, its shape (for example, width, height, aspect ratio, black pixel density)
By checking the attribute of each black connected component rectangle, that is, each black connected component rectangle, a rectangle (hereinafter referred to as a character candidate rectangle) included in the character string area, a graphics (graphic, table,
A rectangle corresponding to a frame, a straight line component), a rectangle corresponding to an image such as a photograph, or a rectangle corresponding to noise is determined, and these are stored in the storage unit 12 (step d). This black connected component rectangle identification processing may be realized by a known method. In the subsequent processing, attention is paid to the character candidate rectangle.

Then, as shown in FIG. 18B, the marginal distribution is obtained by projecting the tip position (or end position, or the entire width of the rectangle) of each character candidate rectangle in the direction perpendicular to the character string direction, By analyzing this peripheral distribution shape, the column backward position of the input document image is detected, and the position where the beginning (or end) position of the character candidate rectangle is aligned in the direction perpendicular to the character string direction is detected. Define 50. Then, these are stored in the storage unit 12 as a part of the document structure data (step e).

In addition to the above, the column position detecting process may be realized by detecting a continuous blank area extending in the direction perpendicular to the character string. At this time, the direction of the character string of the input document can be known by referring to “information regarding the document structure of the input document” stored in the knowledge base 45. However, since this column position detection process does not need to be executed for a single column document, the user can also decide whether or not to execute it. That is, the execution / non-execution of the column position detection process is described in “Information related to the document structure of the input document” stored in the knowledge base 45 and defined by the operator. Later, with reference to this knowledge, it is possible to decide whether or not to execute the column position detection processing. In particular, if the knowledge base 45 does not define whether or not to execute the column position detection process, this process may be always executed.

A rectangle corresponding to a character line by integrating character candidate rectangles in the character string direction in accordance with the column position obtained by the column position detection processing (hereinafter referred to as a character line candidate rectangle).
Is extracted (step f). That is, in the area where no columns are detected, there is overlap in the direction perpendicular to the character string direction,
A character line is extracted by integrating character candidate rectangles that are close to each other in the character string direction in the character string direction. In the area where the column is detected, the character line candidate rectangle is extracted by not integrating the column position and similarly integrating the character candidate rectangles in the character string direction.

FIG. 19 shows an example in which character line extraction processing is applied to the image data shown in FIG. At this time,
In each character line candidate rectangle, the distance between the character candidate rectangles is analyzed, and word detection is performed by integrating adjacent character candidate rectangles to generate a word candidate rectangle (step g).
The word candidate rectangle detection process may be performed in the following procedure, for example.

(1) The distance between the character candidate rectangles is obtained and the histogram thereof is created.

(2) The shape of the histogram is analyzed and the threshold value for separating the intra-word distance and the inter-word distance is obtained.

(3) Based on this threshold value, the distances between the character candidate rectangles are sequentially evaluated, and the character candidate rectangles are integrated as words to generate word candidate rectangles.

As a result, the word candidate rectangles are sequentially managed from the left to the right in the character line candidate rectangles.
At this point, the rectangle data is character candidate rectangle, word candidate rectangle,
It is expressed and managed hierarchically like a character line candidate rectangle.

Next, as shown in FIG. 20, character line candidate rectangles that are in the same column and are adjacent to each other in the direction perpendicular to the character line are integrated, and a block (corresponding to a paragraph) is detected (step h). This block detection processing may be realized by a known method as shown in, for example, (Technical Bulletin, PRU87-89, pp.51-61, 1987.).

After the blocks are extracted in this way, the column positions are determined by using a known method as shown in, for example, (Technical Bulletin, PRU87-89, pp.51-61, 1987.). The order of blocks is determined from the physical positional relationship of the blocks. Further, the order of the character line candidate rectangles is rearranged according to the block order. At this point, the input document is represented by a rectangle for each character, a rectangle for each word, a rectangle for each character line, and a rectangle for each block.
It is hierarchically managed by the physical positional relationship in the character candidate rectangle. However, at this point, each rectangle has only information such as coordinate information and size. Then, the physical position relationship between the block and the character line candidate rectangle, the order, the logical relationship, the position and size of a black connected component rectangle (for example, a figure or a photograph) having attributes other than the character candidate rectangle, and the character candidate rectangle. By performing the layout analysis process based on the mutual relations with, etc., the semantic connection between the rectangles is acquired, and the “title”, “summary”, “body”, “header”,
Attributes such as "footer" and "caption" are assigned (step i).

In this case, the group of rectangles as shown in FIG. 21 (a) is represented by the tree structure shown in FIG. 21 (b), and the character line candidate rectangles in the block and the word in the character line candidate rectangles are the words. The candidate rectangles and the character candidate rectangles in the word candidate rectangles are represented by a tree structure. The data thus obtained is stored in the storage unit 12 as document structure data.

By the above processing, the attribute information, the semantic connection relation information, and the physical positional relation information regarding the word, the character string, and the block are obtained from the input image. The layout analysis process here is performed by, for example, “Tsujimoto, S., and
Asada, H.: "Understanding Multi-articled Document
s ", in Proc. 10th Int. Conf. Pattern Recognition, p
A known method such as that shown in p.551-556, 1990. "may be used.

To extract the character string corresponding to the title or keyword to be added to the input image, the recognition target character string extraction unit 44 refers to the knowledge base 45 previously defined by the operator and stores it in the storage unit 12. It is realized by extracting a rectangle corresponding to a desired character string from the tree-structured document structure data (step j). Then, in the knowledge base 45, a character string area that can be regarded as a title or a keyword of the input document is described by the operator. That is, the operator defines a character string corresponding to a keyword of a desired input image in the knowledge base 45 based on attributes, semantic connection relations, and physical position information. From this, the desired character string is "attribute information", "semantic connection relation information",
It is possible to extract using any of the “physical position information”.

For example, when it is desired to extract the subject (1) in a document as shown in FIG. 25, if the "base" is described in the knowledge base 45, the recognition target character string extraction unit 44 stores it. The document structure data represented by the tree structure stored in the unit 12 is searched to extract all the rectangles with the attribute of "title". At this time, a character string area to be subjected to character recognition processing may be displayed on the display unit 3 for each input document (step k). When the character string area is not correctly extracted or when a plurality of character string areas including the character string to be extracted are extracted, the operator can use the mouse of the operation unit 1 to input the correct character string area. May be instructed or specified (surrounding the actual image).

After the character string to be subjected to character recognition processing is extracted in this way, individual character patterns are then cut out from the character string area. This processing is realized by integrating the adjacent continuous black pixel areas already extracted in step c into a character pattern. In order to evaluate this proximity, a threshold value related to the distance is set, and adjacent character elements in the vertical direction of the character are integrated into one character pattern. As a result, separated characters such as "i" and "%" are integrated and extracted. Basically, the pattern integration of the character elements on the left and right is not performed. As a result, a character such as "movement" is extracted as two patterns such as "heavy" and "force". Since the two patterns extracted in this way have the possibility of being one character and the possibility of being two characters, it is necessary to consider each of these two possibilities. Therefore, in such a case, for example, character recognition may be performed on all of the plurality of cutout candidates, and finally the contextual consistency may be evaluated to determine one. Conversely, there are also character patterns in which a plurality of characters are in contact. For such a character pattern, for example, the character pattern shown in FIG. 22 (a), as shown in FIG. 22 (b), the marginal distribution (in the direction perpendicular to the character string of black pixels forming the character pattern) The position where the minimum point of the projected component) f is taken is determined to be the boundary position of the plurality of contacting character patterns. After that, by examining the pattern shape in the vicinity of the boundary position, it is determined that the recessed portion near the boundary position is the cutting position of the character string pattern, and the character string pattern is cut at the cutting position. It comes to cut out individual character string patterns.

When individual character patterns are cut out in this way, each character pattern is then associated with a category of characters to which the pattern may belong. This association is performed, for example, by calculating the degree of coincidence by superimposing the statistically created standard pattern and the unknown pattern, as in the composite similarity method (step l). As a result, the first to Mth characters associated with the unknown pattern are categorized and output in order from the character having the highest matching, together with the code of the character category and the degree of matching. The above list is the "display" 2
Is displayed on the screen (step m), and the user operates the "operation unit" 1
The recognition result may be corrected while operating.

When the title or keyword of the input document is determined by the above processing, the character string code is stored in the optical disk 11 in association with the compressed input image already stored in the storage unit 12, and the filing processing is performed. To finish. The stored data may not be the input image itself, but may be a part of it or processed data such as a recognition result.

Here, the knowledge base 45 will be specifically described. In this knowledge base 45, "information regarding the document structure" created by the operator for each document having the same format or structure and "information regarding the character string information corresponding to the title or keyword of the input document existing in the input image". Are stored respectively. Before the image filing work is performed, the knowledge is created interactively by the operator activating the knowledge acquisition support unit 41 and operating the operation unit 1 and presenting the information by the display unit 3. In addition, the operator provides "information about the document structure" corresponding to the defined input document and "information about the character string information corresponding to the title or keyword of the input document existing in the input image" from the knowledge base 45 at the time of the image filing work. Select or create a new one. The document structure analysis and understanding process and the character recognition target character string extraction process at the time of image input work are executed with reference to this knowledge.

In the "information on the document structure", for example: -character string direction of input document (vertical direction, horizontal direction) -structure of input document (number of columns) -size of input document (A4, B4, etc.)-Input document The attributes (text, table, figure, etc.) are defined, and in "Information about the character string information corresponding to the title or keyword of the input document existing in the input image", the character recognition target existing in the input document The character string area is defined.

Next, the knowledge acquisition support unit 41 used for creating the knowledge base 45 will be described. The knowledge acquisition support unit 41 is a processing unit activated by the operator when the operator constructs the knowledge base 45 before performing the image filing process, and does not operate during the filing process. The knowledge acquisition support unit 41 defines, corrects and adds "information about document structure" and "information about character string information corresponding to the title or keyword of the input document existing in the input image" for the input document.
The operator can interactively make changes, confirmations, evaluations and the like by operating the operation unit 1 and presenting information on the display unit 3. In the interactive design of knowledge in the knowledge acquisition support unit 41, for example, the operator does not need to set complicated information such as the number of characters and the coordinate position where the character exists in order to define the character string area, and the document structure can be easily set. It is possible to define the position of the character string. When the operator defines the knowledge and constructs the knowledge base 45, the external expression presented by the knowledge acquisition support unit 41 to the operator is, for example, the window 5 on the screen 51 as shown in FIG.
2 is also considered, and panel format is also considered. The window 52 is displayed at an arbitrary position on the screen by, for example, pointing to the icon 53 displayed in the icon form on the upper part (lower part, left end, or right end) of the screen 51 as shown in FIG. Is displayed in. The window used for defining the document structure or the character recognition target area may be composed of one scrollable window, or may be composed of a plurality of windows that are sequentially displayed step by step. When a window that is used to define the document structure or the character recognition target area is configured with one scrollable window, the operator sequentially selects the document structure and the character recognition target character string area from the top according to the instruction in the window. Define it. When a window used for defining the document structure and the character recognition target area is composed of a plurality of windows, the user first defines the document structure of the input document in one window and then The definition concerning the character recognition target character string area is defined in a newly generated window. In this case, each window may be automatically generated every time the definition in the previous stage is completed.

Next, using the example of the actual document shown in FIG. 25, the knowledge acquisition support unit 41 performs "information on the document structure".
And the case of defining "information regarding character string information for character recognition" will be described.

FIG. 25 shows the title page of "Information Processing Society of Japan". The title page must include the title, author's name, abstract, etc., as well as part of the text. When this document is stored as an image, it is necessary to add a search index (that is, the title of the input image) to the image. In order to automatically extract such a character string, the character string area must be defined in advance, but using the present invention makes it possible with a simple operation. Here, FIG.
The document structure of the input image and the character string area of the subject (1) in the figure will be defined using the present invention.

In order for the operator to define the document structure of the input image and the character recognition target character string, the operator must first designate the icon 53 shown in FIG. As a result, the window 52a as shown in FIG. 26 is first displayed on the screen of the display unit 3.

When the operator selects item (1) in this window 52a, the operator can apply the already defined "knowledge about the document structure and character string to be recognized" to the input document. When item (4) is selected, "knowledge about the document structure and character recognition target character string" of the input document can be newly defined using windows 52c to 52g shown in FIGS. 28 to 32.

When item (1) is selected in the window 52a shown in FIG. 26, the window 52a disappears from the display unit 3 and the window 52b shown in FIG. 27 is displayed on the screen of the display unit 3. Then, when the item (2) of the window 52b is selected, the windows 52c to 52g shown in FIGS. 28 to 32 are sequentially displayed, and some of the parameters can be changed and applied to the input document. Further, by selecting the item (3) of the window 52b, the defined "knowledge about the document structure and character recognition target character string" can be applied to the input document as it is. In this case, the definition of "knowledge about document structure and character recognition target character string" for the input document is completed.

Here, the window 52a shown in FIG. 26 is used.
Item (4) is selected to newly define "knowledge about document structure and character recognition target character string" for the document of FIG.

In order to select each described item in each window, the mouse or the like of the operation unit 1 is used to indicate the blank square mark at the beginning of each item. . In this case, when a blank square mark is designated, it changes to a black square mark. In this state, if information is written in a blank part with an underline, this input information will be adopted as knowledge about the input document. In addition, when the black square mark is specified, it changes to a white square mark, the information written in the underlined blank part is canceled (the underlined part becomes blank again), and the operator enters new information. Will be able to.

If item (4) of the window 52a shown in FIG. 26 is selected, the window 52a will be displayed on the display unit 3.
28 disappears and the window 52c shown in FIG.
Displayed on the screen. The operator has this window 5
2c will be used to define the document structure of the input image.

For the window 52c, the operator
The items are selected in order from the above items, and predetermined information is entered in the underlined blank area of each item using the keyboard of the operation unit 1 or the like. In this case, the item (5) is an item related to the definition in the character string direction of the document. Since the document shown in FIG. 25 is in horizontal writing, the user may indicate horizontal writing in item (5) with a mouse or the like. In the character line extraction process (step f) in the processing procedure of FIG. 17, the black connected component rectangles are integrated based on this information to extract the character line.

Item (6) relates to the definition of the number of columns of the input document. Since the document shown in FIG. 25 has two columns, the operator writes "2" in the underlined blank part of the item (6).
Should be entered. Here, the column extraction process (step e) is executed in the process procedure of FIG. 17 only when "2" or more is entered in the underlined blank part.

Item (7) is an item relating to the definition of the size of the input document. Since the input document shown in FIG. 25 is B5 size, the operator may enter "B5" in the underlined blank part of the item (7). Item (8) is an item relating to the definition of the attribute of the input document. Since the input document is mainly composed of text data, the operator may indicate "text" in item (8) with a mouse or the like. Here, for example, when "table" is selected, the table detection process and the table structure analysis process are executed in the document structure analysis process in the filing process. When a drawing is selected, the column detection process (step e) and the block detection process (step h) are not performed, and the character string extraction process corresponding to the drawing is performed. When the item (9) is designated with a mouse or the like, all the information defined on this window can be canceled after the window 52c is opened. When ending the definition of the document structure in the window 52c, when the item (10) is finally designated by a mouse or the like, the window 52c disappears from the screen of the display unit 3, and the window 52d is displayed on the screen.
The document recognition target area can be defined. At this time, if any one of the items (5) to (8) in the window 52c is omitted or missed in selection, the display section 3 displays “There is no appropriate definition in the window. Please display a warning message such as "Please."

In order to define the character string area of the subject (1) in the document shown in FIG. 25, first, the character string area extraction processing range is defined in the window 52d shown in FIG.

Here, when the item (41) is selected, a desired character string can be defined by the attribute. At this time, the window 52h shown in FIG. 33 is displayed on the screen, and the item corresponding to the attribute of the desired character string may be selected. With respect to the subject (1), the definition can be completed by this, and the definition by the position coordinate information will be further described.

In this case, the window 52c shown in FIG.
Since the number of columns is defined as "2" in the item (6) of (2), the item (11) may be selected in the window 52d shown in FIG. In addition, if the number of columns is set to "1" in the item (6) in the window 52c, the window 5
In 2d, if item (12) is selected and defined as "the first to the uppermost character string area extraction processing range obtained by dividing the entire image into three vertically" is defined, the document structure analysis processing during filing processing It is carried out only for a third region (upper part). Also, the window 52 shown in FIG.
In addition, in item d, if item (13) is selected, the extraction processing range can be defined as shown in FIG.
When is selected, the extraction processing range can be defined by a rectangle having the absolute coordinates of two arbitrary points as vertices. For the item selection here, in order not to leave ambiguity in the definition in the following window, one item from (11) to (14) is set to 1
You can select only once. When it is desired to end the definition in this window 52d, when the item (16) is finally designated by the mouse or the like, the window 52d disappears from the screen of the display unit 3, and the window 52e shown in FIG. 30 is displayed on the screen. It is possible to define the character recognition target character string area in more detail. The operator uses this window 52e
Thus, the definition of the area of the block including the character recognition target character string is performed.

Items (1 in the window 52e shown in FIG.
Item 7) is an item that defines the parameter referred to in the block detection process (step h) in the process procedure of FIG. In the input document shown in FIG. 25, the operator measures the lower limit of the width of the blank area that can be regarded as a separator between blocks and writes the size in mm in the underlined blank part of the item (17).

If 6 mm is appropriate for the input document shown in FIG. 25, "6" may be entered in the underlined blank part of the item (17). Item (18) defines the area of the block containing the character recognition target character string. The blocks are automatically detected and sequenced in the document structure analysis process, but the user guesses which block contains the character recognition target character string and fills in the underlined blank part of the item (18). For example, the subject (1) in FIG. 25 is included in the second block, so the user must enter "2" in the item (1
8) Fill in the blank space with underline. The item selection here allows only one block to be defined in order not to leave ambiguity in the definition in the following windows.
When item (18) is selected without selecting item (17), αmm or β times the average character line width of the input document is set in the underlined blank area of item (17). Here, if the item (18) is not selected by the user, the block extraction processing (step h) in the processing procedure of FIG. 17 is not executed. Alternatively, it may be always implemented.

Then, when the definition in the window 52e shown in FIG. 30 is desired to be ended, the item (20) is added at the end.
Window is displayed with the mouse, etc.
31 disappears from the screen, and the window 52f shown in FIG. 31 is displayed on the screen, and the character recognition target character string area can be defined in more detail.

This window 52f is for defining the character recognition target character string area within the character recognition target character string extraction range defined so far. To determine the character string area of the subject (1) of FIG. 25, the operator may select the item (21) from the items (21) to (30). That is, since the above-mentioned subject (1) is composed of all the characters (columns) included in the second block, it is necessary to recognize and code all those characters.

The definition of the character string area using the window for the document shown in FIG. 25 is completed.

Next, other items (22) to (30)
The types of character string areas that can be defined by these will be described.

Item (22) defines a plurality of (or one) continuous character line areas in a certain area (the area where the character string to be recognized as character string defined up to that point exists) as the character line to be recognized. be able to. In the item (23), a plurality of (or one line) continuous character strings existing before and after an arbitrary ruled line in a certain area can be defined as a character recognition target character string. For example, the character line area included in the bibliographic item (2) in FIG. 25 can be defined by the item (23). The item (24) can define a plurality of (or one) continuous character strings surrounded by two ruled lines included in a certain area as a recognition target character string area. Item (25) includes the character string width (vertical width for horizontal writing character string, horizontal width for vertical writing character string) of the character strings included in a certain area within the range of the numerical value written in the underlined blank part. Objects can be defined as recognition target character strings. For example, with respect to the character string of the subject (1) in FIG. 25, the character string area can be defined by sequentially inputting "3.5" and "5" in the underlined blank area. Item (26) is
A plurality of (or one) underlined character strings included in a certain area can be defined as a recognition target character string area. The item (27) can define a plurality of (or one) continuous character lines surrounded by a rectangle of a certain area as a recognition target character string area. Item (28)
Can set an outline character string included in a certain area as a recognition target character string. When this item (28) is selected, (amount of black pixels / amount of white pixels) of the character line candidate rectangles extracted in the character line extraction process (step f) of the processing procedure of FIG. 17 is γ or more. Is extracted as a white character line, and black pixels are further inverted so that it can be handled in the same manner as a normal character line. Item (29) is a character string included in the recognition target character string area defined in the previous items (21) to (28) and written by the user in the underlined blank part from the keyboard or the like. The included character line can be defined as the recognition target character string. When this item (29) is defined, all the character patterns included in the target character string area defined up to that time are recognized in the character recognition processing (step 1) of the processing procedure of FIG. Extract the character line containing the character string specified by the user from. At this time, if "Inclusion" is indicated by a mouse or the like in the item (29) (Including or removing), all the character codes including the character string designated by the user can be registered. All character codes except the character string specified by the user can be registered by pointing at. The item (30) is an item that can be selected only when any one line of the input document is defined as the recognition target character string area in any of the items (21) to (29). That is, if any one of the items (21) to (29) is selected and a plurality of lines are defined as the recognition target character string, the item (30) cannot be selected (at this time, the item (30) is selected). If selected, a warning message prompting redefinition is displayed at an arbitrary position on the display unit 3). This item (3
If 0) is selected, only the word specified by the user can be defined as the recognition target character string.

Thus, each of the above items (21) to (3
In 1), a recognition target character string area can be defined for a character string included in each area defined in the previous items (1) to (20). Therefore, when newly defining another area, it is necessary to specify the item (32) with a mouse or the like. When the item (32) is designated, the window 52d shown in FIG. 29 is displayed again, and the definition work can be continued.

To end all the definitions, first select the item (33) in the window 52f shown in FIG. 31 with the mouse or the like, and use the information defined up to that time as the "information on the document structure" and "information about the document structure" for the input document. The information regarding the character string information corresponding to the title or the keyword of the input document existing in the input image ”is stored in the knowledge base 45.

Finally, when the item (34) is designated, the window 52f disappears from the screen of the display unit 3, and the window 52f shown in FIG.
A window 52g shown in 2 is displayed on the screen. In the window 52f, various conditions are set when the defined knowledge is applied to the input document. Items (35) and (36) are defined knowledge when a plurality of documents are continuously processed. The item (35) is selected when the target documents are collectively input, and the item (36) is selected when the target documents are randomly input. ing. Both items can be defined more than once by selecting "duplicate". Item (37) is selected when the character recognition target character string area is displayed on the input image displayed on the display unit 3 or the document structure data represented by the rectangular information during the filing work. 38) displays the character recognition result on the display unit 3 during filing work,
It is selected when the correction work is performed. Then, when the item (40) is selected, a menu regarding filing work as shown in FIG. 33 is displayed, and preparation for filing processing is completed.

Here, each of the windows 52a to 5a described above.
In addition to the above-described display method, 2g is such that windows 52a to 52g are overlapped as shown in FIG. Alternatively, the tiles may be displayed as shown in FIG.
It may be defined by 2a to 52g. Further, regarding the definition regarding the area of the recognition target character string described as described above, the knowledge acquisition support unit 41 may check whether the content has “duplication” or “contradiction”. Furthermore, when "test" is selected on the operation unit 1 in the menu for controlling the activation of the knowledge acquisition support unit 41, the operator defines "character string information corresponding to the title or keyword of the input document existing in the input document". It may be possible to confirm from the sample document whether "information" is defined as desired.

Therefore, in this way, the physical position and size of the block in which the character string components extracted from the input image are integrated in a predetermined relationship are analyzed to obtain the semantic connection relationship between the blocks. Further, by adding attribute information to each block, it becomes possible to accurately detect the character string component described at any position corresponding to the title or keyword of the input image by these semantic connection relations and attribute information. , Documents with various formats can be batched while automatically adding search titles. In addition, the setting of the information regarding the title character string described in the input document can be performed interactively with the device by an appropriate display on the display unit, and the work related to the filing process by the operator can be significantly reduced. .

In the above-described fourth embodiment, (1) the direction of the character string of the input document and (2) the structure (step) of the input document are stored as "information about the document structure of the input document" stored in the knowledge base 45. Group, etc.), (3) size of input document,
(4) The attribute of the input document is defined, and the character recognition target column area is defined in "information regarding the character string information corresponding to the title or keyword of the input document existing in the input document". The type of the input document and the attributes of the character string (rectangle) that should be the keyword or title of the input document image (that is, "title", "summary", "body", "header", "footer", Any one of "captions" may be stored as knowledge. In this case, the operator inputs the document type for each input document during the image filing work. At this time, in the recognition target character string extraction unit 44, the type of the input document input by the operator and the knowledge base 4 are input.
5, the attribute of the character string to be extracted (that is, the rectangle in the document structure data) is acquired, and then the document structure data group of the input document obtained by the layout analysis processing (stored in the storage unit 12 ) To a valid rectangle (string area)
Will be extracted.

Further, in the above-mentioned fourth embodiment, it was necessary to use the knowledge acquisition support unit 41 in order to construct the knowledge base 45. However, "It corresponds to the title or keyword of the input document existing in the input document". It is also possible to define "information regarding character string information" more easily than in the fourth embodiment without using the knowledge acquisition support unit 41. This method will be described below. This method is effective when filing a large number of documents of the same format.

First, one sample document is input, document structure data is extracted by understanding the layout structure of the document, and the result is displayed on the screen 54 of the display unit 3 in word units as shown in FIG. Document structure data 55, character line unit document structure data 56, and block unit document structure data 57, respectively. Then, the operator uses a mouse or the like to indicate, by using a mouse, what can be regarded as "a character string area which should be a title or a keyword to be added to the input document" among the displayed ones, "the input document existing in the input document". The information about the character string information corresponding to the title or keyword "is created. That is, "attributes", "positions", "sizes", "orders", "logical relationships", "mutual relationships", etc. related to what is instructed with a mouse etc. are automatically extracted and systematized. Information about the character string information corresponding to the title or keyword of the input document existing in the document "
Will be stored in the knowledge base 45.

(Fifth Embodiment) In this embodiment, character recognition candidates obtained by character recognition for keyword image information corresponding to titles and keywords existing in the input image are not specified, and the character recognition candidates are not identified. It is stored as it is, and the most suitable recognition result is selected as a title or keyword of an image from a plurality of recognition candidates when an image is searched.

FIG. 38 explains up to the registration of a key word image, and here, a key word is added to an input drawing to register an optical disk or the like.

In this case, the input drawing 61 is stored in the image database 63 by operating the image with the image input device 62 such as a scanner. Here, the image is displayed on the display 64, and the operator uses the mouse or the like to designate the area to be registered as the keyword of the input image.

FIG. 39 shows an example of a screen for registering a keyword area. In this case, the keyword area 642 is shown for the image 641 of the input drawing displayed on the display 64. Keyboard area 642 in this case
As a keyword, a keyword position registered in advance or a keyword position by an operation such as automatic detection or direct designation with a mouse is given.

The keyword image 65 stored in the image database 63 and cut out from the input image is sent to the character recognition device 66, where character recognition is performed. Then, all the recognition candidates are stored in the character recognition candidate storage unit 67. In this case, the character recognition candidate storage unit 67 is shown in FIG.
As shown in (b), the recognition order is stored in descending order.

Thereafter, the input drawings are sequentially registered according to the above procedure. Note that the keyword registration, the keyword image cutout processing, the character recognition processing, and the like do not necessarily have to be performed for each image registration, and may be performed in batches.

As described above, in this embodiment, the character recognition result is not uniquely determined from the character recognition candidates but is registered as a keyword including the character recognition candidates.

FIG. 40 is for explaining retrieval of keyword images. In this case, a drawing image is selected from the image database 63 based on the keyword given by the operator. That is, the keyword 68 provided by the operator is collated with the character recognition candidate storage unit 67 by the retrieval device 69, and the drawing image 70 saved as the keyword 68 is converted into the image database. It comes to take out from 63.

Then, when the character recognition candidates are confirmed by the operator's confirmation operation 71, the surplus character recognition candidates are removed from the character recognition candidate storage unit 67, and the character recognition result is uniquely determined at this stage. become.

Next, the character recognition candidate storage section 67 will be described in detail. Fig. 41 (a) shows the cut keyword image 7
2 is shown. Then, by performing a character cutting process on the keyword image 72, FIG.
Character images are identified by cutting out the character images 721 to 727 shown in FIG.

The character recognition results are rearranged in descending order of the “similarity value” evaluated in the character recognition processing. Figure 41
The figure (c) has shown the character recognition result 73 of the extracted character images 721-727. Here, the similarity value can range from 0 to 1000.

FIG. 42 (a) shows a case in which the character recognition results up to the third place are selected as character recognition candidates, and FIG. 42 (b) shows a case in which a different number of candidates is given for each character category. In this case, the number of candidates is determined by the character category of the first character recognition result. In the case of FIG. 42 (b), one is for the characters'T 'and'Y', two are for the characters'- ',
If the characters are "0", "O", "B", and "1", four characters are given in advance.

FIG. 42C shows an example of selecting a candidate based on the similarity value in the character recognition process. It is conceivable to register only the character recognition result whose similarity value is larger than a predetermined value as a character recognition candidate. Alternatively, if the similarity value of the first place is sufficiently large and the difference between the similarity values of the second place and the second place is also large, only the recognition result of the first place is registered as a character recognition candidate, but the difference of the similarity value with the second place remains. If it is not large, a method of registering both the first and second positions of the character recognition result can be applied. FIG. 42 (d) shows a case where a character recognition candidate is selected from the character recognition result using the information on the subset (character type) of each character. Here, the keyword image 72
1st character: English capital letters 2nd character: English capital letters 3rd character: English capital letters 4th character: English capital letters 5th character: Symbol ('-' or '/') 6th character: Uppercase English letters 7th character: Numbers can be used. At this time, "5th character is always'
It is also possible to set restrictions such as'is' (hyphen). Information about this subset is keyed.
It is set by the operator when registering the image.

In character recognition of the keyword image 72, 1
There may be some candidates when extracting as characters. In the case of FIG. 41B, the character image 726 is cut out as one character, but it may be a pattern generated by contacting the characters “1” and “3”. Therefore, as shown in FIG. 43, it is possible to output the character recognition candidates including the character cutout candidates. In this case, a character cutout candidate that considers the presence of faint or separated characters may be considered. Further, if the number of characters in the keyword can be given when registering the keyword, the number of candidates for character extraction can be reduced.

Although an example of selecting character recognition candidates has been described above, if it is not necessary to compress the data amount of the character recognition candidate storage unit 67, the character recognition result 73 can be used as it is.
Further, the character recognition candidate selection methods may be used in combination.

In character recognition, the character recognition result may be rejected due to addition of noise to the character image, blurring, or an error in cutting out one character.

FIG. 44 is for explaining the processing when the character recognition result is rejected. In this case, the processing from 74 to 80 in the figure is the same as that described in FIG. When the result of character recognition by the character recognition device 80 is rejected (81), the operator is asked whether or not to type the keyboard using the keyboard (illustrated). 82). If Yes, the key word is stored in the character recognition candidate storage section 84 by the key word registration processing 83 by the operator typing. The typed keyword is registered in the first column of character recognition candidates.

On the other hand, when the reply 82 in the drawing is No to the operator, the character recognition processing is performed again by changing the parameters and the like. As a concrete method, it is conceivable to change the sensitivity of the image input device 75 to start the image input or change the parameters in the character recognition processing. Also, the registration key by the operator
It is also possible to return to the designation 77 of the code and ask the operator to confirm the cutout position. Here, when the process returns to the input of the drawing image, the input drawing is again input to the image input device 7
Must be set to 5. However, the image input device 75
Is capable of handling a multi-valued image, the input drawing is stored in the multi-valued image memory 85 as a multi-valued image,
It is possible to obtain the drawing image with the image input sensitivity changed by performing the binarization process 86. If the image filing process is performed in batches, all drawings are registered and then confirmation correction operation is performed on the drawings whose keyword registration is rejected.

Here, the character recognition results of the keywords are not uniquely determined, but they are confirmed at the time of subsequent retrieval. Therefore, if there is a request for a keyword-related process before the search, for example, a keyword list is displayed, only the first candidate among the recognition candidates is used.

Next, the retrieval of keyword images will be described with reference to FIG.

In this case, a keyword candidate 88 having a number of characters equal to the number of characters of the keyword 90 to be searched.
Is extracted from the character recognition candidate storage unit 87. Then, each keyword candidate 88 is evaluated for similarity to the keyword 90 and the registered key corresponding to the keyword 90 is evaluated.
Word 89 is decided. The similarity evaluation can be judged by whether or not a keyword is included in the character recognition candidates. Then, the drawing image corresponding to the registered keyword 89 is selected from the image database and presented to the operator. After the operator confirming operation, the character recognition result of the keyword is uniquely determined, and the character recognition candidate 87 is corrected according to the character recognition candidate 91 for the keyword. If the character recognition result is already uniquely determined, the character recognition candidate 87 is not corrected.

Similarities between character recognition candidates and keywords will be described with reference to FIGS. 46 (a) and 46 (b). in this case,
The similarity is evaluated based on the total score by assigning points such as 10 points when they match the 1st place candidate, 9 points when they match the 2nd place candidate, and so on. And in the case of the illustrated example, the key
The similarity between the word 92 and the character recognition candidate 93 is 39.
It is evaluated as a point, and the similarity with the character recognition candidate 94 is evaluated as 37 points.

As a method of giving a score, a method of giving a weight to the first place such as 1st place 10 points, 2nd place 7 points, 3rd place 5 points and a method of using a similarity value at the time of character recognition There are various possibilities such as giving the same score to similar characters.
In the method of giving the same score to similar characters, character recognition candidates 93
The second character of the second character will be given the same score as the first character. That is, the similarity to the keyword 92 is evaluated as 40 points.

Returning to FIG. 45, the retrieval device detects the registered keyword 89 corresponding to the keyword 90 given by the operator according to the similarity described above. The method will be described with reference to FIG.

In this case, in the first method, if there is a candidate having a similarity larger than the score given in advance, the candidate is registered with the key.
This is to be detected as a word, and in the example of FIG. 47 (a), a keyword candidate 9 out of keyword candidates 95 to 97 is selected.
5 is detected as a registration keyword. The second method is to evaluate the similarity with all the keyword candidates and detect the one having the highest similarity among them as the registered keyword. The example of FIG. 47 (a) Keyword candidate 9
The keyword candidate 97 out of 5 to 97 will be detected as the registered keyword. In the case of the second method, as shown in FIG. 47 (b), keyword candidates 98, 9
If 9 have the same score, the method of evaluating the similarity may be changed, or the method of presenting to the operator for selection may be adopted. When changing the method of evaluating the similarity, it is conceivable to change the score distribution for the first, second and third places.

Next, an example in which the operator is made to select will be described with reference to FIG.

In this case, FIG. 48 (a) shows a state in which drawing images 100 to 105 corresponding to the keyword candidates having the same points are displayed on the display 106. Operation
This allows the computer to select the drawing image corresponding to the keyword. Figure 48 (b) shows the key
Keyword image 10 in the drawing image corresponding to the word candidate
7 to 112 are displayed on the display 113, which allows the operator to select a keyword image and consequently the corresponding drawing image is determined. Keyer
When presenting the key image, the keyword position must be stored together with the character recognition candidates.

By the way, the search keyword may not be detected by the search device. In such a case, a similar pattern is generated. FIG. 49 illustrates this situation.

In the figure, the keyword image 114 is character-recognized and character recognition candidates 115 are obtained. When the operator inputs the keyword 116, the retrieval device 117 retrieves the registered keyword. in this case,
If the character recognition candidate is not detected by the keyword 116, the similar pattern generator 11 is based on the similar character data registered in advance in the similar character database 118.
9, the keyword similar pattern 120 is automatically generated, and the similarity between them and the character recognition candidate is searched for by the retrieval device 12
Check 1 to detect registered keywords. If the keyword cannot be retrieved even with this, the operator directly selects as shown in FIG.

FIG. 50 is for explaining the similar pattern generator 119. In this case, a similar character is registered in the similar character database 118 for each character.
In this, not only similar characters but also character cutout candidates are registered. As characters similar to the character “B” in FIG. 50, “8”, “0”, ..., “13”, ... The similar pattern does not necessarily have to be similar characters, and may be a character reflecting the erroneous recognition characteristic of the character recognition device. That is, if the used character recognition device erroneously recognizes the character pattern of the character “blame” as the character “ki”, the character “ki” may be registered as a similar pattern of the character “blame”.

As another coping method when the registered keyword cannot be detected by the retrieval device, character recognition is performed again. This is possible if the keyword image cutout position is also stored together with the character recognition candidates. If the keyword image is analyzed and the character is recognized with the intention of detecting the keyword, the character may be correctly recognized.

As shown in FIG. 51, it is also possible to clearly distinguish the data whose character recognition result is confirmed by the search from the uncertain data. In the above example, the case where the number of character recognition candidates is limited to one is defined as the definite data, but as shown in FIG. 51, the definite data is the optical disk 123 and the uncertain data is the magnetic disk 124 for the input. It may be stored in. That is, when the input keyword image is character-recognized and it is determined that the character recognition result is reliable, it is stored in the optical disc 123 as definite data, and when the character recognition result is unreliable, a plurality of character recognition candidates are obtained. If it exists, it is temporarily stored in the magnetic disk 124 as uncertain data.

When the uncertain data in the magnetic disk 124 and the definite data in the optical disk 123 are searched by the retrieval device 125, but the uncertain data is detected as the retrieval keyword. Confirms the data by the confirmation work 126. That is, the uncertain data in the magnetic disk 124 is moved to the optical disk 123 storing the definite data.

If the searching device cannot detect the registered keyword by any of the methods, the searching device searches the keyword and the drawing image other than the one in which the character recognition result is confirmed, or the key. The word image is displayed on the display as shown in FIG. 48 and can be selected by the operator.

Accordingly, in this way, character recognition is performed on the keyword image cut out from the keyword area designated for the input image, and the character recognition result is stored including the character recognition candidates. The image is searched based on the similarity to the keyword given during image search, and the character recognition result is uniquely determined from among the character recognition candidates depending on whether the presented image is the desired one. Even if the recognition of is unclear,
By leaving the correct recognition result among multiple recognition candidates, the recognition result is uniquely determined from among the recognition candidates at the time of image search, and it is not necessary to uniquely determine the recognition result at the time of image input. You will be able to search for images.

Depending on the number of the recognition candidates to be stored, the correct recognition result may not be included in the recognition results. It is effective to confirm, correct, confirm, or allow the operator to input the keyword without relying on the keyword image recognition, and to adopt the method of this embodiment when the operator does not do such a thing. Target.

[0195]

According to the present invention, when an operator designates an arbitrary point on the input image displayed on the screen, an appropriate area is set first and the input image within the area is set. The arrangement direction of the character string components is determined, the region is enlarged along the arrangement direction of the character string components based on this determination, and character recognition and coding are executed and coded for the character string components of the enlarged region. Since the input image will be filed using the character string as a keyword, the operator only needs to specify an arbitrary point on the input image to perform character recognition and code for the keyword such as title information to be given to the input image. It becomes possible to input the data in a digitized form, and the filing processing work of the operator can be significantly reduced.

Further, according to the present invention, the physical position and size of the block obtained by integrating the character string components extracted from the input image in a predetermined relationship are analyzed to obtain the semantic connection relationship between the blocks. Further, since attribute information is added to each block and a character string component described at an arbitrary position corresponding to the title or keyword of the input image can be accurately detected by the semantic connection relation and the attribute information, the same type To the desired character string in the document such as
Even if there are variations in the number of characters or the number of lines and misalignment has occurred, it is possible to accurately detect the character string component that should be the keyword, and automatically search all documents with various formats in batch. Filing can be done efficiently while giving a title, and information about the title character string described in the input document can be set interactively with the device by an appropriate display on the display unit, and the operator's filing process The work related to can be significantly reduced.

Further, according to the present invention, character recognition is performed on the keyword image cut out from the keyword area designated for the input image, the character recognition result is stored including the character recognition candidates, and the image retrieval is performed. Even if the recognition of the keyword image is unclear, it is correct in multiple recognition candidates because the image that is similar to the character recognition candidate is preferentially presented rather than the similarity of the character recognition candidate is judged by the keyword given at some time. The recognition result is left and the one having a high similarity to the keyword is selected from the recognition candidates at the time of search to present the corresponding image, and the operator can obtain a desired image. That is, an appropriate image search can be performed without confirming the recognition result of the keyword image. Further, when the image is presented, the operator determines that the desired image is obtained and gives a confirmation instruction so that the recognition result of the keyword of the image is uniquely determined as the keyword used for the search. Good.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of a first embodiment of the present invention.

FIG. 2 is a diagram for explaining a filing process according to the first embodiment.

FIG. 3 is a diagram showing an example of a document subjected to filing processing according to the first embodiment.

FIG. 4 is a diagram for explaining a filing process according to the first embodiment.

FIG. 5 is a diagram for explaining a filing process according to the first embodiment.

FIG. 6 is a diagram showing an example of generating a black pixel connected component in the first embodiment.

FIG. 7 is a diagram showing an example of integration of adjacent black pixel connected components in the first embodiment.

FIG. 8 is a diagram showing an example of extracting a black pixel connected component that may form a character string in the first embodiment.

FIG. 9 is a diagram showing an example of expanding the processing application range in the character string direction in the first embodiment.

FIG. 10 is a diagram showing a pointing device applied to the first embodiment.

11 is a diagram illustrating a correspondence relationship between a position coordinate obtained by an instruction from the pointing device in FIG. 10 and a character string component.

FIG. 12 is a diagram illustrating a method of cutting out a character pattern applied to the first embodiment.

FIG. 13 is a diagram for explaining expansion of a character string area in the second embodiment of the present invention.

FIG. 14 is a diagram illustrating a correspondence relationship between a position coordinate obtained by an instruction of a pointing device and a character string component in the third embodiment of the present invention.

FIG. 15 is a diagram showing a schematic configuration of a fourth embodiment of the present invention.

FIG. 16 is a diagram for explaining filing processing according to the fourth embodiment.

FIG. 17 is a flowchart for explaining filing processing according to the fourth embodiment.

FIG. 18 is a diagram showing generation of a black pixel connected component in the fourth embodiment.

FIG. 19 is a diagram showing generation of character line components in the fourth embodiment.

FIG. 20 is a diagram showing generation of blocks in the fourth embodiment.

FIG. 21 is a diagram showing an example of document structure data according to the fourth embodiment.

FIG. 22 is a diagram illustrating a method of cutting out a character pattern according to the fourth embodiment.

FIG. 23 is a diagram for explaining a knowledge acquisition support unit used in the fourth embodiment.

FIG. 24 is a diagram for explaining a knowledge acquisition support unit used in the fourth embodiment.

FIG. 25 is a diagram showing an example of a document filed according to the fourth embodiment.

FIG. 26 is a diagram showing a window representation in the fourth embodiment.

FIG. 27 is a diagram showing a window representation in the fourth embodiment.

FIG. 28 is a diagram showing a window representation in the fourth embodiment.

FIG. 29 is a diagram showing a window representation in the fourth embodiment.

FIG. 30 is a diagram showing a window representation in the fourth embodiment.

FIG. 31 is a diagram showing a window representation in the fourth embodiment.

FIG. 32 is a diagram showing a window representation in the fourth embodiment.

FIG. 33 is a diagram showing a window representation in the fourth embodiment.

FIG. 34 is a diagram for explaining a knowledge acquisition support unit used in the fourth embodiment.

FIG. 35 is a diagram showing a window representation in the fourth embodiment.

FIG. 36 is a diagram showing a window representation in the fourth embodiment.

FIG. 37 is a diagram for explaining a knowledge acquisition support unit used in another embodiment.

FIG. 38 is a diagram for explaining keyword image registration in the fifth embodiment.

FIG. 39 is a view showing an example of a screen when registering a keyword area in the fifth embodiment.

FIG. 40 is a view for explaining a keyword image search in the fifth embodiment.

FIG. 41 is a diagram for explaining a character recognition candidate storage unit used in the fifth embodiment.

FIG. 42 is a diagram for explaining selection of character recognition candidates in the fifth embodiment.

FIG. 43 is a diagram showing an output state of character recognition candidates including the character cutout candidates of the fifth embodiment.

FIG. 44 is a diagram for explaining the processing when the character recognition result of the fifth embodiment is rejected.

FIG. 45 is a diagram for explaining retrieval of keyword images in the fifth embodiment.

FIG. 46 is a diagram for explaining the similarity between character recognition candidates and keywords in the fifth embodiment.

FIG. 47 is a diagram for explaining detection of registered keywords in the fifth embodiment.

FIG. 48 is a view for explaining selection of word candidates by an operator in the fifth embodiment.

FIG. 49 is a view for explaining similar pattern automatic generation in the fifth embodiment.

FIG. 50 is a view showing a similar pattern generator used in FIG. 49.

FIG. 51 is a view for explaining how to handle confirmed data and uncertain data in the fifth embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Operation part, 2 ... Control part, 3 ... Display part, 4 ... Image input part, 5 ... Output part, 6 ... Image compression part, 7 ... Recognition target character string extraction part, 8 ... Character recognition part, 9 ... Dictionary. Department, 10 ... Registration Department,
11 ... Optical disc, 12 ... Storage unit, 17 ... Image, 18 ...
Title character string, 19 ... Central part, 20, 201 ... Processing applicable range, 21, 22 ... Black pixel connected component, 23 ... Character candidate rectangle, 24 ... Mouse body, 25, 26, 27 ... Button, 2
8, 31 ... Character string area, 29, 30 ... Coordinates, 41 ... Knowledge acquisition support unit, 42 ... Document structure analysis unit, 43 ... Document structure data generation unit, 44 ... Recognition target character string extraction unit, 45 ... Knowledge base, 50 ... Column position, 51 ... Screen, 52, 52a ...
52g ... window, 53 ... icon, 55 ... document structure data in word units, 56 ... document structure data in character line units, 57 ... document structure data in block units, 61 ... input screen, 62 ... image input device, 63 ... image Database,
64 ... Display, 65 ... Keyword image, 66 ... Character recognition device, 67 ... Character recognition candidate storage unit, 68 ... Keyword
Reference numeral 69 ... Search device, 70 ... Drawing image, 71 ... Confirmation operation, 118 ... Similar character database, 119 ... Similar pattern generator, 120 ... Similar pattern, 123 ... Magnetic disk, 124 ... Optical disk.

Claims (3)

[Claims] 1. A display unit for displaying an input image on a screen, a unit for designating a position coordinate of an arbitrary point on the screen of the display unit, and an appropriate region is set from the position coordinate designated by this unit. Means for determining the arrangement direction of the character string components from the input image within the area set by this means, means for expanding the area along the arrangement direction of the character string components determined by this means, A recognition target character string extraction means having a means for extracting a character string component in the expanded area, and character recognition and coding of the character string component extracted by the recognition target character string extraction means, and the coded An image filing system comprising: a means for filing a character string as a keyword of the input image in association with the input image. 2. A means for extracting character string components from an input image, a block in which each character string component extracted by this means is integrated in a predetermined relationship, and a physical position and size of these blocks are detected. Means for analyzing and obtaining a semantic connection relationship between each block, means for assigning attribute information to each block from the semantic connection relationship between each block by this means, and information for storing a predetermined character string in advance Means for extracting the character string component of the block to which the attribute information is assigned by referring to the information about the predetermined character string stored in this means, and character recognition of the character string component extracted by this means. And a means for encoding and filing the encoded character string as a keyword of the input image in association with the input image. Image filing system, characterized in that. 3. A means for designating a keyword area for an input image, a means for cutting out an image of the keyword area designated by this means from the input image as a keyword image, and a character image for the keyword image cut out by this means. A means for recognizing, a means for storing the result of character recognition by this means including the character recognition candidates, and a high similarity by judging the similarity of each character recognition candidate with respect to the keyword given at the time of image retrieval. An image filing system comprising means for preferentially presenting images.