JPH07210576A - Image retrieving device, image storage device and retrieving method - Google Patents

Abstract

PURPOSE:To provide a document retrieving device for easily performing retrieval. CONSTITUTION:This device is provided with a storage means 107 for storing information, character input means 104 for inputting character strings, image input means 101 for inputting images, image display means 102 for displaying images, image area indicating means 103 for indicating the partial areas of images, and controller 109 for controlling these means. The image containing the character string fetched from the image input means 101 is stored in the storage means 107, the recognition table of the storage means 107 stores information provided with an index for identifying the stored image, the area information of the image indicated by the image area indicating means 103 and the character string provided by performing character recognition to that area and concerning the image to be displayed on the image display means 102, when the image area information in the recognition table is referred to and already recognized in the case of indicating the area with the image area indicating means 103 for retrieving the image again, the correspondent image is provided based on the index for identifying the image in the recognition table.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a search device for an image containing characters, a storage device, and an image search method.

[0002]

2. Description of the Related Art There are several methods used to store an image containing characters as image information (hereinafter referred to as "document image"). As one of them, there is a method of accumulating character information as an image. This method has a feature that the format to be read is saved because the document is stored in the read format. In the document image search by this method, keywords that characterize the image are registered in advance for the document image that has been stored, the character string input when searching is compared with each keyword, and the document of the corresponding keyword is searched. This is done by taking out the image. For inputting a character string in keyword registration, in addition to a method of inputting using a keyboard or the like, as shown in Japanese Patent Laid-Open No. 03-278269, an area of a displayed document image is designated and the designated area is designated. There is also a method of character recognition and using the character recognition result as a keyword.

FIG. 5 shows an example of the configuration of a conventional document image storage / retrieval device which stores such a document image as it is and inputs a keyword by a character recognition method.

Reference numeral 501 is an image input means. The image input means 501 takes in a character image as bitmap information. For example, the image input device 501 is an image scanner.

Reference numeral 502 is an image display means. The image display unit 502 displays image information, and the user can see this display. For example, the image display unit 502 is a liquid crystal or C
Bitmap displays such as RT displays.

Reference numeral 503 is an image area designating means. The user can specify the area or point of the document image displayed on the image display means 502 by using the image area designating means 503.

Here, as an example, the image area designating means 502
A method of designating an area when is a pointing device such as a mouse will be described. If the area to be specified is a rectangle, by specifying both ends of the line segment with the mouse, a rectangular area having the line segment as a diagonal line can be specified. For example, the mouse button may be pressed at the upper left apex of the rectangular area to be designated, the mouse may be moved to the lower right apex of the rectangular area while the button is being held, and the button may be released.

Reference numeral 504 is a character input means. For example, the character input unit 504 is a keyboard. Character input means 5
04 inputs a character string that can be used as a keyword.

The character input means 504 is replaced by the image display means 502.
It is also possible to use a transparent tablet that can be input with a pen attached to the display surface of. When using a tabt, it is also possible to input characters by recognizing characters written with a pen or the like. Also, one tablet is used as the image area designating means 303 and the character inputting means 50.
Sharing with 4 is also conceivable.

Reference numeral 505 is a document image. Document image 505
Is read from the image input means 501. It is assumed that the document image 505 handled by this apparatus includes not only figures and pictures but also characters as in a general document.

Reference numeral 506 is a keyword table. This keyword table will be described based on the example of FIG.

Reference numeral 601 is a keyword table. An attribute “image ID” 602 stores an image ID for identifying a document image
Is. An attribute “keyword” 603 represents a keyword added to the document image. A group 604 is registered in the keyword table. The set 604 is the attribute “image ID”
This indicates that the keyword “natural language” is registered in the image in which the value of 602 is 5.

In FIG. 5, 507 is a storage means. The storage unit 507 is a secondary storage device in the computer. For example, the storage unit 507 is a hard disk, an optical disk, or the like. The storage unit 507 stores a keyword table 506 and a plurality of document images 505.

Reference numeral 508 is a memory. Memory 508
The program and data are stored in. For example, the memory 508 is a semiconductor memory in a general computer system.

Reference numeral 509 denotes a central processing unit, and this unit 5
An image input unit 501, an image display unit 502, an image area designating unit 503, a character input unit 504, a storage unit 507, and a memory 508 are mutually connected and control these. In particular, the central processing unit 509 is the memory 5
The program stored in 08 is executed to perform the above control. Further, the central processing unit 509 has a character recognition function. The central processing unit 509 and the memory 508 form a control means.

Next, the operation of the above embodiment will be described with reference to the flowchart.

FIG. 7 is a flowchart showing a procedure for registering a keyword in a document image in the document image storage / retrieval device. Hereinafter, each step will be described in order.

(Step 701) The area of the document image displayed on the image display means 502 is changed to the image area designating means 5.
Specify using 03.

(Step 702) The central processing unit 5 processes the document image displayed in the area designated in step 701.
Character recognition is performed by 09. A general method can be used for the character recognition technology used here. Literature "Introduction to character recognition"
(Shin-ichiro Hashimoto, Ohmsha, 1982) gives an overview of character recognition technology.

(Step 703) The result of character recognition in step 702 is corrected. Generally, there are cases where character recognition does not result in correct recognition. In this case, it is necessary to find and correct the incorrect part of the recognition result. For example, after the character recognition result is displayed on the image display unit 502, if the recognition result is incorrect, the user inputs the correct recognition result from the character input unit 504 to correct it.

(Step 704) The recognition result of step 703 is registered as a keyword of the document image. The keyword registration is the character string that is the recognition result and the attribute "image ID".
This is done by adding a set consisting of 602 and 602 to the keyword table 506.

FIG. 8 is a flowchart showing a procedure for searching a document image by a keyword in the document image storage / retrieval apparatus. Hereinafter, each step will be described in order.

(Step 801) A keyword is input from the character input means 504.

(Step 802) A document image in which the keyword input in step 801 is registered is selected.
That is, the keyword table 506 having the keyword input in step 801 as the attribute “keyword” 603.
Search for pairs. Then, the attributes of the retrieved set “image I
The value of “D” 602 is selected. For example, when the keyword table is shown in FIG. 5 and the recognition result of step 702 is "natural language", the set 604 is searched, so that the attribute "image ID" is found.
The value “5” of 604 is selected.

(Step 803) The document image selected in step 802 is read from the storage means 507 and displayed on the image display means 502. For example, when the image ID of the document image selected in step 802 is “5”, the document image whose attribute “image ID” 602 is “5” stored in the storage unit 507 is read, and this document image is displayed. The image is displayed on the image display unit 502.

[0026]

However, when a certain document image is displayed as a result of the search during the search operation,
In order to know more details about an arbitrary word in the document image, it is sometimes desired to search for the word.

Even in such a case, in the conventional apparatus, although the keyword to be input as an image is currently displayed, it is necessary to manually input the keyword, which is complicated.

There is also a strong demand for reducing the time and effort required for keyword registration.

In consideration of the problems of the conventional document image storage device, the present invention realizes an image search device, an image storage device, and a search method that can easily perform re-search and keyword registration. It is intended.

[0030]

According to the present invention of claim 1, a storage means for storing information, a character input means for inputting a character string, an image input means for inputting an image, and an image for displaying an image. Display means, image area instruction means for instructing a partial area of the image, control means for controlling the storage means, the character input means, the image input means, the image display means, and the image area instruction means An image including a character string captured from the image input means is stored in the storage means, an index for identifying the stored image, area information of the image instructed by the image area instructing means, and A group having at least a character string obtained by character recognition of the area is stored in the recognition table of the storage means, and in the image displayed on the image display means, the image area finger is used for re-searching the image. When the area is designated by the means, if the image area information in the recognition table is referred to and already recognized, the corresponding target image is obtained based on the index for identifying the image in the recognition table. It is an image search device.

Further, according to the present invention of claim 2, a storage means for storing information, a character input means for inputting a character string, an image input means for inputting an image, and an image display means for displaying the image, An image area designating unit for designating a partial area of the image, a storage unit, the character input unit, the image input unit, the image display unit, and a control unit for controlling the image region designating unit, An image including a character string taken in from the image input means is stored in the storage means, an index for identifying the stored image, area information of the image instructed by the image area instructing means, and character recognition of the area. A group having at least the character string obtained by the above is stored in the recognition table of the storage means, and in the image displayed on the image display means, in the image area designating means for keyword registration. When designating an area, the image area information in the recognition table is referred to, if already recognized, the character string in the recognition table is registered as a keyword, and if not registered, the area information An image storage device is characterized in that a set of a character string obtained by character recognition of the area and a corresponding image identification index is registered in a recognition table, and a keyword is registered at this time.

According to the third aspect of the present invention, an image including a character string is displayed on the screen by an arbitrary keyword in a predetermined database, a predetermined character string in the displayed image is designated as an area, and the area is designated. Based on the character string, another image having the character string as a keyword is re-searched from the database and displayed, and displayed.

[0033]

According to the first aspect of the present invention, in the image displayed on the image display means, when the area is designated by the image area designating means for re-image retrieval, the image area information in the recognition table is referred to. If already recognized, the corresponding target image is obtained based on the index for identifying the image in the recognition table.

According to the present invention of claim 2, in the image displayed on the image display means, the image area information in the recognition table is referred to when the area is designated by the image area designating means for the registration of the keyword. , If already recognized,
The character string in the recognition table is registered as a keyword, and if not registered, a new recognition table of the area information, the character string obtained by character recognition of the area, and the corresponding image identification index is newly added. Create and register keywords.

[0035]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a detailed diagram of a document image retrieval apparatus according to an embodiment of the present invention. 101 is an image input means. The image input means 101 takes in a character image as bitmap information.

Reference numeral 102 is an image display means. The image display means 102 displays image information. The user can see the display of the image display means 102.

Reference numeral 103 is an image area designating means. The user can specify the area of the document image displayed on the image display means 102 by using the image area designating means 103. You can also specify points as well as areas.

Reference numeral 104 is a character input means. The character input means inputs a character string.

Reference numeral 105 is a document image. Document image 105
Is read from the image input means 101. FIG. 2 is an example of a document image.

Reference numeral 106 is a recognition table. Here, the recognition table 106 will be described with reference to FIG. Reference numeral 301 in FIG. 3 is a recognition table. Here, 302 is an attribute “image ID” that stores an image ID for identifying the document image 105.

Reference numeral 303 is an attribute "image area" for designating an area in the image represented by the attribute "image ID" 302.

Reference numeral 304 denotes an attribute "character recognition result" for storing the result of character recognition of the area of the image designated by the attribute "image ID" 302 and the attribute "image area" 303.

Reference numeral 305 denotes an attribute "keyword flag" indicating whether or not the attribute "character recognition result" 304 is a keyword. When the value of the attribute “keyword flag” 305 is 1, it is the same value as that of a keyword.

306, 311, 312, etc. are the sets stored in the recognition result table 301.

Reference numeral 307 denotes the attribute “image ID” 30 of the set 306.
It has a value of 2. The value 307 indicates that the image ID of the set 306 is related to the image of 5.

Reference numeral 308 denotes an image area ((722, 25
1), (1341, 354)) (see FIG. 2). However, ((a, b), (c, d)) has two points (a,
b) and (c, d) are assumed to represent a rectangular area having a diagonal line segment.

Reference numeral 309 indicates that the character recognition result of the image area 308 is the character string "user I / F".

Reference numeral 310 is the value of the keyword flag of the set 306. Since this value is 0, the character recognition result 309 “user I / F” is not a keyword.

Reference numeral 107 is a storage means. Storage means 107
Is a secondary storage device in a computer. Storage means 107
Stores a recognition table 106 and a plurality of document images 105.

Reference numeral 108 is a memory. Memory 108
The program and data are stored in.

Reference numeral 109 denotes a central processing unit, which includes image input means 101, image display means 102, and image area designating means 1.
03, the character input means 104, the storage means 107, and the memory 108 are mutually connected, and these are controlled and character recognition is performed. The central processing unit 109 is the memory 10
The above-mentioned control and the like are performed by executing the program stored in 8. The central processing unit 109 and the memory 1
08 constitutes a control means.

Next, the operation of the above embodiment will be described based on the flowchart.

FIG. 4 is a flowchart showing the operation procedure of keyword registration and keyword search in the document image storage / retrieval apparatus. Hereinafter, each step will be described in order.

(Step 401) Now the image display means 10
1, a document image 105 is displayed, and a scene in which an arbitrary word about the document is registered as a keyword,
Alternatively, the corresponding document image 105 is searched by using a predetermined keyword and is displayed on the image display means 101, and in the document, further, in a scene where an arbitrary word is to be searched again, the arbitrary word is changed to a region. In order to specify, a point on the document image 105 is specified by the image area specifying means 103.

(Step 402) If the point designated in Step 401 is included in the area registered in the recognition table 301, the process proceeds to Step 407. For example, since the point 206 in FIG. 2 has coordinates (1131, 339), it is within the already registered area 205 (registered as a group of 306 in FIG. 3). If not included, the process proceeds to step 403.

(Step 403) The coordinate on the image is further incremented by 1.
Specify the points to determine the area. For example, in step 403, the coordinates (52,
Consider the case where the point 207 of 633) is designated. In step 403, by further specifying the point 208 of the coordinates (420, 745), the regions ((52, 633), (42
Area 209 representing 0,745)) is determined.

(Step 404) The central processing unit 109 performs character recognition on the document image in the area 209 determined in step 403.

(Step 405) If the recognition result of step 404 is incorrect, it is corrected. For example, consider a case in which the area of the document image representing "character recognition" is erroneously recognized as "again recognition" in step 404. At this time, the user uses the character input means 104 to input the character string “character recognition” which is the correct recognition result.

(Step 406) The table which recognizes the group of the area recognized in step 405, the recognition result and the document image ID is recognized.
Register to 1. However, for the time being, the value of the keyword flag 305 is set to 0 in this step.

(Step 407) Attribute "keyword flag" in the set corresponding to the character recognition result in the recognition table 301.
If the value of 305 is 0 and it is not the keyword of the document, the process proceeds to step 408, while if it is 1, the process proceeds to step 4
Move to 10.

(Step 408) If it is not the keyword, the user is inquired whether or not the recognition result is to be the keyword. The reply to the inquiry is made by the character input means 104, for example. During the re-search operation, it is not usually used as the keyword of the document. That is, when it is answered that the keyword is not used, the process proceeds to step 410. Further, when the keyword of the document itself is exceptionally used even during the re-search operation, or the keyword is used during the keyword registration operation, the process proceeds to step 409.

(Step 409) The recognition result is registered as a keyword by setting the value of the keyword flag of the set corresponding to the recognition result to 1. This completes the keyword registration work.

(Step 410) In the re-search work, the recognition table is searched using the recognition result as a keyword. For example, when the recognition result is the character string “character recognition”, the recognition table 301
From among those, a set in which the value of the attribute “character recognition result” is “character recognition” is searched. As a result, in addition to the set 311 in which the original recognition result is stored, the set 312 in which the value of the attribute “image ID” 302 is 8 is searched.

(Step 411) The document image corresponding to the search result of step 410 is displayed.

In this way, since the area information can be used in the keyword registration work, the recognition work can be omitted and the operation becomes simple.

Further, during the re-search work, it is not necessary to newly input a keyword as in the conventional case, and when the area is used, the recognition work can be omitted if the area is already registered.

In the case of the re-search, there is no area information, and the re-search can be performed through the character string recognition. Even in this case, it is possible to save the trouble of inputting a keyword newly existing on the screen from the keyboard as in the conventional case.

[0069]

As is apparent from the above description,
INDUSTRIAL APPLICABILITY The present invention can realize an image search device, an image storage device, and a search method that can easily perform re-search and keyword registration.

[Brief description of drawings]

FIG. 1 is a block diagram of an apparatus according to an embodiment of the present invention.

FIG. 2 is an example of a document image according to an embodiment of the present invention.

FIG. 3 is an example of a recognition table in one embodiment of the present invention.

FIG. 4 is a flowchart of a keyword registration procedure and a search procedure according to an embodiment of the present invention.

FIG. 5 is a block diagram of a device in the related art.

FIG. 6 is an example of a keyword table in the related art.

FIG. 7 is an example of a keyword registration procedure in the related art.

FIG. 8 is an example of a search procedure in the related art.

[Explanation of symbols]

 101 Image Input Means 102 Image Display Means 103 Image Area Designating Means 104 Character Input Means 105 Document Images 106 Recognition Tables 107 Storage Means 108 Memory 109 Central Processing Unit (Control Means)

Claims (4)

[Claims] 1. Storage means for storing information, image input means for inputting an image, image display means for displaying the image,
An image area designating unit for designating a partial area of the image, a storage unit, the image input unit, the image display unit, and a control unit for controlling the image region designating unit are provided. An image including the captured character string is stored in the storage means, and an index for identifying the stored image,
A group having at least area information of the image designated by the image area designating means and a character string obtained by character recognition of the area is stored in the recognition table of the storage means and displayed on the image display means. In the identified image, when the area is instructed by the image area instructing means for re-searching the image, the image area information in the recognition table is referred to, and if the image is already recognized, the image in the recognition table is displayed. An image retrieval apparatus, which obtains a corresponding target image based on an index to be identified. 2. Storage means for storing information, image input means for inputting an image, image display means for displaying the image,
An image area designating unit for designating a partial area of the image, a storage unit, the image input unit, the image display unit, and a control unit for controlling the image region designating unit are provided. An image including the captured character string is stored in the storage means, and an index for identifying the stored image,
A group having at least area information of the image designated by the image area designating means and a character string obtained by character recognition of the area is stored in the recognition table of the storage means and displayed on the image display means. In the registered image, when the area is instructed by the image area instructing means for the registration of the keyword, the image area information in the recognition table is referred to, and if already recognized, the character string in the recognition table is changed. Registered as a keyword, and if not registered, the area information and
An image storage device characterized in that a set of a character string obtained by character recognition of the area and a corresponding image identification index is registered in a recognition table, and at this time, a keyword is registered. 3. An image containing a character string is displayed on a screen by an arbitrary keyword in a predetermined database, a predetermined character string in the displayed image is designated as an area, and based on the character string designated as the area. An image retrieval method, wherein another image having the character string as a keyword is re-retrieved from the database and displayed. 4. The image retrieval method according to claim 3, wherein the region-specified character string is obtained by a character recognition operation.