JP2002063172A - Method for retrieving image and device for the same, and recording medium with image retrieval program recorded - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and device for retrieving an image, capable of readily realizing designation of an image imaged by image retrieval at retrieving of an image, and efficiently retrieving a desired image from among a picture data base and a recording medium for recording an image retrieval program. SOLUTION: Keywords, categories to which those keywords are belonging, images corresponding to the keywords, and the featured values of the images are managed as a classification tree, and this classification tree is transferred from a user interface part 110 to an application program 200, so as to be displayed to a image retriever. The image, corresponding to the keyword selected by the image retriever from the displayed classification tree, is displayed to the image retriever, and the featured value of the picture selected by the picture retriever is acquired, and an image DB300 is retrieved from a DB retrieving part 130 by using the featured value and category of the acquired image as a retrieval key in a classification managing part 140.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to the color of an image,
More particularly, the present invention relates to an image search method and apparatus for searching for a similar image using a feature amount including a shape as a search key. Searching Method and Apparatus for Retrieving Similar Images Using Text Showing Image Features and Categories as Search Keys When Searching for Images Desired by Image Retrievers in an Environment Where Various Images are Managed with Categories And a recording medium storing an image search program.

[0002]

2. Description of the Related Art In recent years, as personal computers have become more sophisticated and lower in price, an environment has been established in which image data can be easily browsed and processed by end users. Along with this, in order to efficiently manage the stored image data, researches for centrally managing the image data and realizing a search for desired image data have been actively conducted.

[0003] As a method of designating a search key in such a conventional image search, a first method of assigning a keyword to an image and returning the image as a result of a text search based on the keyword, and a method of assigning each image to an image, A second method of extracting image-specific data (hue, saturation, luminance, shape, size, position, etc.) and returning an image based on a similarity calculation result using the specific data as a search key There is.

[0004]

In the above-described conventional image retrieval, in the first method, since the correspondence between an image and a keyword is based on human subjectivity, the same keyword is always assigned to the same image. However, there is a problem that a search result desired by an image searcher is not always obtained.

In the second method, the image searcher specifies an image to be imaged as a search key, so that the image searcher must search for an image he or she wants from the images registered as the search key. There's a problem.

That is, in the conventional method, the search result desired by the image searcher may not be obtained, or it may be difficult for the image searcher to specify the image to be imaged when executing the image search. There is a problem that it takes time and effort to extract a desired image from the image database.

[0007] The present invention has been made in view of the above,
An object of the present invention is to provide an image search method and apparatus capable of easily realizing designation of an image to be imaged by an image searcher in an image search and efficiently searching for a desired image from an image database. It is to provide a recording medium on which a program is recorded.

[0008]

According to one aspect of the present invention, there is provided an image retrieval method for retrieving a similar image using a feature amount including a color and a shape of the image as a retrieval key. , Keywords are specified for images serving as search keys, each keyword is managed in a classification tree for each category, an image corresponding to the keyword of the classification tree specified by the image searcher is obtained, and the features of this image are obtained. The point is to search for similar images using the amount as a search key.

According to the first aspect of the present invention, a keyword is specified for an image serving as a search key, each keyword is managed in a classification tree for each category, and the classification tree specified by the image searcher is specified. Get the image corresponding to the keyword,
Since a similar image is searched using the feature amount of the image as a search key, the image searcher can easily select a desired image image by tracing the classification tree, and the method of assigning the keyword and the image is determined by the image searcher. Even if the image is different from the image, it is possible to easily search for a correct search image by checking the actual image.

The present invention described in claim 2 is the same as the claim 1.
In the invention described above, a gist is that a search is performed from images managed together with the category using a category as a search key in addition to the feature amount.

According to the second aspect of the present invention, a search is performed from an image managed together with the category using the category as a search key in addition to the feature amount. A similar image search using a specified category as a search key becomes possible.

Further, the present invention according to claim 3 provides the invention according to claim 1.
In the invention according to the second aspect, in the classification tree of the node corresponding to the specified search key, a plurality of search request statements are generated and searched according to the hierarchy, and the search result is sorted based on the hierarchical structure of the classification tree. That is the gist.

According to a fourth aspect of the present invention, in the first or second aspect, a search is performed using a partial classification tree including a classification tree node corresponding to the search key.

According to a fifth aspect of the present invention, there is provided an image retrieval apparatus for retrieving a similar image using a feature amount of an image as a search key, wherein the image database manages images together with the feature amount of each image. A keyword, a category to which the keyword belongs, an image corresponding to the keyword, a classification tree managing means for managing a feature amount of the image as a classification tree, and a classification tree managed by the classification tree management means are displayed to an image searcher. A feature amount obtaining unit that displays an image corresponding to the keyword selected by the image searcher from the displayed classification tree to the image searcher and obtains a feature amount of the image selected by the image searcher from the classification tree management unit; And a search unit that searches the image database using the acquired feature amount of the image as a search key and outputs the search result.

According to the fifth aspect of the present invention, the classification tree is displayed to the image searcher, and an image corresponding to the keyword selected by the image searcher is displayed from the displayed classification tree to display the classification tree. The feature amount of the image selected by is obtained from the classification tree management means, an image database is searched using the feature amount of the image as a search key, and the search result is output. A desired image can be easily selected, and even when the keyword and the image assignment method are different from those of the image searcher, it is possible to easily search for a correct search image by checking the actual image.

Further, the present invention according to claim 6 provides the invention according to claim 5.
In the invention described above, it is preferable that the search unit performs a search from images managed together with the category, using a category corresponding to the keyword selected by an image searcher as a search key in addition to the feature amount. Make a summary.

According to the sixth aspect of the present invention, since a search is performed from images managed together with the category using the category as a search key in addition to the feature amount, the feature amount of the image and the classification tree are used. A similar image search using a specified category as a search key becomes possible.

According to a seventh aspect of the present invention, in the fifth or sixth aspect of the present invention, the search means includes a plurality of search request statements according to a hierarchy in a classification tree of a node corresponding to the specified search key. Is generated and searched, and the search result is sorted based on the hierarchical structure of the classification tree.

The present invention described in claim 8 provides the present invention according to claim 5.
In the invention according to the sixth aspect, it is preferable that the search unit performs the search using a partial classification tree including a classification tree node corresponding to the search key.

According to a ninth aspect of the present invention, there is provided a recording medium storing an image search program for searching for a similar image by using a feature amount including a color and a shape of an image as a search key, wherein the image serving as a search key is stored. , Keywords are managed in a classification tree for each category, an image corresponding to the keyword of the classification tree specified by the image searcher is obtained, and the feature amount of this image is used as a search key. The gist of the present invention is to record an image search program for searching for similar images on a recording medium.

According to the ninth aspect of the present invention, a keyword is specified for an image serving as a search key, each keyword is managed in a classification tree for each category, and the classification tree specified by the image searcher is specified. Get the image corresponding to the keyword,
Since an image search program for searching for a similar image using the feature amount of the image as a search key is recorded on a recording medium, the distribution of the image can be improved by using the recording medium.

According to a tenth aspect of the present invention, in accordance with the ninth aspect of the present invention, there is provided an image search program for searching an image managed together with the category using a category as a search key in addition to the feature amount. The point is to record on a recording medium.

The present invention according to claim 11 is the invention according to claim 9 or 10, wherein a plurality of search request statements are generated in a classification tree of a node corresponding to the specified search key according to a hierarchy. The gist of the present invention is to record an image search program for searching and sorting search results based on a hierarchical structure of a classification tree in a recording medium.

According to a twelfth aspect of the present invention, in the ninth or tenth aspect of the present invention, there is provided an image search program for performing a search using a partial classification tree including a classification tree node corresponding to the search key. It shall be recorded in the summary.

[0025]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an image search device according to an embodiment of the present invention. The image search apparatus 100 shown in FIG. 1 manages image categories and keywords in a classification tree, provides an image searcher with an input interface for an image serving as a search key, and generates a search request in accordance with a specified image key. A user interface unit 110 for providing an image obtained as a search result, wherein an application program 200 operated by an image searcher is connected via a communication network, and a search request generation unit 120 for generating a search request A DB search unit 130 that performs a search on an image DB (database) 300 that stores and manages real images and feature amounts of the real images, and an image DB 300
, And a classification tree management unit 140 that manages a keyword, a category to which the keyword belongs, an image file name corresponding to the keyword, and a feature amount of the image as a classification tree. The image DB3
00 has a category image correspondence table 310 described later.

The user interface unit 110 accesses the classification tree management unit 140, transmits the obtained classification tree to the application program 200, and inputs from the application program 200 operated by the image searcher the key image characteristic amount. The data is received, and the input data is transmitted to the search request generation unit 120. Further, it receives a similar image as a search result from the DB search unit 130 and returns it to the application program 200.

The search request generation unit 120 generates a search request based on the feature amount of the key image received from the user interface unit 110, and transmits the search request to the DB search unit 130. DB
The search unit 130 receives the search request from the search request generation unit 120, and makes an inquiry to the image DB 300 based on the search request. In addition, a similar image is obtained as a search result and returned to the user interface unit 110.

The classification tree management unit 140 registers and manages a keyword, a category to which the keyword belongs, an image file name corresponding to the keyword, and an image feature amount as a classification tree, and stores this classification tree information via the user interface unit 110. To the application program 200.
The classification tree management unit 140 is designed to edit the classification tree by a system administrator. Image DB management unit 1
50 edits the category image correspondence table 310 included in the image DB 300. This editing authority is owned by the system administrator.

The application program 200 is a program operated by an image searcher, and is capable of displaying a classification tree including categories and keywords. It also has a key image feature amount transmission function and a similar image display function as a search result.

The image DB 300 manages real images and feature amounts of each real image, and transmits similar images as search results in response to a search request. Further, it has a correspondence table 310 between each real image and a category. This category-image correspondence table 310
Are managed in the image DB 300, and when a category is specified in addition to the feature amount of the key image as the search request, the category of the image can be narrowed down from the correspondence table.

Next, the operation of the image retrieval apparatus configured as described above will be described.

First, the preparation phase will be described with reference to the flowchart shown in FIG. In this preparation phase, first, as a key image setting, a feature amount of a key image serving as a condition for image search is extracted (step S1 in FIG. 2).
1). This feature amount is extracted by issuing an extraction request to the image DB 300 by the image DB management unit 150.

Next, a category and a keyword are determined, and a classification tree having, as an attribute value, a feature amount of a key image serving as a condition for image retrieval extracted in step S11 as described above is constructed (step S13). This classification tree is constructed by the system administrator for the classification tree management unit 140. FIG. 3 shows an example of constructing a classification tree in which categories have two layers.

Next, for the image DB 300 which manages the actual images and the feature amounts, the category to which each image belongs is managed by the category image correspondence table 310 (step S15). The management of the image DB 300 and the category is performed by the system administrator for the image DB management unit 150. FIG. 4 shows a setting example of the category image correspondence table 310.

As described above, the classification tree is constructed, and the correspondence between each image and the category in the image DB 300 is managed in the category image correspondence table 310. Next, the image searcher can use the image search apparatus to obtain a desired image. The execution phase of searching for an image will be described.

First, a classification tree composed of categories and keywords managed by the classification tree management unit 140 is transmitted from the classification tree management unit 140 to the application program 200 via the user interface unit 110, and displayed on the application program 200. FIG. 5 shows an image of the screen in this case.

Next, while browsing the classification tree displayed in the application program 200, the image searcher judges the category closest to the image imaged by the image searcher from the top of the classification tree, and selects a keyword. Perform Here, a case where “clownfish” is selected will be described as an example. Then, the image names (clownfish 1.jpg, clownfish 2.jpg) of the key image candidates below the keyword (clownfish) selected in this way are obtained from the classification tree, and displayed as the key image candidates in the application program 200. .

Next, the image closest to the image imaged by the image searcher is selected from the images displayed in this manner (clownfish 1.jpg, clownfish 2.jpg). Here, it is assumed that “clownfish 2.jpg” is selected as an example.
Then, the feature amounts (a3, b3, c3, d3) of the key image selected in this way are acquired from the classification tree management unit 140. At the same time, the category (animal, fish) to which the keyword “clownfish” belongs is also acquired from the classification tree management unit 140, and the data is transferred to the search request generation unit 120 via the user interface unit 110.

Upon receiving the data, the search request generator 120 generates the following search request.

[0040]

[Table 1] Search request 1-1 Search item: Image name Search condition: Category 2 matches and features close to each other (fish here) Search request 1-2 Search item: Image name Search condition: Category 1 matches Items belonging to category 2 and having similar features (birds and mammals in this case) Search request 2 Search item: Image name Search condition: Items matching category 1 and having similar features Search request 3 Search item: Image name Search conditions In the classification tree shown in FIG. 3, since the category has two levels, the above-described three patterns of search requests are generated. However, if there are N levels, the search request is direct ( N + 1) search requests for the number of patterns and siblings associated with the direct line (excluding entries under the top) are generated. FIG. 6 shows the priority of generating a plurality of search requests generated in this way.

The search request generated as described above is sent from the search request generation unit 120 to the DB search unit 130,
B search unit 130 transmits the search request to image DB 300
An SQL sentence is generated so as to be able to be searched for.

The DB search unit 130 first transmits the SQL sentence (the lowest category, fish here) generated based on the search request 1-1 to the image DB 300, and searches the image DB 300. The hits are returned as search results. Next, an SQL sentence (birds and mammals in this case) generated based on the search request 1-2 is transmitted to the image DB 300 to search the image DB 300, and a hit is returned as a result of the search.

Then, S generated based on search request 2
The QL sentence (category belonging to the next lower rank in the lowermost layer, here, animal) is transmitted to the image DB 300 to search the image DB 300, and a hit is returned as a result of the search. As described above, the search including the higher-order category as the search request is sequentially performed while the sibling relationship of the corresponding entry is preferentially generated as the search condition in order from the search request having the lower category. Here, the category image correspondence table 310
If a category that does not exist in is used as a search request, a normal feature amount search is performed.

In this case, “Kasago.jpg” is used as “the category 2 matches” which is a part of the search condition of the search request 1.
"Hanakumanomi.jpg" and "Karemanomi.jpg" are hits, and they are sorted in the order of feature amount and returned as search results.

The search result is returned from the DB search unit 130 to the application program 200 via the user interface unit 110 and displayed.

As described above, in the image retrieval apparatus according to the present embodiment, a variety of images are managed together with categories in an image retrieval engine capable of retrieving similar images using feature amounts including colors and shapes of the images as retrieval keys. In order to efficiently search for similar images in an environment, the classification tree management unit 140 manages the keyword, the category to which the keyword belongs, the image file name corresponding to the keyword, and the feature amount of the image as a classification tree. By tracing the classification tree based on the category, selecting a corresponding keyword, confirming the image file name corresponding to the selected keyword, acquiring the feature amount of the real image from the confirmed image file name, By searching for similar images using an image search engine based on the amount and the category to which the keyword belongs An image image searcher image when performing an image search easily specified, is intended to return as a search result with priority image that matches the category.

As a result, it becomes possible for an image searcher to specify an image serving as a search key for each keyword by following the category. When a plurality of images correspond to each keyword, an image key can be selected by specifying an image to be imaged by the image searcher from among the plurality of images. Furthermore, it is possible to preferentially return an image that matches the category to which the keyword belongs as a search result. In addition, the search result has advantages of simply returning an image as a search result by a keyword and a search by a similarity calculation result based on a characteristic amount unique to each image, thereby improving search efficiency.

The processing procedure of the image search method of the above embodiment is recorded as a program on a recording medium, and this recording medium is incorporated in a computer system, and the program recorded on the recording medium is downloaded or installed in the computer system. By operating the computer system with the program, it is of course possible to function as an image search device that performs an image search method. By using such a recording medium, the distribution of the image can be enhanced. You can do it.

[0049]

As described above, according to the present invention,
A keyword is specified for an image serving as a search key, each keyword is managed in a classification tree for each category, an image corresponding to the keyword of the classification tree specified by the image searcher is obtained, and the feature amount of this image is obtained. Searches for similar images using the search key as a search key, so that the image searcher can easily select the desired image image by keyword by tracing the classification tree. Even in the case where the assigning method is different from the image searcher's image, it is possible to easily search for the correct search image by checking the actual image.

Further, according to the present invention, a search is performed from images managed together with the category using the category as a search key in addition to the feature amount, so that the image is defined by the feature amount of the image and the classification tree. A similar image search using a category as a search key becomes possible.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image search device according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a processing procedure in a preparation phase of the image search device according to the embodiment illustrated in FIG. 1;

FIG. 3 is a diagram showing an example of constructing a classification tree in the image search device of the embodiment shown in FIG. 1;

FIG. 4 is a diagram showing an example of a category image correspondence table in the image search device of the embodiment shown in FIG.

5 is a diagram showing an example of a screen image of a classification tree displayed in an application program in the image search device of the embodiment shown in FIG.

FIG. 6 is a diagram showing a priority order of generation of a plurality of search requests in the image search device of the embodiment shown in FIG. 1;

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 image search device 110 user interface unit 120 search request generation unit 130 DB search unit 140 classification tree management unit 150 image DB management unit 200 application program 300 image DB 310 category image correspondence table

Continuation of front page (72) Inventor Takashi Hoshi 2-3-1, Otemachi, Chiyoda-ku, Tokyo F-term in Nippon Telegraph and Telephone Corporation (reference) 5B050 BA15 CA07 EA04 EA07 FA02 FA19 GA08 5B075 ND06 NK06 NK31 NK39 NK43 NK46 NR12 PP03 PP13 PQ02 5L096 AA02 AA06 FA74 GA38 JA03 JA22 KA09 MA07

Claims (12)

[Claims] An image search method for searching for a similar image using a feature amount including a color and a shape of an image as a search key, wherein a keyword is specified for an image serving as a search key, and each keyword is classified into a category An image search method characterized by separately managing a classification tree, obtaining an image corresponding to a keyword of the classification tree specified by an image searcher, and searching for a similar image using a feature amount of the image as a search key. . 2. The image search method according to claim 1, wherein in addition to the feature amount, a search is performed using a feature amount of an image managed together with an upper category of the designated keyword as a search key. 3. A method of generating and searching a plurality of search request sentences according to a hierarchy in a classification tree of a node corresponding to the specified search key, and sorting search results based on a hierarchical structure of the classification tree. 3. The image retrieval method according to claim 1, wherein: 4. The image search method according to claim 1, wherein the search is performed using a partial classification tree including a classification tree node corresponding to the search key. 5. An image search apparatus for searching for a similar image using a feature amount of an image as a search key, wherein the image database manages the image together with the feature amount of each image, a keyword, a category to which the keyword belongs, and a keyword A classification tree management means for managing the corresponding image and the feature amount of the image as a classification tree; displaying the classification tree managed by the classification tree management means to an image searcher; Displaying an image corresponding to the keyword selected by the image searcher and obtaining a characteristic amount of the image selected by the image searcher from the classification tree management unit; An image search device for searching the image database as a search key and outputting search results. 6. The search means, in addition to the feature amount,
6. The image search device according to claim 5, wherein a search is performed from images managed together with the category, using a category corresponding to the keyword selected by the image searcher as a search key. 7. The search means generates and searches a plurality of search request sentences according to a hierarchy in a classification tree of a node corresponding to the specified search key, and searches for a search result based on a hierarchical structure of the classification tree. 6. The method according to claim 5, wherein
Or the image search device according to 6. 8. The image search apparatus according to claim 5, wherein the search unit performs the search using a partial classification tree including a classification tree node corresponding to the search key. 9. A recording medium storing an image search program for searching for a similar image using a feature amount including a color and a shape of the image as a search key, wherein a keyword is designated for the image serving as a search key. The method is characterized in that each keyword is managed in a classification tree for each category, an image corresponding to the keyword of the classification tree specified by the image searcher is obtained, and a search for similar images is performed using the feature amount of this image as a search key. Recording medium on which an image search program is recorded. 10. The recording medium according to claim 9, wherein a search is performed from images managed together with the category using a category as a search key in addition to the feature amount. 11. A method of generating and searching a plurality of search request sentences according to a hierarchy in a classification tree of a node corresponding to the specified search key, and sorting search results based on a hierarchical structure of the classification tree. A recording medium on which the image search program according to claim 9 or 10 is recorded. 12. The recording medium according to claim 9, wherein a search is performed using a partial classification tree including a classification tree node corresponding to the search key.