JPH11134344A - Method for classifying and storing picture and device therefor, and method for retrieving picture and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and device for efficiently operating a manual picture classifying and storing work to improve picture retrieving precision, and to provide a method and device for retrieving a picture to improve picture retrieving precision based on the above result. SOLUTION: At first, one picture is selected, and conventional similar picture retrieval is operated. Then, a picture which is judged to be similar among the several retrieved results is marked. Tag information is registered in those marked pictures as similar one. Then, one of pictures with which any tag information is not added is arbitrarily selected, and the similar picture retrieval is operated again, and a picture which is judged to be similar is marked. This work is repeated so that this work can be sequentially and efficiently attained, and the picture judged to be similar by a human being can be classified. Also, at the time of retrieving a picture, a picture having the same mark as that of a picture to be retrieved is preferentially retrieved so that the retrieved result which is further close to the sense of human being can be obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for classifying similar images in a system for performing a similar search for image information, a method for preferentially searching based on a result of the classification, and a device therefor.

[0002]

2. Description of the Related Art Conventionally, when searching for similar images, a method has been adopted in which images having similar colors, colors, and contours in the order of similarity among images are selected. In this image search method, a result is obtained not with binary values of similar or dissimilar but with a numerical value (similarity) indicating how similar the image is. That is, a method of sorting by this numerical value and returning the result in the order of similarity is adopted.
Here, it is called ranking. In other words, the higher the rank (similarity is the lowest or the highest rank (it depends on the search engine, but it is common that the lower the number is, the more similar it is)) It is said. Here, since the result is returned for all the existing images, the number of results is limited by the cutting method, that is, the top several or the degree of similarity.

[0003] However, even if it is determined that the similarity is high by this method, it may be determined that it is not similar to a case where a human determines. Since humans also consider similarity due to meaning, this difference cannot be eliminated only by similarity in color or outline. At present, in order to eliminate this difference, the images are classified by human beings that are judged to be similar and the classification based on the meaning of the images is manually performed in advance, and the similar image search is performed after limiting the search range. And improve the accuracy of the search.

[0004]

However, when trying to manually classify images judged to be similar from the beginning, the trouble increases as the number of images increases. Furthermore, when it is difficult to list all the images on-line, it is extremely difficult to classify the images.

[0005] Further, when trying to classify images by meaning, it is usually necessary to extract a theme that can be classified from all images, and to perform a classification operation for each image based on the theme.

Accordingly, the present invention provides a method and apparatus for efficiently classifying and accumulating images by hand to improve the image search accuracy described above, and to improve the image search accuracy based on the results. It is an object of the present invention to provide an image search method and an image search apparatus which enable the image search to be performed.

[0007]

In order to solve the above-mentioned problems, a method according to the present invention is a method for classifying and re-accumulating already stored images into similar ones, comprising the steps of: Using one image as a key, a basic tag is added to the one image in advance, an image similar to the image is searched, and an image extracted as a result of the search is compared with the image used as the key. The operator makes a determination as to whether or not the image has been determined to be similar, the image is determined to be similar to the attached tag corresponding to the basic tag, after the search and the tag addition operation, the tag is added. If there is an untagged image, one of the untagged images is randomly extracted, a second basic tag is added to the extracted image, and an image with the second basic tag is added. With the key as
Perform the above-mentioned search and attached tag attaching operations, and repeat the above operation until there are no images to which no tags are attached. One or more basic tags and the corresponding attached tags obtained as a result of the series of operations are obtained. Or storing an image to which one or a plurality of basic tags and an attached tag corresponding thereto are added.

An image search method for searching an image similar to the search original image from a plurality of images stored by the above-described image classification and storage method, wherein the search original image is provided with a tag. If there is no image, a normal similar image search is performed, and if the original image of the search exists in the stored image as an image having a basic tag, the image is related to the basic tag of the original image of the search. A normal similar image search is performed only on images having attached tags, and a similar image search is performed on non-tagged images. At the bottom of the result, if the original image of the search is present in the stored image as an image having only the attached tag, the image having the basic tag or the attached tag related to the attached tag of the original image of the search of To perform a normal similar image search, and further perform a normal similar image search for an image without a tag, and attach a lower order to the search result for the image with the tag. Features.

Also, the apparatus according to the present invention is an image classification and storage apparatus for reclassifying already stored images into similar ones and storing them again, wherein the one image is used as a key for the one image. A basic tag is assigned in advance, an image similar to the image is searched for, and the operator determines whether or not the image extracted as a result of the search is similar to the image used as the key. If the image for which the operator has made a similar determination is given an attached tag corresponding to the basic tag, and if there is an image to which no tag has been added after the above-described search and tag addition operation, the tag is attached. One of the non-attached images is taken out at random, a second basic tag is added to the taken out image, and the image with the second basic tag is used as a key, and the search and attached tag described above are used. Perform the above operation Tagging means for repeating the process until there are no more images to which no tags have been added, one or more basic tags and their corresponding attached tags, or one or more basic tags obtained as a result of the series of operations. And an accumulating means for accumulating the image to which the attached tag corresponding thereto is attached.

[0010] An image retrieval apparatus for retrieving an image similar to the retrieval original image from a plurality of images stored in the image classification apparatus, wherein the retrieval original image is not tagged. In the case of an image, a normal similar image search is performed, and the original image of the search is an image having a basic tag,
When present in the stored images, a normal similar image search is performed for only images having an attached tag related to the basic tag of the original image of the search, and further, an image having no tag is added. By performing a normal similar image search on the image, the search result for the image to which the tag is attached is placed below the search result, and the original image of the search exists in the stored image as an image having only the attached tag. In this case, a normal similar image search is performed for only the images having the basic tag or the auxiliary tag related to the auxiliary tag of the original image of the search, and the normal similar image is searched for the image without the tag. It is characterized by comprising image search means for performing a search and attaching a lower order of the search result for the image to which the tag is attached.

According to the present invention, it is possible to efficiently perform the classification work by using the manual classification operation of the image in combination with the conventional similar image search and tagging the similar information manually. At the same time, the system accumulates the results and preferentially searches the images tagged at the time of the search, thereby improving the apparent search accuracy.

That is, when classifying and storing images,
First, one of the images is arbitrarily selected, a conventional similar image search is performed, and among the searched results, those determined to be similar are marked. These marked items register tag information as similar items. Next, one of the images without any tag information is arbitrarily selected, and a similar image search is performed to mark an image determined to be similar. By repeating this work, it is possible to work efficiently in a procedural manner, and it is possible to classify things that a person determines to be similar.

Further, when searching for an image, an image having the same mark as the image to be searched is preferentially searched for, so that the search accuracy is improved so as to obtain a search result closer to human feeling. .

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows a system configuration for explaining an embodiment of the present invention. 1 is a search terminal and 2 is an image search server. The image search server 2 includes an image search unit 2
1, image data 22, a tagging processing unit 23, and a tag information storage file 24. 3 is a network. Although the search terminal 1 and the image search server 2 are arranged on the network 3 in the illustrated example, their functions may be executed on one computer. The tag information storage file 24 includes the tag information table 24 shown in FIG.
There is one.

The procedure for classifying images in this embodiment will be described. Here, it is assumed that images A to G are prepared as the image data 22 of FIG. 1 as shown in FIG.

First, the operator selects an image A from all the images.
Is selected as the original image (key image) and a similar image search is performed.
As a result, it is assumed that images B to F are selected. At this time, if the objects that can be determined to be similar are B, C, and E, the operator marks B, C, and E. The tagging processing unit 23 adds the T1 line to the tag information table 241 of FIG. 3 according to the flow of FIG. 2, and adds tag information (basic tag: M, attached tag) to A as a basic tag and B, C, and E as attached tags. tag:
x) is registered.

Next, an image D without any tag is attached.
An arbitrary image is selected from F and G as an original image (key image). Here, the selection may be made by the operator or randomly by the system. In the illustrated example, similar image search is performed using G as an original image, and tagging is performed by similarly marking similar objects. As a result, when A and B are marked, the tagging processing unit 23 adds T2 to the tag information table and attaches the basic tag (M) to G and the attached tag (x) to A and B.

This operation is repeated, and the processing is continued until there are no more images without tags. In the present embodiment,
The process ends when D and F are selected as original images.

The operator only has to perform the operation of marking the images determined to be similar according to the instructions on the screen. Assuming that the processing amount of this operation is N, the total number of images is Gn, and the number of similar images for one image is Nn, this marking operation ends with Gn / Nn <N <Gn.

Next, searching for an image is divided into the following three cases.

(1) In the case of a new image having no tag attached to the original image.

(2) Tag information is attached to the original image.
When there is no basic tag.

(3) A case where a tag is attached to an original image and a basic tag is attached.

The image retrieval processing procedure in this embodiment is performed as follows according to FIG.

In the case of (1), the image search unit 21 performs a similar image search according to a conventional method.

In the case of (2), the image search section 21 extracts images with tags related to the original image, preferentially performs similar image search, ranks and displays the images.

When the image B is selected as the original image, the basic tags are T1, T2, T3, T4 of the tag information table 241.
None of them. Tags related to image B are:
Tags attached to T1 and T2, and related images A and
First rank C, E, and G for similarity, then A, B,
The similarity of the images other than C, E, and G is ranked after the one ranked first.

As a search result, a required number of results are displayed.

In the case of (3), the image search unit 21 extracts the image having the basic tag of the original image and performs a similar image search with priority, and then extracts the image with the other tags and extracts the similar image. Search, rank and display.

When the image A is selected as the original image, since the basic tag is attached to T1 in the tag information table, the images B, C, and E with the attached tag of T1 are ranked for similarity. Are displayed as a result, and only the required number of images are displayed.

Hereinafter, the effect of performing the tagging process according to the above embodiment will be described.

As a case where a person judges that images are similar, there is a case where attention is paid to the meaning of an object of interest such as a flag, a face, or a red car. In a conventional image search, there is no example of performing an image search focusing on the meaning of an image. Therefore, an object lower in the order of similarity may feel more similar to an object lower in the order of similarity.

As an extreme example, in a conventional image search, when an image is searched from a car in front of a white wall, a search is performed for a flag of the Japanese flag, a red flower on a white table, a red apple on a white table, and the like. Because these are ranked in a jumbled order and the result is returned, in some cases the result may be that the apples are most similar.

In the present invention, a tag is attached to distinguish such a thing. Here, the reason why the tags are divided into the basic tag M and the attached tag x is that the image selected for the original image (key image) focuses on the most characteristic part of the original image, The original image has a basic tag, and those that resemble the image are attached tags. If here
If you select an image and the image has only attached tags, it is said that it is similar from some feature, so all images with tags related to that image will be ranked higher as a result I do. In other words, if a result with a tag is positioned at a higher position as a result, it is possible to make it appear that the search accuracy is improved.

That is, assuming that the result of conventional similar image search (priority ranking based on similarity) without considering tags is ten, the result is output in the order of similarity as follows. In contrast, the result when considering the tag attached according to the present invention is: Is output with priority. That is,
The priority according to the present invention is to prioritize priorities by image search and to perform grouping and prioritize the groups (tagged ones) first. In other words, when sorting by similarity, tag information is also sorted.

In the above embodiment, the basic tag and the attached tag are stored in the tag information table 241 in the image search server 2. However, the basic tag and the attached tag are stored in the image of the image data 22 in the image search server 2. The basic tag and the attached tag may be added and stored again, or they may be stored in an image search server different from the image search server 2.

[0038]

As apparent from the above description, the present invention has the following effects.

(1) Since similar images can be confirmed and classified in a procedural manner, work efficiency is improved.

(2) Since it is not necessary to prepare keywords for classification in advance, it is not necessary to examine which keyword is classified.

(3) Marking work in image confirmation classification can be interrupted or resumed, so that the burden of marking work can be reduced.

(4) Since the search result is displayed with the priority of the image marked with meaning increased, the search accuracy can be improved.

[Brief description of the drawings]

FIG. 1 is an apparatus configuration diagram according to an embodiment of the present invention.

FIG. 2 is a flowchart of tag information registration in an embodiment of the image classification method according to the present invention.

FIG. 3 is a diagram showing an example of an image and a tag information table in the embodiment.

FIG. 4 is a flowchart illustrating an embodiment of an image search method according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Search terminal 2 ... Image search server 21 ... Image search part 22 ... Image data 23 ... Tagging processing part 24 ... Tag information storage file 241 ... Tag information table 3 ... Network

Claims (4)

[Claims] 1. An image classification and storage method for classifying already stored images into similar ones and storing them again, wherein one image is used as a key and a basic tag is added to the one image in advance. An image similar to the image is searched, and for the image extracted as a result of the search, the operator determines whether or not the image is similar to the key image,
The image for which the operator has determined the similarity is provided with an attached tag corresponding to the basic tag. After the search and the tag providing operation, if there is an image to which no tag is provided, the tag is provided. One of the images that have not been extracted is randomly extracted, a second basic tag is assigned to the extracted image, and the image with the second basic tag is used as a key, and the search and the attached tag are added. The above operation is repeated until there is no image to which no tag is added. One or more basic tags obtained as a result of the series of operations and the corresponding attached tags, or one or more An image classification and storage method, comprising storing an image to which a basic tag and an attached tag corresponding thereto are added. 2. Searching a plurality of images stored by the method according to claim 1 for an image similar to a search original image.
In the image search method, when the original image of the search is an image to which no tag is added, a normal similar image search is performed, and the original image of the search is stored as an image having a basic tag. If the image is present in the image, a normal similar image search is performed for only images having an attached tag related to the basic tag of the original image of the search, and a normal image search is performed on an image without a tag. When a similar image search is performed, the image is attached below the search result for the image to which the tag is attached. If the original image of the search exists in the stored image as an image having only the attached tag, A normal similar image search is performed only for images that have a basic tag or an auxiliary tag related to the attached tag of the original image of the search, and a normal similar image search is performed for the untagged image. ,Previous Tag attached to the image that is subordinate to the target search result with the image search method characterized by. 3. An image classification and storage device for reclassifying already stored images into similar ones and storing them again, wherein one image is used as a key and a basic tag is added to the one image in advance. Then, an image similar to the image is searched, and the operator determines whether or not the image extracted as a result of the search is similar to the image used as the key. The attached image is given an attached tag corresponding to the basic tag, and after the above-described search and tag attaching operation,
If there is an image to which no tag has been added, one of the images to which no tag has been added is randomly extracted, a second basic tag is added to the extracted image, and a second basic tag is added. Tagging processing means for performing the above-described search and attached tag assigning operations by using the image given with a key as a key, and repeating the above operations until there are no images to which no tags are assigned, and a result of the series of operations obtained. Storage means for storing one or more basic tags and their corresponding attached tags, or an image to which one or more basic tags and their associated attached tags are added Classification storage device. 4. Searching an image similar to a search original image from a plurality of images stored by the apparatus according to claim 3.
If the original image of the search is an image without a tag, a normal similar image search is performed, and the original image of the search is stored as an image having a basic tag. If the image is present in the image, a normal similar image search is performed for only the image having an attached tag related to the basic tag of the original image of the search, and the normal image search is performed on the image without the tag. Perform a similar image search and attach it to the lower order of the search result for the image with the tag, and if the original image of the search exists in the stored image as an image having only the attached tag, A normal similar image search is performed for only images that have a basic tag or an auxiliary tag related to the attached tag of the original image of the search, and a normal similar image search is performed for an image without a tag. And said An image search device, comprising: an image search unit that attaches a search result below a search result for an image with a tag.