JP2000276484A - Device and method for image retrieval and image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently retrieve and display a target image by retrieving images similar to the object image to be retrieved from a database having a directory structure. SOLUTION: A personal computer 10 and a film scanner 30 are connected to a server 20 with large storage capacity through a network N and the image database is structured on the server 20. To retrieve a specific image from the database, multiple images, e.g. images in the image database are statistically classified into the determined number of clusters and images representing the respective clusters are displayed on the screen on the personal computer 10. The relative distances in feature quantity between an image selected out of them and other images are found to calculate similarities. The similarities are set larger and larger as the distances are shorter and shorter, and an image which is registered more recently is retrieved with higher priority. Thumbnail images are displayed in the decreasing order of the similarities. Consequently, the user can obtain a desirable image faster.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for efficiently searching for an image contained in an image database.

[0002]

2. Description of the Related Art Images obtained by using a digital still camera or a scanner are stored in an image database such as an electronic album. By the way, in the case of a character database, a search can be performed using a keyword, so that even a huge amount of data can be searched in a relatively short time. On the other hand, in the case of an image database, since the search target is an image, there is a search problem that a match search cannot be performed like a character.

[0003]

On the other hand, for example, Japanese Patent Laid-Open No. Hei 10-275161 discloses a technique for efficiently searching for an image using a keyword stored in association with the image. However, according to the related art, when an image is registered in an image database, it is necessary to register a corresponding keyword. Therefore, for example, when an enormous amount of images already exist, it is necessary for the user to classify the images and register appropriate keywords for each image before registering the images in the image database, which requires a very time-consuming operation. Become.

On the other hand, a technique for searching for an image similar to the search target image is disclosed in, for example, Japanese Patent Application Laid-Open No. 10-260983. According to this conventional technique, an image is divided into blocks, and the similarity is determined using the respective feature amounts as a matrix. However, since the image is divided for each block and the feature amount is found for each block, the amount of calculation becomes enormous and it takes time.

Further, JP-A-10-254901 discloses that
A technique has been disclosed in which a part of an image is designated in advance, and its attribute is detected to determine the similarity of the image. However, there is a problem that a complicated calculation is required to detect the attribute and it takes time and effort. Further, in order to search for an image, it is assumed that attributes are detected in advance in each part of all the images in the image database, so that there is a problem that it takes time and effort. In addition, when the attribute of the search target image is not known, it takes time to find it.

SUMMARY OF THE INVENTION In view of the above-mentioned problems of the prior art, an object of the present invention is to provide an image search device, an image search method, and an image display device capable of displaying a specific image, which efficiently search for images contained in an image database. Is to provide.

More specifically, a first object of the present invention is to provide an image search method and apparatus capable of efficiently and quickly searching for a target image, and to provide a search method and apparatus having high search accuracy. It is in.

Further, a second object of the present invention is to search for an image,
It is an object of the present invention to provide an image search method and apparatus that allows a user to learn search contents in accordance with the user's preference through operations such as browsing and perform more accurate searches.

Further, a third object of the present invention is to provide a technique capable of facilitating image retrieval on a network such as the Internet, and facilitating uploading of image data extracted by the image retrieval. Is to do.

Further, a fourth object of the present invention is to make it possible to quickly search the image database even if the image to be searched is ambiguous and to easily use the classification information of the image database. It is an object of the present invention to provide a technique for making it easy to input comments and keywords regarding images.

A fifth object of the present invention is to provide a technique for quickly browsing an outline of an image in an image database and quickly searching for an ambiguous image from the image database.

[0012]

In order to achieve the above object, an image search apparatus according to the present invention uses an image database having a directory structure composed of one or a plurality of folders to search for an image similar to an image to be searched. , And display means for displaying the searched image.

The image search apparatus of the present invention has a search means for searching for an image from an image database and a display means for displaying the searched image, wherein the search means are the same or close on a time axis. The image in the folder is searched for an image having a high degree of similarity at the end of the image, and the display unit displays a composite image obtained by overlapping the end of the image and joining a plurality of images. And

The image search apparatus of the present invention has a search unit for searching an image database for an image similar to the search target image, and a display unit for displaying the searched image. A feature is that a similar image is searched by using the frequency at which is selected as one of the feature amounts.

The image retrieval method according to the present invention includes a step of storing an image selection frequency in an image database, and a step of retrieving an image similar to the retrieval target image by using the frequency at which the image is selected as one of the feature amounts. It is characterized by having.

An image search apparatus according to the present invention records a search means for searching an image similar to a search target image from an image database, a display means for displaying the searched image, and a history of the displayed image. And display control means for restricting the display of the image once displayed.

According to the image search method of the present invention, a step of searching for an image similar to an image to be searched in an image database, a step of displaying the searched image as a candidate image, and recording a history of the displayed image. Removing the once displayed image from the candidate images.

An image retrieval apparatus according to the present invention includes a means for classifying images in an image database into categories, a retrieval means for retrieving an image similar to a retrieval target image based on category information from an image database, Display means for displaying the searched image, wherein the display means displays an image representative of each category before displaying the image searched by the search means.

The image retrieval method according to the present invention is characterized in that it comprises a step of classifying the images in the image database for each category, and a step of displaying images representing the categories.

The image search apparatus of the present invention has a search means for searching an image database for an image similar to the search target image, and a display means for displaying the searched image. It is possible to display an image representing
The search means searches for an image containing a color similar to the color corresponding to the image displayed by the display means.

According to the image search method of the present invention, a step of displaying an image representing a color, a step of searching an image database for an image containing a color similar to the color corresponding to the displayed image, Displaying the created image.

The image search apparatus of the present invention has a search means for searching an image database for an image similar to the search target image, and a display means for displaying the searched image. An image not included in the image database is acquired and displayed, and the search means searches for an image similar to the image displayed by the display means.

The image retrieval method of the present invention comprises the steps of acquiring and displaying an image, searching an image database for an image similar to the acquired image, and displaying the retrieved image. It is characterized by having.

[0024] The image retrieval apparatus of the present invention is characterized by having retrieval means for retrieving an image similar to the retrieval target image from the image database and retouching means for retouching the displayed images collectively. .

[0025] The image search method of the present invention is characterized by including a step of searching for an image in an image database and a step of collectively retouching the searched images.

[0026] The image retrieval apparatus of the present invention comprises: display means for displaying an image; storage means for storing a specified position when a position is specified on the image displayed on the display means; Search means for searching for an image in an image database based on the position stored in the storage means.

[0027] The image search apparatus of the present invention is characterized by comprising storage means for storing information about a hard copy for an image in an image database, and search means for searching for an image based on the information about the hard copy. I do.

An image retrieval apparatus according to the present invention comprises an input unit capable of inputting a character keyword, and a setting for setting a feature amount of an image and a network address related to the keyword based on the character keyword input by the input unit. Means.

An image retrieval apparatus according to the present invention relates to an extracting means for extracting a characteristic amount of a specific image, a determining means for determining a character keyword including the characteristic amount extracted by the extracting means, and a character string associated with the determined character keyword. It is characterized by having means for acquiring an image from a network address, and determining means for extracting a feature amount of the acquired image and determining whether or not the feature of the specific image is included.

An image retrieval apparatus according to the present invention is an image retrieval apparatus for preferentially retrieving images in the same category when retrieving images in an image database, wherein a category common to one or a plurality of images is retrieved. It is characterized by having additional means for adding.

[0031] The image retrieval apparatus of the present invention is characterized in that it has an accompanying means for attaching sound information to an image in the image database, and a search means for searching for an image based on the sound information. The image display device of the present invention includes: a selection unit for selecting an image in an image database; an ordering unit for ordering the images in the image database, with the selected image being first, and a display unit for displaying the images according to the ordering. It is characterized by having.

[0032]

According to the image search apparatus of the present invention, search means for searching for an image similar to the search target image from an image database having a directory structure composed of one or a plurality of folders, and the searched image Is displayed, and by using such a directory structure, images can be searched efficiently. According to the image search device of the present invention, the image search apparatus includes a search unit for searching for an image from an image database, and a display unit for displaying the searched image. In the images in the above, search for an image having a high degree of similarity at the end of the image, and the display means displays a composite image obtained by overlapping the ends of the image and joining a plurality of images. When it is desired to synthesize a panoramic image using an image in the database, the search unit automatically searches for an connected image, so that the image can be efficiently synthesized.

According to the image search device of the present invention, the image search apparatus includes a search unit for searching for an image similar to the search target image from the image database, and a display unit for displaying the searched image. , The frequency at which the image was selected
First, since similar images are searched, for example, among the images in the image database, images that are frequently used are considered to have common features, so if the frequency is used, images can be searched more efficiently. it can.

According to the image search method of the present invention, the step of storing the selection frequency of the image in the image database, and searching for an image similar to the search target image, using the frequency of selecting the image as one of the feature amounts. For example, among the images in the image database, frequently used images are considered to have common features. Therefore, if the frequency is used, images can be searched more efficiently.

According to the image search apparatus of the present invention, search means for searching an image similar to the search target image from the image database, display means for displaying the searched image, and history of the displayed image are stored. Display control means for restricting the display of the image once recorded and displayed once, so that the image displayed once by the display means is regarded as an image which is not considered by the user, so that more efficient Image search is possible.

According to the image search method of the present invention, a step of searching for an image similar to the search target image in the image database, a step of displaying the searched image as a candidate image, and a step of storing the history of the displayed image. Record and 1
Removing the once displayed image from the candidate images, so that the once displayed image is regarded as an image which is not considered by the user, thereby enabling more efficient image search. According to the image search device of the present invention,
Means for classifying images in the image database into categories, search means for searching for images similar to the search target image based on category information from the image database, and display means for displaying the searched images. , The display means,
Before displaying the images searched by the search means, the images representative of the respective categories are displayed. Therefore, by viewing the displayed images, the user can immediately recognize the category including the images. , Thereby enabling efficient image retrieval.

According to the image search method of the present invention, a step of classifying the images in the image database into categories,
Displaying the image representative of the category, by viewing the displayed image, the user can immediately recognize what category the image is included in, thereby enabling an efficient image search. ing.

According to the image search device of the present invention, the image search apparatus includes a search unit for searching an image similar to the search target image from the image database, and a display unit for displaying the searched image. , An image representing a color can be displayed, and the search means searches for an image containing a color similar to the color corresponding to the image displayed by the display means. Image retrieval.

According to the image search method of the present invention, a step of displaying an image representing a color, and a step of searching an image database for an image containing a color similar to the color corresponding to the displayed image, And displaying the searched image, so that efficient image search can be performed by using colors.

According to the image search apparatus of the present invention, the image search apparatus includes a search unit for searching for an image similar to the search target image from the image database, and a display unit for displaying the searched image. Acquiring and displaying an image that is not included in the image database, and the search unit searches for an image similar to the image displayed by the display unit. Even if it cannot be pointed out, if a similar image exists outside the image database, more efficient image search can be performed by using this.

According to the image search method of the present invention, a step of obtaining and displaying an image, a step of searching an image database for an image similar to the obtained image,
Displaying the searched image, so that, for example, even when the user cannot point out the features of the image to be searched, if there is a similar image outside the image database, it is possible to use the image more efficiently. Image retrieval can be performed.

According to the image search apparatus of the present invention, the image retrieving means includes the retrieving means for retrieving an image similar to the retrieval target image from the image database and the retouching means for retouching the displayed images collectively. Can be retouched at once. According to the image search method of the present invention, since there is a step of searching for an image in the image database and a step of collectively retouching the searched images, similar images that need to be retouched can be collectively retouched.

According to the image retrieval apparatus of the present invention, display means for displaying an image, and storage means for storing the designated position when the position is designated on the image displayed on the display means And a search unit for searching for an image in the image database based on the position stored in the storage unit, so that a more efficient search can be performed using a specified position such as a face of a person image. It becomes possible.

According to the image retrieval apparatus of the present invention, there are provided a storage unit for storing information about a hard copy for an image in an image database, and a retrieval unit for retrieving an image based on the information about the hard copy. For example, images can be divided according to the number of hard copy times, and more efficient search can be performed.

According to the image retrieval apparatus of the present invention, based on the input means capable of inputting a character keyword, and based on the character keyword input by the input means, the characteristic amount of the image and the network address related to the keyword are set. Since the setting means is provided, more efficient image search can be performed using the character keyword.

According to the image retrieval apparatus of the present invention, extracting means for extracting a characteristic amount of a specific image, determining means for determining a character keyword including the characteristic amount extracted by the extracting means,
Since it has means for acquiring an image from a network address related to the determined character keyword, and determining means for extracting a feature amount of the obtained image and determining whether or not it includes a feature of a specific image, Even if no character keyword is added to the image, the determining means determines the character keyword, thereby enabling more efficient image search.

According to the image search device of the present invention, when searching for images in the image database, the image search device preferentially searches for images in the same category, and is common to one or a plurality of images. Since there is an additional means for adding a category, for example, the user collectively puts favorite images in the same category, and if such a category is added, subsequent searches can be efficiently performed.

According to the image search apparatus of the present invention, since the image search apparatus has the accompanying means for attaching sound information to the image in the image database and the search means for searching for an image based on the sound information, for example, If related voices such as “Jimbuts” and “Fukai” are uttered, more efficient image search can be performed by using them.

According to the image display device of the present invention, a selecting means for selecting an image in the image database, an ordering means for ordering the images in the image database with the selected image at the top, and displaying the images according to the ordering Display means, the images having a high degree of relation to the selected image are sequentially displayed, and the images can be browsed more efficiently.

[0050]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram for explaining the present embodiment. In FIG. 1, a personal computer 10 is connected to a server 20 having a large storage capacity via a network N. On the other hand, for example, a film scanner 30 installed in a lab or the like is also connected to the server 20 via the network N. The server 20 has an image database built therein. Note that the image database does not necessarily need to be built in the server 20, but may be built in the hard disk of the personal computer 10. In such a case, the image data from the film scanner 30 can be input to the personal computer 10 via a storage medium such as a CD-R.

The manner in which images are registered in the image database of the server 20 will be described. In the lab, a negative film obtained by the user shooting with a silver halide camera is converted into image data by the film scanner 30. The image data is transmitted to the server 20 via the network N, and is registered as an image in the image database.

Here, if an image is registered and stored as it is, it takes time to search for a specific image later. Therefore, in the present embodiment, a directory structure is formed to store data relating to images hierarchically.

FIG. 2 is a diagram showing an example of such a directory structure. In FIG. 2, the upper directory contains
For example, a directory with the name “199080101.001” is formed. This is because the name is composed of (4 letters of the Christian era) + (2 letters of the month) + (2 letters of the day) + (3 letters), ie, the first negative read on January 1, 1998. Indicates that it is a film.

Therefore, the registration date of the image can be read from the directory name. Folders related to images in the same negative film (KN001001.JPG
~) Are all stored in this directory.

It should be noted that an image captured by a digital still camera or the like can be stored in such a directory structure in a similar manner. In such a case, since a recording medium such as a memory card corresponds to the negative film, one upper directory is formed for each memory card. Each directory in such an upper directory is hereinafter referred to as a lot.

Further, in this directory, a directory (HTML) for storing an HTML file indicating a list of grouped images is formed. In the present embodiment, the image format is jpeg, FlashP
ix.

FIG. 3 is a diagram showing a modification of the directory structure. In the case of the example shown in FIG. 3, “Album — 01B”, “Album — 04”
B ”and Album — 16B” are formed, which are 1Base and 4Ba, respectively.
This means that images of se and 16Base are stored. The directories below it are the same as those shown in FIG.

Next, a mode for searching for a specific image from the image database constructed in this way will be described. FIG. 4 is a diagram showing a screen display example of the personal computer 10 in FIG. When the user activates the image search software, the display shown in FIG. 4 appears on the screen as an initial screen.

The plurality of images displayed here are images representative of each cluster, for example, by statistically classifying the images in the image database so as to have a predetermined number of clusters. The representative image is, for example, 1 for each lot.
The second image may be used.

The user selects an image which is considered to be similar to the image he / she wants to find (for example, a mountain or a person in a landscape) and a mouse (not shown).
Click on the image using a button. At this time, the selected image will be referred to as a key image in the future.

The personal computer 10 searches the image database for an image similar to the key image. More specifically,
A similarity is calculated by calculating a relative distance between a feature amount corresponding to the selected image and a feature amount of another image. Here, the coordinate axes are set such that the shorter the distance, the higher the similarity.
The image feature data file is recorded on the hard disk of the personal computer 10 or the like. The feature amount includes, in addition to general features such as color and frequency distribution, a time representing a time, for example, a position on a database or a folder name corresponding to a date, and a date and time when an image is registered, or an image. The date and time when the image was captured, and in some cases, the date and time when the image was modified, are also taken into account in the search. In other words, an image closer to the date or time when the selected image was photographed or registered is preferentially searched.

This is because a time factor is often a key when searching in an electronic album. Of the images of the same lot as the key image, the predetermined number of images in the order of the similarity tends to have a larger time factor, especially when the similarity is increased.

The personal computer 10 displays thumbnail images on the display in the order of similarity. In FIG.
When the user clicks the [Lot] button BL on the right side of the image, the personal computer 10 can switch the screen and display all the images of the same lot. In such a case, the similarity may be checked within the same lot, and the images may be rearranged and displayed in descending order of the similarity. As a result, the user can reach an image to be searched for faster.

FIG. 5 shows an example of a screen displaying a similar image when the upper left image in FIG. 4 is selected as a key image. In the example of FIG. 5, the personal computer 10 searches the image database for an image in which two persons have been photographed as main subjects as similar images.

When the image desired by the user is not obtained in the first search, the user can continue the search. For example, in the example shown in FIG. 5, when an image at the lower right is selected by clicking on the image, an image similar to that (for example, an image containing a seascape image) is displayed on the screen in the same manner.

At this time, the personal computer 10 may store the history of displaying the image, and may prevent the image displayed once in the search process from being displayed. Images that the user viewed once but did not select are
The image is regarded as not a search target image, and is prevented from being displayed again, thereby achieving a quick search. The same operation may be repeated until an image to be searched is found. Such a history may be reset by clicking a reset button (not shown).

As other functions, it is conceivable to add the following functions to the personal computer 10. Both are auxiliary functions to make searching easier.

(1) When the [Info] button BI on the right side of the image is clicked, the original image is displayed in a large size instead of the thumbnail, and information about the image such as the shooting date and time and the size are displayed in text. . Such information can be obtained from the information written on the magnetic recording layer of the film, for example, if the image is recorded on a film containing IX240 cartridge (so-called APS film), or written on the header of the image file. It can also be obtained from information.

(2) When the [Key image history display] button BK is clicked, a list of key images selected so far is displayed, and the state at that time can be returned.

(3) By clicking the [Category Display] button BC, it is possible to return to the initial screen.

(4) When the [Lot representative display] button BP is clicked, several images representing the lot are displayed in chronological order as a start screen.

As described above, according to the present embodiment, since the position on the time axis is added to the feature value, it is possible to quickly search for an image to be searched. Further, since the image database has a directory structure corresponding to the time axis, the position on the time axis can be easily obtained.

Further, according to the present embodiment, even if the user cannot specifically specify the image to be searched, for example, there is an image taken at the same time, or three persons Even if the information is ambiguous, such as that the image should have been taken with the mountain in the background, the personal computer 10 can search for a similar image from the image database, so that an efficient search can be performed. There is also an advantage that it is not necessary to add an attribute or the like to an image in advance in performing such a search.

When the image data on the recording medium is copied to the hard disk of the personal computer for each directory structure and used, the access speed increases and the search speed can be increased.

Further, the creation of the image database is not limited to the use of the service of the lab, and the user can use an image search software to create a database having a similar structure using a personal computer at home. It is also possible to create

As a modified example of this embodiment, when the [Category Display] button BC is clicked on the initial screen shown in FIG. 4, a menu bar is displayed so that the representative image displayed on the initial screen can be changed. It is possible.

In such a case, as in the above-described embodiment, a representative image of each cluster statistically classified is displayed as default, and when [Color Image] is selected, a color picker (not shown) is displayed on the initial screen. It may be displayed so that the user can select the dominant color in the image he wants to find.

For example, in general, in a summer mountain scenery, “green”
Color is considered dominant, and in seascapes, the “blue” color is considered dominant. Therefore, by choosing a color,
The image can be narrowed down to some extent. As described above, the search is performed using the feature amount related to the color, and it is determined that the similarity is higher as the dominant ratio of the selected color is higher (that is, as the ratio of the selected color is higher), and the result is displayed on the screen. Will be displayed.

In the above-described modified example, the color is selected by using the color picker. Instead, words representing the contents of the image such as [mountain] and [sea] can be selected. A representative image may be displayed. If a keyword has been added in advance, the corresponding image may be displayed. Alternatively, an image representative of [mountain] and [ocean] may be determined in advance, and an image having a high degree of similarity to the image is displayed. It doesn't matter. PC 1
0 can determine the category to which the image belongs based on the content of the image or, if the keyword is attached to the image, the keyword.

As described above, according to the present modification, the user can select the initial screen that best matches the image he / she wants to search, so that the desired image can be found faster.

As another modification, an image not registered in the database can be taken in as an image displayed on the initial screen shown in FIG. For example, as shown in FIG.
Is clicked, and an image selection dialog box is displayed when this button BT is clicked, and an image outside the database (may be obtained via the Internet) is selected. If there is no thumbnail image of this image, a thumbnail may be automatically created or a part of the image may be trimmed so that the selected image can be displayed.

According to this modification, since the user can select an image outside the database as an option of the start image, the user can select the initial screen that best matches the image he or she wants to search, so that the user can find the image earlier. Can be.

Next, a second embodiment will be described. First, when the image search software is started on the personal computer 10, an initial screen shown in FIG. 7 is displayed. When the user clicks the [Retouch ...] button BR, a menu bar is displayed, and items such as "panorama" and "red-eye correction" are displayed (not shown). Here, when the user selects “panorama”, the personal computer 10 performs the following processing.

The personal computer 10 searches for a combination of images having a high similarity of the feature amount with respect to the edges at the upper, lower, left, and right ends of the image in the same lot (different lots are likely to have different shooting locations). At this time, the similarity between the left end of the target image and the right end of the other images, the similarity between the right end and the left end, the upper end and the lower end, and the lower end and the upper end are obtained, and the image having the similarity higher than a predetermined threshold value is obtained. Is searched as an image constituting the panoramic image.

The personal computer 10 performs pattern matching on the edge having a high degree of similarity, finds an overlapping position, and combines the two images to combine them. Since the pattern matching is already known, its details are omitted. The synthesized image is displayed on the screen.

On the other hand, when the user selects “red-eye correction”, the personal computer 10 performs the following processing. First, the user searches the database for one red-eye image. Next, when the user selects “red-eye correction”, the personal computer 10 determines whether or not an image having a high similarity to the red-eye image further has red-eye. to correct. Further, the personal computer 10 displays an image in which red-eye has been corrected.

As described above, according to the present embodiment, it is possible to automatically search for images that require the selected processing and process them collectively. Note that the batch processing of images is not limited to red-eye, and it is also possible to change, for example, the colors of clothes of the photographed person so as to be uniform.

Next, a third embodiment will be described. In such an embodiment, a part of the image considered to be characteristic to the user is determined according to the operation of the user who clicks the image, and is used for subsequent searches. More specifically, the user first clicks on a portion considered important in the key image, and saves the location together with a file as important information.
The key image can be selected many times and gradually approaches the image desired by the user. Further, in order to extract the feature amount of the image, the complexity and color distribution of the image are used.

First, as the first step, the personal computer 1
0 indicates a key image. Cluster images that are already in the image database and display representative images among them. FIG. 8 shows the relationship between the images plotted on the search coordinates and clustered, in which the vertical axis shows the color of the image and the horizontal axis shows the complexity of the image. The complexity of the image can be evaluated at certain specific spatial frequency components,
The color of an image can be evaluated by hue. Each image is analyzed to determine color and complexity, and plotted on search coordinates, for example, divided into five groups.

By doing so, clustering becomes possible by analyzing the images in the image database. Even when a new image is registered in the database, the same clustering may be followed, and a new attribute input can be performed. You don't have to. Here, the attribute indicates a flower, a person, photographing data, and the like.

Next, as a second step, an image is specified. Specifically, the user is caused to specify a part considered to be most important. In this example, the person's face on the first screen is specified. The X mark displayed on the image of FIG. 9 is the position.

This process can be efficiently learned by being used in combination with the designation of the key image (the image closest to the image desired by the user). Of course, it is also possible to sequentially specify a point of interest in another registration mode without using the same.

Further, as a third step, coordinates are recorded. The xy coordinates for this image are captured and recorded. This result is added to the data of the file or recorded in another file. Note that the personal computer 10 may automatically store information such as the purpose of use, the user, and the use account in association with the image whose position is specified.

When the repetition is specified, a plurality of xy coordinates are generated, but all of them are recorded. As a result, the degree of concentration of the plurality of images indicates the importance in the images. If a plurality of pieces of position data are close to each other, the range of interest is considered to be narrow. Also, it is considered to be wide if dispersed.

As a modification of the third embodiment, when a special place (for example, a corner) in an image is designated, other information can be given instead of giving the recognition of the important position. For example, if a corner is designated, the entire screen is set as the attention area, and if it is within a certain center area, the attention area is determined to be half of the screen size, so that the search can be performed more conveniently.

Further, as a fourth step, the characteristics of the coordinate peripheral image are examined, and the result is displayed. The image around the point of interest is analyzed, and the color and complexity around the designated part are examined. Even if the same subject is photographed with a wide-angle lens and is taken aside, for example, the subject may look different when photographed with a telephoto lens and taken in a large area in the center. Such processing facilitates recognition of the same subject and facilitates retrieval.

FIG. 10 is a diagram showing an example in which searched images are displayed. In each image, the position where the pixel of interest has already been designated is indicated by a star (star). The user may further specify the same location or another location. If the position is the same, it is clear that the point of interest of the image is that position, and if the position is different, it can be determined that the entire image is being viewed.

As a fifth step, the search can be repeated. From among the images displayed in FIG. 10, the user first selects an image he or she thinks, specifies a position, gradually approaches the image desired by the user, and finally reaches the image. It is good to
In such a case, a search map can be used to inform the user of where the user is currently searching.

FIG. 11 is a diagram showing a search map. For example, the first key image is G1 and the next key image is G2.
If the next key image is G3, the next key image is G4, and the images are connected by an arrow, the arrow may surround a certain image GT on the search map. In such a case, it can be seen that the image desired by the user is likely to be within the area surrounded by the arrow. On the other hand, if the key images are duplicated even though all the images have not been searched, a warning can be displayed assuming that the search has entered a closed loop.

As a modification of the present embodiment, it is conceivable that the number of hard copies of an image is recorded. Since image search is often performed to obtain a hard copy in the end, for example, an image with a large number of hard copies is considered to be an important image for the user, and if it is recorded, it is more efficient. The possibility that search can be performed is increased. Further, the hard copy can be reduced in priority as a search target, and by changing the display order, the user's convenience can be improved at the time of search.

Further, by using the information on the time of creation or modification of an image to display an image that is always close, the user can easily recall the memory, and the search can proceed efficiently. For this purpose, it is conceivable that an image is formed in a directory structure for one photographing (one film or one memory in a digital camera).

As described above, according to the present embodiment, since a search is performed based on the content specified in the position in the image, it is possible to perform a search with higher accuracy gradually during use.
In addition, by allowing the user to specify a portion considered to be important in the image, the point of interest in the image becomes clear, and a highly accurate search can be performed.

Further, since the position is specified together with the key image for the image search, no special labor is required, so that the search can be simplified. By displaying the information at the same time in an image for which a point of interest has already been specified, the user can specify a second point of interest when there are a plurality of points of interest.
Further, the user can grasp whether or not the user has already paid attention while browsing / searching the screen.

According to this embodiment, a plurality of persons,
Alternatively, when performing a search for a plurality of purposes, an image search suitable for each can be performed. Further, by displaying the coordinates of the key image used for the search, the user can visually grasp how the search is being performed. Further, by leaving a record of the hard copy, the search for the hard copy is facilitated, and only the necessary image can be hard copied. In addition, image search accuracy can be improved.

Next, a fourth embodiment will be described. FIG. 12 is a diagram showing a display screen of the personal computer 10 and shows an example of searching for an image in the “image database” folder of the server 20 using a search keyword. As shown in FIG. 13, the personal computer 10 stores the search keyword in association with the feature amount and the related URL. In the example shown in FIG. 13, for the keyword "sky", the feature of the color of L ^* , a ^* , and b ^* and the feature of the place where the color exists are given. UR
L is used to search for a home page on the Internet related to “sky”. In the example of FIG. 13, L ^* = 7
5, a ^* =-7, b ^* = 35, upper 30% in the vertical direction of the image
-A 100% area is represented in the horizontal direction.

When the search button is pressed by the user, the personal computer 10 analyzes the image data in the "image database" folder and determines whether or not there is an image area satisfying the feature amount shown in FIG. Images satisfying the conditions are sequentially displayed on the screen. The URL related to “sky” is displayed at the bottom. When the user selects this display, the personal computer 10
Connects to the URL, downloads an HTML file, image data, or the like, and displays the image in another window (not shown) or at the bottom.

When the user selects a URL button on the browsing screen, the personal computer 10 sequentially connects to the URLs in Table 1 and downloads image data. Further, the downloaded image is searched for an image that satisfies the condition of the feature amount and displayed on the screen. In the lower part, the URL registered in FIG. 13 is also displayed.

At the time of downloading, the personal computer 10 creates a folder with the URL name, and stores HTML in the folder.
Save files, image data, etc. Then, a search is performed on the images in this folder.

FIG. 14 is a diagram showing an example of an image search using an image file. First, the personal computer 10 reads the initial image data from the server 20 and displays it on the screen. Here, when the user presses the search button, the feature amount of the image is extracted and collated with the feature amount of the registered keyword.
The image data is downloaded from the URL of the keyword including the feature amount, searched, and displayed on the screen.

When a search folder is selected, the personal computer 10 extracts feature amounts from images in the folder,
An image including the feature amount of the initial image is displayed on the screen. It is also possible to select a part of the initial image and perform a search using the feature amounts (colors, coordinates) of the selected part.

It is also possible to prepare some templates like the example shown in FIG. 13 in advance so that the user can edit them. Also, instead of the URL in FIG.
It is also possible to set a network address such as an IP address. Furthermore, the feature value is
It is also possible to have a width before and after.

As described above, according to the present embodiment, image data can be downloaded from the Internet and used in response to a user's request, so that a broader search can be performed.

Next, a fifth embodiment will be described. During the search process, images similar to the key image will be sequentially displayed on the personal computer 10. For example, when there are many similar images, these are sequentially searched as similar images, and a narrow screen is displayed. Is displayed, the target image is not easily reached, and it is expected that the search will take time. In the fifth embodiment, such a problem is mitigated as follows.

In this embodiment, as in the above-described embodiment, a plurality of images similar to the key image are sequentially selected from the image database and displayed.
By selecting one image, a plurality of images to be displayed next are determined. This is repeated until all images are displayed.

The personal computer 10 determines the display order of the images as follows. The similarity of an image in the database that has not been displayed yet to the selected image is determined, and when arranged in descending order of similarity, images similar to each other are not displayed. Images are thinned out and displayed. That is, for example, only one image of almost the same image is displayed, such as an image continuously shot at the same place, thereby enabling high-speed browsing. For the calculation of the similarity, the color, frequency characteristic, structure, photographing time, and the like of the image content can be used as in the above-described embodiment.

Further, the personal computer 10 can categorize the images in the database based on, for example, a directory name or the like, and display the images collectively for each category. When images are sequentially displayed and an image similar to the image to be searched is displayed, it is possible to switch to the image search mode based on the image. At that time, by performing a search including the previously displayed image, the overlooked image is also searched.

The personal computer 10 can switch the display screen as follows. For example, a plurality of images can be displayed, and each time the image is clicked, a plurality of images based on the image can be displayed. Further, images are switched and displayed one after another at an arbitrary time (see FIG. 15A), all are switched at one time, or images are displayed so as to flow (FIG. 15).
(See (b)). Such a display is Ja
It can also be realized by va.

In this case, the display interval can be arbitrarily adjusted by the user, the display can be paused, or the frame can be displayed frame-by-frame with the previous or subsequent image. As shown in FIG. 15, the display number of the image can be additionally displayed at the lower right of the image so that the user can know how far the display has progressed.
Such a number may be stored as a history in association with the image. In addition, a scroll bar or the like may be provided below the image to display the currently viewed image of the whole image. Further, it is also possible to superimpose a part of the image and shift the image horizontally or vertically for display. In addition, the image displayed last may be displayed as “LAST”, for example, so that the user can recognize that it is the last image.

According to the present embodiment, images in the image database can be quickly browsed, and a display that can be easily browsed by the user can be provided, which is more user-friendly.

By adding a keyword or the like to an image in the image database, the image can be searched more efficiently. However, the input of a keyword or the like requires considerable input when there are many images, and the contents of the keyword itself may not be known to the photographer himself. Furthermore, even if a random keyword is attached, it may not be used in a later search. Also, typing using the keyboard is usually a difficult task for those unfamiliar with the keyboard,
There is also the problem that it takes time. Therefore, in a sixth embodiment described below, such a problem is mitigated as follows.

FIG. 16 is a diagram for explaining the sixth embodiment. As shown in FIG.
Selects and displays a plurality of images from an image database. The image is displayed one by one from images clustered by the feature amount of the image information, for example, an image in which the shooting time is shifted every predetermined period or every other week, or a random number or the like is used. It may be possible to display a simple image.

Next, the user puts a check mark on a favorite image in the display image. More specifically, among the displayed images, images that match the theme (for example, any theme such as “new year card candidate” or “snap during travel”) considered by the user, so-called “favorite” The check box CB on the right side of the image is clicked with a mouse or the like to place a check mark. FIG.
In the example, a check mark is attached to the lower center image and the lower right image. The personal computer 10 registers the image in the selected category. Further, the images may be displayed in the display order used in the high-speed browsing.

Further, the user clicks an image close to the theme from the displayed images, and clicks on the personal computer 10.
In addition, it is possible to display a plurality of images having a high degree of similarity to the image, images close in time, and the like, or to display a new image at random. By repeating such steps, it is possible to check favorite images and images that match the theme.

In order to confirm the checked image,
The personal computer 10 can display a list of images, delete an image, and additionally register an image in response to a user request. The user collectively registers the images classified into this category using comments, keywords, audio information, and the like. Also, for example, a voice that the user has blown in, such as “New Year card”, may be registered for each image.

The audio information can be inputted by a microphone (not shown), converted into characters in the personal computer 10 and recorded together with the image in the image database, or the audio itself can be recorded together with the image. Is also good.
In such a case, the similarity of the images can be determined by analyzing the frequency of the recorded audio and determining whether or not the frequency characteristics are similar.

At the time of image search, the category newly created by the user as described above can be used as one category for image search. For example, in the case of a search function in which images in the same category are used as one index of similarity, images in the selected category are also used as one index of similarity.

[0128] Images registered across a plurality of selected categories are considered important by the user and may have a high utility value. Therefore, the priority of the search based on the category information should be increased. It is good to When a face of a specific person is registered as a selection category, an image including the same or similar face is registered in the category, and a category (that is, an album) for each person is completed.

As described above, according to the present embodiment, the user can add a new category to the image in the image database in addition to the category automatically determined by the personal computer 10, so that each user can search more. It is easy to do.

Although the present invention has been described with reference to the embodiments, it should be understood that the present invention should not be construed as being limited to the above embodiments, and that modifications and improvements can be made as appropriate. is there. For example, image search software for executing the image search method according to the above-described embodiment can be recorded on a recording medium such as an FD or a CD and installed on the personal computer 10.

[0131]

According to the present invention, images contained in an image database can be searched efficiently.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the present embodiment.

FIG. 2 is a diagram illustrating an example of a directory structure according to the embodiment;

FIG. 3 is a diagram showing a modification of the directory structure.

FIG. 4 is a diagram showing a screen display example of the personal computer 10 of FIG.

FIG. 5 is a diagram showing an example of a screen displaying a similar image when an upper left image in FIG. 4 is selected as a key image.

FIG. 6 is a diagram illustrating an example of a display screen provided with an image capture button.

FIG. 7 is a diagram showing an example of a display screen provided with a retouch button.

FIG. 8 is a diagram illustrating an example of clustering by search coordinates.

FIG. 9 is a diagram illustrating a display screen as an example in which a position in an image is designated.

FIG. 10 is a diagram illustrating an example in which searched images are displayed.

FIG. 11 is a diagram showing a search map.

FIG. 12 is a diagram showing a display screen of the personal computer 10.

FIG. 13 is a diagram showing a search keyword, an image feature amount, and a related URL and association.

FIG. 14 is a diagram illustrating an example of an image search using an image file.

FIG. 15 is a diagram showing an example in which an image is displayed on a display screen.

FIG. 16 is a diagram for explaining a sixth embodiment.

[Explanation of symbols]

 10 PC 20 SERVER 30 FILM SCANNER

 ──────────────────────────────────────────────────の Continued on the front page (72) Inventor Shoji Madoka 2970, Ishikawacho, Hachioji-shi, Tokyo Inside Konica Corporation (72) Inventor Shinichiro Kito 2970, Ishikawacho, Hachioji-shi, Tokyo Inside Konica Corporation (72) Inventor Yusuke Dobu 2970 Ishikawa-cho, Hachioji-shi, Tokyo F-term in Konica Corporation (reference) 5B050 CA08 EA12 EA19 FA02 FA05 FA10 FA12 GA08 5B075 ND06 NK02 NK06 NK25 NR12 PP13 PP30 PQ02 PQ46 PQ48 PQ74 PR04 PR06 Q08

Claims (69)

[Claims] An image database having a directory structure including one or a plurality of folders has a search unit for searching for an image similar to the search target image, and a display unit for displaying the searched image. An image search device, characterized in that: 2. The image processing apparatus according to claim 1, wherein the search unit searches for an image.
2. The image search apparatus according to claim 1, further comprising means for obtaining a relative position on the time axis from the search target image, and searching for a similar image using the relative position as one of the feature amounts. 3. A relative position on a time axis from an image to be searched is obtained from at least one of a folder name, a time when an image is registered, a time when an image is shot, and a time when an image is corrected. Item 3. The image search device according to Item 2. 4. The apparatus according to claim 2, wherein the relative position on the time axis from the search target image is obtained from data stored in one or a plurality of recording media via a network.
Or the image search device according to 3. 5. The search means, when searching for an image,
5. The image search apparatus according to claim 2, wherein all or a part of the images included in the same or close folders on the time axis are set as similar image candidates. 6. The image processing apparatus according to claim 2, wherein the display unit displays an image included in a folder that is the same as or similar on the time axis to an image that is a candidate for a similar image. Image retrieval device. 7. A means for extracting a plurality of representative images from the images contained in the folder, wherein a similarity to the representative image is obtained as a first step of the search, and a second step is performed if the similarity is high. 7. The image search apparatus according to claim 1, wherein a similarity is obtained and searched for all images in the folder. 8. An image database comprising: a search unit for searching for an image from an image database; and a display unit for displaying the searched image. An image search apparatus that searches for an image having a high degree of similarity at an end of an image, and wherein the display unit displays a composite image obtained by overlapping the ends of the image and joining a plurality of images. 9. A search unit for searching an image similar to the search target image from the image database, and a display unit for displaying the searched image, wherein the search unit is characterized by a frequency of selecting the image. An image search device for searching for a similar image as one of the quantities. 10. A method for storing an image selection frequency in an image database, and a step of searching for an image similar to the search target image using the frequency of selecting an image as one of the feature amounts. How to search for images. 11. A search means for searching an image similar to the search target image from the image database, a display means for displaying the searched image, and a history of the displayed image is recorded and displayed once. And a display control means for restricting display of the displayed image. 12. The image search device according to claim 11, wherein the display control means has a function of resetting the history. 13. A step of searching for an image similar to the search target image in the image database; a step of displaying the searched image as a candidate image; and a step of recording a history of the displayed image and displaying the image once. Excluding the selected image from the candidate images. 14. The method according to claim 13, further comprising the step of resetting the history. 15. A means for classifying images in an image database into categories, a search means for searching for an image similar to the search target image based on category information from the image database, and displaying the searched images. An image search device, comprising: display means, wherein the display means displays images representative of respective categories before displaying the images searched by the search means. 16. The method according to claim 15, wherein the display unit classifies the statistically classified cluster as a category.
An image search device according to claim 1. 17. The apparatus according to claim 15, wherein the display unit classifies the images into categories based on a keyword added to the images. 18. The apparatus according to claim 15, wherein said display unit classifies the images into categories based on the contents of the images. 19. An image search method comprising: classifying images in an image database for each category; and displaying images representing the categories. 20. The method according to claim 19, further comprising the step of categorizing the statistically classified clusters into categories.
Image search method described in. 21. The image search method according to claim 19, further comprising the step of classifying the images into categories based on the keywords added to the images. 22. The image search method according to claim 19, wherein the images are classified into categories based on the contents of the images. 23. A search unit for searching an image similar to the search target image from the image database, and a display unit for displaying the searched image, wherein the display unit can display an image representing a color. Wherein the search means searches for an image containing a color similar to the color corresponding to the image displayed by the display means. 24. The display device according to claim 23, wherein the display unit displays a color picker as an image representing a color.
An image search device according to claim 1. 25. A step of displaying an image representing a color, a step of searching an image database for an image containing a color similar to the color corresponding to the displayed image, and displaying the searched image. And an image search method. 26. The method according to claim 25, further comprising the step of displaying a color picker as an image representing the color.
Image search method described in. 27. Search means for searching an image database for an image similar to the search target image, and display means for displaying the searched image, wherein the display means is included in the image database. An image retrieval apparatus characterized by acquiring and displaying a missing image, and wherein the retrieval means retrieves an image similar to the image displayed by the display means. 28. The apparatus according to claim 27, wherein said display means displays the acquired image by reducing or trimming the acquired image. 29. A method comprising: obtaining and displaying an image; searching an image database for an image similar to the obtained image; and displaying the searched image. Image search method. 30. The method according to claim 29, further comprising a step of reducing or trimming the input image and displaying the image. 31. An image search apparatus comprising: a search unit for searching an image database for an image similar to a search target image; and a retouching unit for collectively retouching displayed images. 32. An image search method comprising: searching an image in an image database; and retouching the searched images collectively. 33. A display unit for displaying an image, a storage unit for storing a specified position when a position is specified on the image displayed on the display unit, and a storage unit for storing the specified position. A search unit for searching for an image in the image database based on the determined position. 34. The designated position is a region of interest for an observer observing an image displayed on the display means, and further includes a coordinate determining means for obtaining and storing coordinates of the designated position. The image retrieval device according to claim 33, wherein: 35. The apparatus according to claim 33, wherein the specified position is indicated on the image by the display unit. 36. Corresponding to the image whose position is designated,
The image search device according to any one of claims 33 to 35, wherein at least one of a use purpose, a user, and a use account is stored. 37. The image search apparatus according to claim 33, wherein an image specified in the past corresponds to a position in the coordinates of the search space. 38. An image search apparatus comprising: a storage unit for storing information about a hard copy for an image in an image database; and a search unit for searching for an image based on the information about the hard copy. 39. A system according to claim 3, further comprising display means for preferentially displaying an image that has not been hard copied.
9. The image search device according to 8. 40. The image retrieval apparatus according to claim 33, wherein the images in the image database are grouped in the order in which the image creation or modification time is short. 41. An apparatus comprising: input means capable of inputting a character keyword; and setting means for setting a feature amount of an image and a network address associated with the keyword based on the character keyword input by the input means. Characteristic image retrieval device. 42. Extracting means for extracting a characteristic amount of a specific image, determining means for determining a character keyword including the characteristic amount extracted by the extracting means, and acquiring an image from a network address associated with the determined character keyword An image search apparatus comprising: means for extracting a feature amount of an acquired image and determining whether or not a feature of a specific image is included. 43. The image according to claim 4, wherein the feature amount of the image includes information on a color and an area in the image.
43. The image search device according to 1 or 42. 44. An image retrieval apparatus for preferentially searching for images in the same category when retrieving images in an image database, comprising an additional unit for adding a category common to one or more images. An image search device, characterized in that: 45. The apparatus according to claim 44, wherein said adding means forms a new category by selecting one or a plurality of images. 46. The image retrieval apparatus according to claim 44, further comprising a classification unit configured to determine the content of the image and to classify the image for each category. 47. The image search device according to claim 44, further comprising display means for displaying images of the same category. 48. The image retrieval apparatus according to claim 44, wherein a new category is formed by selecting images in the same category displayed by said display means. 49. The image search device according to claim 44, further comprising an adding unit that adds a comment to images of the same category. 50. The image search device according to claim 44, wherein when an image common to different categories is included, the search priority of the image is increased. 51. The image search method according to claim 44, wherein when a person is included in the image, images including a face image of a specific person are classified into the same category. apparatus. 52. The image search device according to claim 51, wherein an image including a face image similar to the face image of the specific person is also included in the same category. 53. An image retrieval apparatus, comprising: an attendant unit for attaching sound information to an image in an image database; and a search unit for searching for an image based on the sound information. 54. The image search device according to claim 53, wherein the search means converts the sound information into a character keyword and searches for an image. 55. The image search device according to claim 53, wherein the search unit searches for an image based on the degree of approximation of a frequency characteristic included in the sound information. 56. The image processing apparatus according to claim 53, further comprising a selection unit for selecting an image, wherein the attachment unit attaches common sound information to the image selected by the selection unit. An image search device according to claim 1. 57. A system comprising: a selection means for selecting an image in an image database; an ordering means for ordering the images in the image database with the selected image as a head; and a display means for displaying images according to the ordering. Characteristic image display device. 58. The image display device according to claim 57, wherein the ordering means sets an image having a higher similarity to the selected image in an order closer to the selected image. 59. The image processing apparatus according to claim 57, wherein the ordering means sets an image closer to the selected image to an image having a shorter shooting time in an order closer to the selected image.
An image display device according to claim 1. 60. The image processing apparatus according to claim 57, wherein the ordering unit sets the order of images closer in content to the selected image closer to the selected image.
The image display device according to any one of the above. 61. The image display device according to claim 57, wherein the display unit displays the images ordered by the ordering unit in a predetermined order. 62. The image display device according to claim 57, wherein said display means does not display an image displayed once. 63. The image display device according to claim 57, wherein said display means sequentially displays the ordered images at predetermined time intervals. 64. The image display device according to claim 63, further comprising changing means for changing the predetermined time interval. 65. The image display device according to claim 63, further comprising an interruption unit for interrupting sequential display of images. 66. The image display apparatus according to claim 57, wherein said display means displays information indicating a relationship between a displayed image and an image to be displayed. . 67. The image display device according to claim 57, further comprising a search unit that searches for a similar image based on the image displayed by the display unit. 68. The image display apparatus according to claim 57, further comprising means for notifying that the last ordered image has been displayed. 69. The image display device according to claim 57, further comprising storage means for storing a display order of the images displayed by said display means.