JPH11238077A - Image retrieval device and method and medium recording image retrieval program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve retrieval accuracy in the case of retrieving images similar to a key image based on the color feature of the images. SOLUTION: This device is constituted so as to retrieve an image similar to a designated key image from stored images. In this case, the image retrieval device calculates a similarity degree based on the color feature between the images and retrieves the similar images based on the similarity degree. The image retrieval device calculates the similarity degree by especially increasing weight for color corresponding to a color tone key imparted to the key image at the time of calculating the similarity degree between the images.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieval apparatus and method for retrieving images similar to a designated image based on the color characteristics of the image, and a recording medium storing an image retrieval program. About.

[0002]

2. Description of the Related Art In an image database that manages image data, image data includes search keys such as keywords,
It is registered together with search conditions such as image features such as image shape and color. The image database searches for a desired image from registered image data based on search conditions specified by the user. For example, when a keyword is specified by the user in the image database, an image to which a keyword matching the keyword is added is searched from the registered images, and a predetermined image (hereinafter, “key image”) is searched. Is specified, an image similar to the specified image is searched.

When searching for an image similar to a key image, the image is searched for based on the image features such as the shape, texture pattern and color of the image. That is, the feature amount is extracted from the key image, the extracted feature amount is compared with the feature amount of the image registered in the image database, and the similarity indicating the degree of similarity between the two images is calculated. A registered image determined to be similar based on the similarity is output as a search result.

For example, in an image database for retrieving an image similar in color to a key image, the color of the image is extracted as a feature, a similarity is calculated from the feature, and a key image is calculated based on the similarity. Search for images similar to. At this time, a color that has a large proportion in the image or an average color of many colors included in the image is extracted as a color feature, and a similar image is searched.

[0005]

However, in this case, since the search is not always performed using the color of the user's attention, even if the searched images are similar in color as a whole, the user cannot search for the color. Judging from the sensation, it may not always be said that they are similar.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has as its object to search for an image similar to a designated image based on the color characteristics of the image. An image search apparatus and method capable of obtaining the image search method, and a recording medium storing an image search program.

[0007]

An image retrieval apparatus according to the present invention comprises: an image storage means for storing an image in association with a color key serving as a color search key; a specification means for specifying a key image; A first feature acquisition unit for acquiring a color key and a color feature of a key image designated by the designation unit; a second feature acquisition unit for acquiring a color feature of an image stored in the image storage unit; The degree of similarity between the two images is compared by comparing the color characteristics of the designated image acquired by the characteristic acquisition unit with the color characteristics of the image stored in the image storage unit acquired by the second characteristic acquisition unit. And a search unit that searches the image stored in the storage unit for an image similar to the designated image based on the similarity. When the similarity calculation means calculates the similarity, the image search device increases the weight particularly for the color corresponding to the color key of the key image acquired by the first feature acquisition means.

An image search method according to the present invention stores an image in association with a color key which is a search key for color,
An image retrieving method for retrieving an image similar to a key image from images stored based on the color characteristics of the image, wherein a key image is specified, and a color key and a color characteristic of the key image are obtained. Obtaining a color feature of the stored image; calculating a similarity indicating a degree of similarity between the two images from the color feature of the key image and the color feature of the stored image; Searching for an image similar to the key image from the stored images based on the key image. When calculating the similarity, the weight corresponding to the color key of the key image is increased.

A recording medium according to the present invention is a computer-readable recording medium, and records the following image search program. The image search program is an image search program that stores an image in association with a color key that is a color search key, and searches for an image similar to the key image from among the stored images based on the color characteristics of the image. So,
Specifying a key image, obtaining a color key and a color characteristic of the key image, obtaining a color characteristic of the stored image, a color characteristic of the key image, and a color characteristic of the stored image. Calculating a similarity indicating the degree of similarity between the two images, and searching for an image similar to the key image from the stored images based on the similarity. In this case, the weight of the color corresponding to the color key of the key image is increased.

In the image search apparatus and method and the image search program, the color feature may include a plurality of colors and an occupation ratio of each color in the image.

In the image retrieval apparatus and method, and the image retrieval program, when calculating the similarity, the color corresponding to the color key of the designated image is centered, and the distance from the color in the color space is determined. The weight may be changed by changing the weight.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image retrieval apparatus according to the present invention will be described below with reference to the accompanying drawings. The image search apparatus according to the present embodiment includes a match search for searching for an image that matches a condition indicated by a search key based on a search key such as a keyword, and an image search based on a color feature of the specified image. A similarity indicating the degree of similarity between them is calculated, and a similar image search for searching for a similar image of the designated image based on the similarity is performed.

In particular, when performing a similar image search, the image search apparatus of the present embodiment calculates the similarity by increasing the weight of a color that is easily noticed by a human when viewing the image. In this case, the color of the “color key”, which is a search condition relating to the color added to the image, is used as a search key as a color that is easily noticed by humans. Thereby, a similar image that the user feels more similar can be searched.

<Overall Configuration of Image Retrieval Apparatus> FIG. 1 shows a schematic configuration diagram of an image retrieval apparatus (hereinafter, referred to as a "system") of the present embodiment. As shown in FIG. 1, the system includes a central processing unit (hereinafter, referred to as a “CPU”), and is mainly configured by a control device 1 that controls the entire system.
For the CPU, for example, Pentium manufactured by Intel Corporation is used. The control device 1 includes a display 2 for displaying images, characters, etc., a display for operations, a keyboard 3 and a mouse 4 for performing various inputs and instruction operations, and a data storage medium. Write data /
A floppy disk device 5b and a hard disk device 6 for reading, a printer 7 for printing characters and image data, and a scanner 8 for capturing image data
And a CD-ROM device 9b for reading data stored in the CD-ROM 9a, a speaker 10 for audio output, and a microphone 11 for audio input.

FIG. 2 shows a block diagram of the present system. C
The PU 201 stores, via the data bus 220, a ROM 203 in which a program for controlling the system is stored.
And a RAM 204 for temporarily storing programs and data executed by the CPU 201 for control. A circuit connected to the CPU 201 via the data bus 220 includes a display control circuit 205 for controlling the display 2 for displaying images, characters, and the like, and a keyboard control circuit 2 for controlling transfer of input from the keyboard 3.
06, a mouse control circuit 207 for controlling transfer of an input from the mouse 4, a floppy disk device control circuit 208 for controlling the floppy disk device 5b, and a hard disk device control circuit 20 for controlling the hard disk device 6.
9, a printer control circuit 210 for controlling output to the printer 7, and a scanner control circuit 21 for controlling the scanner 8.
1, a CD-ROM device control circuit 212 for controlling the CD-ROM device 9b, a speaker control circuit 213 for controlling the speaker 10, and a microphone control circuit 214 for controlling the microphone 11. Further, the CPU 201
Is connected to a clock 202 for generating a reference clock required for operating the system.
An expansion slot 215 for connecting various expansion boards is connected via a data bus 220. A SCSI board is connected to the expansion slot 215 and the SCSI
Floppy disk device 5b, hard disk device 6, scanner 8, or CD-ROM device 9 via I board
b or the like may be connected.

Although the floppy disk 5a and the hard disk device 6 are used as image data storage media in the above system, other information storage media such as a magneto-optical disk (MO) may be used. Although the scanner 8 is used as the image data input device, other data input devices such as a still video camera and a digital camera may be used. Further, although the printer 7 is used as an output device, another output device such as a digital copying machine may be used.
In the present system, a program for implementing the control processing is stored in the ROM 203. However, a part or all of the program is stored in an information storage medium such as a floppy disk 5a, a hard disk device 6, or a CD-ROM 9a, and the program and data are read out from the information storage medium to the RAM 204 as necessary, and are read out. It may be executed.

<Image Database> The present system has an image database including image data and additional information serving as search keys for storing and managing the image data. The image database is logically configured on an information storage medium such as the hard disk device 6. FIG. 3 shows an example of the configuration of the image database 50 of the present system. The image database 50 manages “image data” as image information and various search information added to the image data. Search information for image data includes "keyword"
And a search key such as “color key”, and a feature amount that is information indicating a feature of the image data. The feature amount includes information such as a color feature indicating a feature related to a color of the image, an image shape, a texture pattern, and the like. In the image search device of the present embodiment,
In particular, an image search is used by focusing on the color features of the image.

<Control Operation of Image Searching Apparatus> A specific control operation of the present system will be described below with reference to a flowchart.

<Main Routine> FIG. 4 is a flowchart showing a main routine of a program executed by the CPU 201 in the present system. When the program is started, first, an initialization process for initializing flags and the like necessary in the subsequent processes and displaying an initial menu screen is performed (S1). On the initial menu screen, selection items for selecting a predetermined process are displayed as icons, and when one of the selection items is selected by the user, the predetermined process is executed. In the present system, on a setting screen such as an initial menu screen displayed on the display 2 or the like, a user selects various processes and inputs setting values via the keyboard 3 or the mouse 4 or the like. . After step S1, it is determined whether a menu is selected by the user on the initial menu screen (S2). If "image registration" is selected in step S2, the process proceeds to an image registration process (S3) for performing a process such as registering image data in the image database 50 together with its color characteristics, and then proceeds to step S7. If "key search" is selected in step S2, a key search for performing a process of searching the image database 50 for image data matching the specified search condition based on the specified search key and a condition between the search keys. The process proceeds to processing (S4), and then proceeds to step S7. If "similar image search" is selected in step S2, the process proceeds to a similar image search process (S5) for searching the image database 50 for image data similar to the designated image data, and then to step S7. move on. If "other menu" is selected in step S2, other menu processing is performed (S6).
Proceed to step S7. If no menu is selected in step S2, the process proceeds to step S7 without doing anything.
In step S7, other processes are executed. When all processes are completed, the process returns to step S2, and thereafter, the same processes are repeated.

Here, the other menu processing (step S7) is basically the same as that of a general search system, and is not directly related to the present invention. Therefore, the description is omitted here. (Step S3), the key search process (Step S4), and the similar image search process (Step S5) will be described in detail below.

<Image Registration Process> The image registration process (step S3) will be described with reference to the flowchart of FIG. In this processing, the color feature is extracted from the image specified by the user, and the color feature, the color key determined based on the color feature, and the image specified together with the color key specified by the user are converted into an image. A process for registering in the database 50 is performed.

As shown in FIG. 5, first, an image designated to be registered by the user is read (S31). Next, color features are extracted from the designated image (S32).
Here, extraction of the color feature will be described.

In this system, the color space is divided into predetermined areas, and one color is defined for each area. FIG. 6 shows an example of the division of the color space. Here, an HLS color space including hue, lightness, and saturation is used as the color space. The color space may be an RGB color space. These pieces of information are managed by a color space area table logically configured on an information storage medium such as the hard disk device 6. The present system extracts the ratio (image occupancy) of each defined color in the image as a color feature. FIG. 7 shows an example of the color features extracted in this way. In addition, as the color feature, in addition to the occupancy, an object color on the image, a background color, or the like may be used.

Returning to FIG. 5, after extracting the color features (S32), the extracted color features are added to the designated image data (S3).
3). Thereafter, the color is specified based on the extracted color feature, and a color key corresponding to the color is automatically added to the image (S34). The color key is also defined by the HLS color space, for example, as shown in FIG. Here, the name of the color key is associated with a value in the HLS space. Instead of the name of the color key, a value in the HLS space may be directly used, or a number may be used. The color space may be an RGB color space instead of the HLS color space.

Thereafter, the color key designated by the user is added to the designated image (S35). That is, it is determined whether or not the user has designated a color key to be given to the image,
When there is a color key specified by the user, the color key is added to the specified image. Lastly, the extracted color features and the image data specified based on the color features automatically or together with the color key given by the user are registered in the image database 50 (S36). At this time, in addition to the color feature and color key, file information such as a file name and an image size, and setting information by a user such as a keyword and a classification name are also registered for the image. After registration in the image database 50, the process returns.

<Key Search Processing> The key search processing (step S4) will be described with reference to the flowchart of FIG. Here, based on various search keys such as a keyword and a color key designated by the user and conditions between the search keys, an image having additional information matching the conditions is extracted from the image database 50 (match search). )I do.

In FIG. 9, first, a search key set by the user and an AND condition and an OR condition between the search keys are read (S41). At this time, a plurality of search keys may be set, and a keyword, a class name, and the like can be set as the search key in addition to the color key. Also, when a plurality of search keys are set, A
ND and OR conditions can also be set.

Next, search conditions are set based on the read settings (S42). That is, a search key such as a color key or a keyword is replaced by an AND condition
By combining the R conditions based on the conditions, one search condition is set as a whole.

Thereafter, in steps S43 to S46, an image matching the set search condition is extracted from the images registered in the image database 50. That is,
A process of searching for an image similar to the search condition from among images to be searched (hereinafter, referred to as “target image”) registered in the image database 50 is performed.

That is, in the image database 50,
It is determined whether there is any target image that has not yet been compared with the search condition (S43). If there is no target image that has not been compared, the process proceeds to step S47. On the other hand, if there is a target image that has not been compared, the additional information of the target image is read from the image database 50 (S4).
4). Thereafter, the search condition is compared with the search key added to the target image, and it is determined whether or not they match (S45). If they match, the target image is added to the search result list (S46). If they do not match, step S46 is skipped. Steps S43 to S46 are repeated until the similarity is calculated for all the target images in the image database 50. Step S4
When the comparison with the search condition is performed for all the target images in the image database 50 in 3, a search result output process is performed (S 47), and the process returns.

In the search result output process in step S47,
Outputs the image file name of the image registered in the search result list. Alternatively, image data may be output.

<Similar Image Search Process> The similar image search process (step S5) will be described with reference to the flowchart of FIG. In this processing, an image similar to the key image specified by the user is searched from the target images registered in the image database 50. That is, the similarity between the two images is calculated from the color characteristics of the key image and the color characteristics of the target image in the image database 50, and a process of searching for an image similar to the key image based on the similarity is performed. . In this case, in calculating the similarity of the color feature, the similarity is calculated by weighting the color corresponding to the color key added to the image as a color that is particularly noticeable to humans.

Specifically, as shown in FIG. 10, first, a key image designated by the user is read (S5).
1). Next, the color feature amount of the key image and the color key added to the key image are read from the image database 50 (S52, S53). The color key of the key image may be the one specified by the user together with the key image. Then, a search feature is created from the feature of the key image and the color key (S54). Here, the search feature quantity is created as shown in FIG. 11 by referring to the color space area table shown in FIG. 6 and specifying a color area including the color of the tint key. Then, in steps S55 to S57,
The feature amount for search and the feature amount of the target image registered in the image database 50 are compared, the similarity between the two images is calculated, and the similarity of the image group registered in the image database 50 is calculated based on the similarity. Search for an image similar to the key image from inside.

That is, in the image database 50, the similarity with the key image has not been calculated yet.
It is determined whether there is an image to be searched (S55). If there is a search target image in the image database 50, the feature amount of the target image is read from the image database 50 (S56). Then, a similarity is calculated from the feature amount of the key image and the feature amount of the target image (S57). Steps S55 to S57 are repeated until the similarity is calculated for all the target images in the image database 50. After that, when the similarity is calculated for all the target images in the image database 50 (step S55).
NO), a search result output process is performed (S58), and the process returns.

In the search result output process in step S58,
Among the images included in the search result list, image file names are output as search results in descending order of similarity. At this time, a threshold value of the similarity may be provided, and based on the calculated similarity, the image file name may be output only for those whose similarity is higher than a predetermined value (threshold). Alternatively, a predetermined number of image file names may be output in descending order of similarity. Further, the image of the search result may be displayed on a display device such as a display.

Hereinafter, the similarity calculation processing in step S57 will be described. In the similarity calculation process, when calculating the similarity based on the color feature, the similarity is calculated by increasing the weight particularly for the color equal to the color key added to the key image. For example, the key image and the target image are shown in FIG.
And the color keys assigned to the key image are “orange 1” and “green 2”. In this case, the similarity between the key image and the target image is calculated by the following equation. Similarity R = 1.0−sum / 2.0 where difference = | (color feature of target image) − (color feature of key image) | weighted difference = difference / weight Total = difference of difference of all color features Summation As shown in FIG. 12, at the time of similarity calculation, weights are assigned to the color feature “orange” including the color key “orange 1” and the color feature “green” including the color key “green 2”. Is larger than the weight for the other color features.

As described above, in the image search device of the present embodiment, when calculating the similarity between images using the color features,
The calculation is performed with particular weight given to the color corresponding to the color key assigned to the designated image. For this reason, the color to which the user pays more attention is weighted, and a similar image closer to human feeling can be obtained. At this time, since the color key used in the normal match search is used, it is not necessary to newly add information for similar image search.

<Modification> In the above description, the weight is increased only for the color (color area) corresponding to the color key, but the color (color) is centered on the color corresponding to the color key. The weight of an area may be maximized, and the weight of a color (color area) having a short distance in the color space from the color may be changed so as to decrease in accordance with the distance in the color space. For example, in FIG. 12, when the color corresponding to the color key is “blue”, “blue” is 1.5, “greenish blue” and “blue purple” are 1.2, The weight may be set to 1.1 for “blue-green” and “purple”.

The similarity between images is calculated using only color features, but may be calculated in combination with other feature amounts. Also in this case, when calculating the contribution of the color feature in the similarity, the same effect as described above can be obtained by weighting the color corresponding to the color key and calculating.

[0040]

According to the image retrieval apparatus, the image retrieval method, or the recording medium in which the image retrieval program is recorded according to the present invention, when calculating the similarity, the key image is added to the key image as a color that is easily noticed by humans. Calculation is performed with particular weight given to the color of the color key. For this reason, a search result close to a human similarity can be obtained, and search accuracy can be improved.

[Brief description of the drawings]

FIG. 1 is a diagram schematically illustrating the configuration of an image search device according to an embodiment.

FIG. 2 is a block diagram mainly showing a control device of the image search device.

FIG. 3 is a diagram showing an example of the configuration of an image database.

FIG. 4 is a flowchart of a main routine of the image search device.

FIG. 5 is a flowchart of an image registration process.

FIG. 6 is a diagram showing an example of the configuration of a color space area table.

FIG. 7 is a diagram showing an example of extracted color features.

FIG. 8 is a diagram showing an example of a color key.

FIG. 9 is a flowchart of a key search process.

FIG. 10 is a flowchart of a similar image search process.

FIG. 11 is a diagram illustrating an example of a feature amount for search.

FIG. 12 is a view for explaining calculation of similarity.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Control device 2 ... Display 3 ... Keyboard 4 ... Mouse 5a ... Floppy disk 5b ... Floppy disk device 6 ... Hard disk 9a ... CD-ROM 9b ... CD-ROM device 50 ... Image database 201 ... CPU 203 ... ROM 204 ... RAM.

Claims (9)

[Claims] 1. An image storing means for storing an image in association with a color key which is a search key relating to color, a specifying means for specifying a key image, a color key and a color of a key image by the specifying means. A first feature acquisition unit for acquiring a feature; a second feature acquisition unit for acquiring a color feature of an image stored in the image storage unit; and a color of a key image acquired by the first feature acquisition unit. A similarity calculating unit that calculates a similarity indicating a degree of similarity between the two images by comparing the feature with a color feature of the image stored in the image storing unit acquired by the second feature acquiring unit; And searching means for searching for an image similar to the key image based on the similarity from among the images stored in the storage means, wherein the similarity calculating means calculates the similarity, The first feature acquiring means An image search apparatus characterized in that a weight corresponding to a color key of a key image acquired by a step is increased in weight. 2. The image search device according to claim 1, wherein the color feature includes a plurality of colors and an occupation ratio of each color in the image. 3. The image retrieval apparatus according to claim 1, wherein said similarity calculating means calculates a color of a key image obtained by said first feature obtaining means when calculating said similarity. An image search apparatus characterized in that a color corresponding to a taste key is centered and a weight is changed according to a distance from the color in a color space. 4. An image search method for storing an image in association with a color key serving as a color search key and searching for an image similar to the key image from the stored image based on the color characteristics of the image. A specification step of specifying the key image; a first characteristic obtaining step of obtaining a color key and a color characteristic of the key image; and a second characteristic obtaining step of obtaining a color characteristic of the stored image. A similarity calculating step of calculating a similarity indicating a degree of similarity between the two images from the color characteristics of the key image and the color characteristics of the stored image; and calculating the similarity based on the similarity. A search step of searching for a similar image from the stored images. In the search step, when calculating the similarity,
An image search method, wherein a weight corresponding to a color corresponding to a color key of the key image is increased. 5. The image search method according to claim 4, wherein the color feature includes a plurality of colors and an occupation ratio of each color in the image. 6. The image search method according to claim 4 or 5, wherein in the search step, when calculating the similarity, a color corresponding to a color key of the key image is centered,
An image search apparatus characterized in that a weight is changed according to a distance from a color in a color space. 7. A computer-readable recording medium, wherein an image is stored in association with a color key serving as a color search key, and an image similar to the key image is stored based on the color characteristics of the image. An image retrieval program for retrieving from the list, a designation step of designating the key image; a first feature acquisition step of acquiring a color key and a color feature of the key image; and a color of the stored image. A second feature obtaining step of obtaining a feature; a similarity calculating step of calculating a similarity indicating a degree of similarity between the two images from a color feature of the key image and a color feature of the stored image; A search step of searching the stored image for an image similar to the key image based on the similarity, and calculating the similarity in the search step. When,
A recording medium for recording an image search program, wherein a weight corresponding to a color corresponding to a color key of the key image is increased. 8. The recording medium according to claim 7, wherein the color feature includes a plurality of colors and an occupancy of each color in an image. 9. The recording medium according to claim 7, wherein in the searching step, when calculating the similarity, a color corresponding to a color key of the key image is centered,
A recording medium for recording an image search program, wherein a weight is changed according to a distance in a color space from the color.