JPH08263522A - Image retrieving method - Google Patents

Abstract

PURPOSE: To retrieve an image file having an image part approximate to a template image without specifying a key word by selecting the matching image file according to a matching function representing how much the template image and image file are matched with each other. CONSTITUTION: This method is equipped with a CPU 20, a computing element 22 which performs operation such as color conversion and reduction (thinning-out), etc., a hard disk drive 32, a magnetooptic disk drive (MO) 24, a CD-ROM drive 36, a video player 38, an image memory 40, a display part 42 such as a CRT, an input operation part 44 such as a mouse and a keyboard, and an external interface 46, and those elements are connected to one another through a bus 50. An image data base 30 is stored on an external storage device such as the hard disk drive 32, magnetooptic disk drive 34, and CD-ROM drive 36, an image is fetched as a still picture from the video player 38, and an image retrieval system can be connected to even an external data base through the external interface.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for retrieving an image file having characteristics similar to a desired characteristic from an image database including a plurality of image files.

[0002]

2. Description of the Related Art Conventionally, in the case of retrieving an image file from an image database, it has been usual to execute a retrieval by using a keyword registered in advance for each image file.

[0003]

However, with this method, it is impossible to perform a search for the characteristics of an image file in which no keyword is registered. Also,
Prior to the search, it was necessary to register the keywords.

The present invention has been made to solve the above problems in the prior art, and an object of the present invention is to provide a method for searching by specifying the characteristics of an image file without specifying a keyword. .

[0005]

In order to solve the above-mentioned problems, the invention described in claim 1 uses an image database including a plurality of image files to generate an image file having characteristics similar to desired characteristics. A method of searching, comprising: (a) designating a template image to be searched; and (b) calculating a matching function indicating a degree of matching between the template image and an image file in the image database, Selecting an image file matching the template image from the image database.

Since the image file to be matched is selected according to the matching function indicating the degree of coincidence between the template image and the image file, it is possible to search for an image file having an image portion close to the template image without specifying a keyword. it can.

According to the image search method described in claim 2,
The step (a) includes a step of designating a mask area to be excluded from the search target in the template image, and the step (b) is an image file matching the part of the template image excluding the mask area. From the image database.

If a portion of the template image to be excluded from the search target is designated as the mask area, the matching process can be performed only on the image portion to be searched.

In the image search method described in claim 3,
The step (a) includes a step of designating a partial area of the document image as a template image.

In this way, the template image can be easily designated.

According to the image search method described in claim 4,
In the step (a), a step of designating a shape of a template image using a pointing device, a step of designating a desired color for the shape, and an image of the shape of the designated color are set as the template image. And a step.

In this way, even if there is no original image, the template image can be easily set by designating an arbitrary shape and its color.

According to the image search method of the fifth aspect,
The template image is a thinned image, and the step (b) includes a step of matching the template image with a thinned image for each of the plurality of image files.

In this way, the amount of image data to be processed can be reduced, so that the matching process can be performed at a higher speed.

According to the image search method of the sixth aspect,
The step (a) includes a step of designating a plurality of template images to be searched, and the step (b) includes a step of selecting an image file matching the plurality of template images.

In this way, it is possible to successfully search for an image file including the image portions of a plurality of template images to be searched.

According to a seventh aspect of the present invention, there is provided a method for retrieving an image file having a feature similar to a desired feature from an image database including a plurality of image files, which comprises (a) a color distribution to be retrieved. By designating as template data, and (b) calculating a matching function indicating the degree of matching between the template data and the color distribution of the image files in the image database,
Selecting an image file matching the template data from the image database.

Since an image file to be matched is selected according to a matching function indicating the degree of matching between the template data and the color distribution of the image file, an image file having a color distribution approximate to the template data is selected without specifying a keyword. You can search.

[0019]

1 is a block diagram showing the structure of an image retrieval system to which an embodiment of the present invention is applied. This image search system, CPU 20, matching, color conversion,
A computing unit 22 that performs computations such as reduction (thinning), a hard disk device 32, and a magneto-optical disk device (MO) 24.
A CD-ROM device 36, a video player 38,
An image memory 40, a display unit 42 such as a CRT, an input operation unit 44 such as a mouse or a keyboard, and an external interface 46 are provided. These respective elements are connected to each other via a bus 50. Image database 30
Is a hard disk device 32 or a magneto-optical disk device 34.
And a CD-ROM device 36 or other external storage device. The image can be captured as a still image from the video player 38. The image retrieval system can also connect to an external database through the external interface 46.

FIG. 2 is a flow chart showing the procedure of the embodiment. In step S1, template data indicating an image portion to be searched is set. FIG. 3 is an explanatory diagram showing the processing content of step S1. First, FIG.
As shown in (A), the reference image is displayed on the display unit 42. As this reference image, an arbitrary original image such as a photo original or an illustration can be used. Next, FIG.
As shown by a broken line in (B), the operator designates a desired image portion as a template image using a pointing device such as a mouse or a trackball. FIG.
In the example of (B), a rectangular first image portion TP1 including a house
And a rectangular second image portion TP2 including a tree are designated as template images. FIG. 3 (C),
(D) shows two extracted template images TP1,
TP2 is shown.

In step S2 of FIG. 2, the operator selects a matching method. The following matching methods are available.

[Matching method 1]: Color image data (RGB data, YMCK data, etc.) of a template image is converted into monochrome data having gradation (hereinafter referred to as "gradation monochrome data"), and this gradation monochrome is obtained. Match using the data.

[Matching method 2]: Matching is performed using all the color components of the color image data of the template image.

[Matching method 3]: One or more color components are designated, and matching is performed on the designated color components.

[Matching method 4]: Color image data of the template image is converted to data of another color system (YCrCb).
System) and perform matching using the converted data.

[Matching method 5]: The color image data of the template image is converted into binary data, and matching is performed using the binary data.

When the matching method 1 is selected, that is, when the matching is performed using the gradation monochrome data, the degree of coincidence between the template image and each image file is shown according to the following formulas 1 to 4, for example. Correlation function f (x, y) (hereinafter referred to as "matching function")
To calculate.

[0028]

[Equation 1]

[Equation 2]

(Equation 3)

[Equation 4]

Here, (x, y) is the coordinates in the template image, t (x, y) is the image data of the template image, and g (x, y) is the image data of the image file to be matched ( Hereinafter referred to as "target image data"),
N is the number of pixels in the template image. Whether or not the target image and the template image match is determined by whether or not the matching function f (x, y) is equal to or more than a predetermined threshold value. The threshold value is set in step S4 described below and can be changed as appropriate.

The matching function f (x, y) given by the above equations 1 to 4 can also be used for the matching method 5 using binary data.

When the matching method 2 is selected, a matching function for color image data is used. In this embodiment, the target image data g (x, y) and the template image data t (x, y) are, for example, the following Equation 5:
As shown in 6 and 6, consider the case where each component is composed of RGB components.

[0032]

(Equation 5)

(Equation 6)

At this time, if the above expressions 2 to 4 are expanded to color image data, the following expressions 7 to 9 are obtained.

[0034]

(Equation 7)

(Equation 8)

[Equation 9]

In the matching method 3, when some of the plurality of color components are designated (for example, when only the R component is designated among the RGB components), the undesignated colors in the formulas 7-9. You can set the ingredient as 0.

When the matching method is specified in step S2 of FIG. 2, the operator specifies the range of the image file to be searched in the image database in step S3. For example, if the image database is
If the image files are classified into image files such as “landscape”, “house”, and “person”, the “house” image group is searched for the first template image TP1 shown in FIG. 3C. It is designated as the target range, and the image group of “landscape” is designated as the search target range for the second template image TP2 shown in FIG.

Alternatively, an image file containing a specific character string in the file name can be set as the search target range. For example, for the first template image TP1 shown in FIG. 3C, an image file including the character string “HOUSE” as the file name is designated as the search target range,
For the second template image TP2 shown in (D), an image file containing a character string of "NATURE" or "TREE" can be designated as the search target range. In addition,
The entire image database may be designated as the search target range.

In step S4 of FIG. 2, the computing unit 22
The matching process between the image file in the designated search target range and the template image is executed. FIG. 4 is an explanatory diagram showing a matching processing method. As shown in FIG. 4, the target image of the matching process is usually larger than the template image. Therefore, the template image TP1 is sequentially moved within the region of the target image, and the matching function f (is calculated according to the above formulas 1 to 4 (or formulas 1, 7 to 8) for each position (X, Y) of the template image TP1. x, y) is calculated. Then, when the value of the matching function f (x, y) becomes equal to or larger than the predetermined threshold value in at least one place, it is determined in step S5 that the target image is a search result candidate image.

If the target image is a candidate image, then FIG.
From step S5 to step S6, and the display unit 42
The data related to the candidate image is displayed on. For example, the candidate image itself may be displayed as the data related to the candidate image, or the file name of the candidate image may be displayed.

In step S7, it is determined whether or not an unprocessed image file remains in the search target range. If there is an unprocessed search image file, the steps S4 to S6 are repeated. In this way, the matching process is performed for each image file in the search target range, and an image file having an image portion similar to the template image is searched.

As shown in FIGS. 3C and 3D,
When two template images TP1 and TP2 are designated, step S1 of FIG. 2 is performed for each template image.
~ S7 are executed respectively.

In the above embodiment, the reference image (see FIG.
A part of (A)) is extracted as a template image, and an image file having an image portion similar to this template image is searched for by a matching process. Therefore,
An image file having a part similar to an arbitrary image part can be easily searched for without using a keyword.
In particular, when the part of the template image is a part of the image file of the image database extracted, the image file including the template image can be searched well.

If the image database has thinned-out images in each image file, the thinned-out template images are created by thinning out the template images at the same thinning-out rate. Then, a matching function between the thinned-out template image and each thinned-out image in the image database may be calculated, and the matching degree may be checked.
By doing so, the amount of image data to be processed by the matching process can be reduced, so that there is an advantage that the matching process can be performed at a higher speed.

As a method of designating the template image, various methods other than the method shown in FIG. 3 can be considered.

For example, as shown in FIG. 5, a plurality of characteristic image portions TPa to TP in the image portion of the house to be searched.
It is also possible to specify f as a template image. In this case, the specified template images T
An image file matching all of Pa to TPf is searched. For example, each template image TPa to TPf
The sum of the matching functions of is used as an evaluation value, and an image file whose evaluation value is equal to or larger than a predetermined threshold may be determined as a candidate image. Alternatively, an image file in which all the values of the matching functions of the template images TPa to TPf are equal to or more than a predetermined threshold may be determined as the candidate image.

As described above, when a plurality of relatively small image portions are set as the template image, the data amount of the template image can be reduced, and therefore, there is an advantage that the matching process can be performed at a higher speed. Further, there is also an advantage that an image portion desired to be excluded from the matching target can be successfully excluded from the template image.

Further, as shown in FIG. 6, it is possible to designate an area to be excluded from the matching processing target in the template image as a mask area. By doing so, it is possible to successfully exclude from the template image the image portion that is desired to be excluded from the matching target.

In the above embodiment, the template image is extracted from the reference image, but the operator sets the template image by designating the shape (outer shape) of the template image and the color in the shape. Is also possible. In FIG. 7, the operator uses the pointing device to shape the template image (human outline).
Is input and the inside of each closed figure is colored with a desired color to show the template image TP3. This makes it possible to search for an image file including a desired characteristic image portion (human image in the example of FIG. 7) even when there is no reference image. Thus, when using an abstract image as a template image,
The level of the threshold used to determine whether or not to make a candidate image is
Set to a relatively low value.

Instead of designating a template image for matching, it is also possible to designate a color distribution of the image, for example, a color distribution histogram as template data. In this case, an image file having a color distribution histogram similar to the designated color distribution histogram is searched. FIG. 8A is a graph showing an example of a color distribution histogram designated as template data, in which the horizontal axis represents gradation z (level of image data),
The vertical axis represents the frequency Ht. In this case, for example, a matching function f (Hg, Ht) shown in the following Expression 10 can be used to check the degree of matching between the template data Ht (z) and the color distribution histogram Hg (z) of the target image. it can.

[0050]

[Equation 10]

At this time, the total frequency of the template data Ht (z) and the color distribution histogram Hg of the target image.
If the sum of the frequencies of (z) is normalized to the same value (for example, the frequencies are expressed as a percentage), the matching function f (Hg, Ht) can be obtained regardless of the size of the target image.

As the matching function f (Hg, Ht), instead of Equation 10, the following Equation 11 or Equation 1
It is also possible to use the functions defined by 2-15.

[0053]

[Equation 11]

[0054]

(Equation 12)

(Equation 13)

[Equation 14]

(Equation 15)

Here, M is the total number of gradations, and for example, when the gradation range is 0 to 255, M = 256.

When the template data Ht (z) is designated for the color image data, the operator designates a color distribution histogram as shown in FIG. 8A for each color component. At this time, only some of the color components may be used as the template data Ht (z). The same density distribution histogram can also be used as template data for gradation monochrome data.

As described above, if the color distribution histogram is designated as the template data, it is possible to retrieve an image file having a characteristic color distribution histogram without designating a keyword.

As shown in FIG. 8B, it is possible to specify the gradation range as a mask range and not execute the matching function calculation in this mask range. By doing so, it is possible to successfully search for an image file having a distribution similar to the characteristic gradation part (the part other than the mask range) of the color distribution histogram.

The present invention is not limited to the above embodiments, and can be implemented in various modes without departing from the scope of the invention.

[0060]

As described above, according to the invention described in claim 1, since the image file to be matched is selected according to the matching function indicating the matching degree between the template image and the image file, the keyword is designated. Without,
An image file having an image portion similar to the template image can be searched.

According to the second aspect of the invention, the matching process can be performed only on the image part to be searched by designating the part to be excluded from the search target in the template image as the mask area.

According to the invention described in claim 3, the template image can be easily designated.

According to the invention described in claim 4, even when there is no document image, the template image can be easily set by designating an arbitrary shape and its color.

According to the invention described in claim 5, since the data amount of the image data to be processed can be reduced, the matching process can be performed at a higher speed.

According to the sixth aspect of the present invention, it is possible to search the image file including the image portions of the plurality of template images to be searched with high accuracy and with good accuracy.

According to the invention described in claim 7, since the image file to be matched is selected according to the matching function indicating the matching degree between the template data and the color distribution of the image file, the keyword is not designated, An image file having a color distribution similar to the template data can be searched.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an image search system to which an embodiment of the present invention is applied.

FIG. 2 is a flowchart showing a procedure of an embodiment.

FIG. 3 is an explanatory diagram showing a template data setting method.

FIG. 4 is an explanatory diagram showing a matching processing method.

FIG. 5 is an explanatory diagram showing a method of performing a search by designating a plurality of template images.

FIG. 6 is an explanatory diagram showing a mask area set in a template image.

FIG. 7 is an explanatory diagram showing a template image in which a shape is input by an operator.

FIG. 8 is a graph showing an example of a color distribution histogram.

[Explanation of symbols]

 20 ... CPU 22 ... Arithmetic unit 30 ... Image database 32 ... Hard disk device 34 ... Magneto-optical disk device 36 ... CD-ROM device 38 ... Video player 40 ... Image memory 42 ... Display unit 44 ... Input operation unit 46 ... External interface 50 …bus

Claims (7)

[Claims] 1. A method of retrieving an image file having a feature similar to a desired feature from an image database including a plurality of image files, the method comprising: (a) specifying a template image to be retrieved; ) Selecting an image file that matches the template image from the image database by calculating a matching function that indicates the degree of matching between the template image and the image file in the image database. How to search for images. 2. The image search method according to claim 1, wherein the step (a) includes a step of designating a mask area to be excluded from a search target in the template image, and the step (b) includes An image retrieval method comprising a step of selecting an image file matching the part of the template image excluding the mask area from the image database. 3. The image search method according to claim 1, wherein the step (a) includes a step of designating a partial area of the original image as a template image. 4. The image search method according to claim 1, wherein in step (a), a shape of a template image is designated using a pointing device, and a desired color is designated in the shape. An image search method comprising: a step of setting an image of the shape of the designated color as the template image. 5. The image search method according to claim 1, wherein the template image is a decimated image, and the step (b) includes the template image and the plurality of image files. An image search method including a step of matching a thinned image for each. 6. The image search method according to claim 1, wherein the step (a) includes a step of designating a plurality of template images to be searched, and the step (b). Is an image search method including a step of selecting an image file that matches the plurality of template images. 7. A method for retrieving an image file having features similar to a desired feature from an image database including a plurality of image files, the method comprising: (a) designating a color distribution to be retrieved as template data. , (B)
Selecting an image file that matches the template data from the image database by calculating a matching function that indicates the degree of matching between the template data and the color distribution of the image files in the image database. Characteristic image retrieval method.