JPH07319883A - Device and method for retrieving picture - Google Patents

Abstract

PURPOSE:To provide an image retrieval device/method, by which flexible retrieval of a natural image can be realized as against retrieval designation by means of image information consisting only of a main part constituting a screen from an external part. CONSTITUTION:This device is provided with a picture accumulation means 1 accumulating a color natural image, an index bestowal means 2 bestowing an index by using the combination of basic graphics for the respective accumulated color natural color pictures, a retrieval designation means 3 designating the retrieval of the natural image by the combination of the basic graphics, a retrieval means 4 which sequentially retrieves the color natural image having the indexes similar to the combination of the basic graphics which is retrieval- designated by the retrieval designation means 3 among the color natural pictures of the image accumulation means 1 and an output means 5 outputting the retrieved color natural images.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieving apparatus and an image retrieving method, and in particular, a color natural image is provided with a predetermined index, and based on this index, conditions most externally specified for retrieval are assigned. The present invention relates to an image search device and an image search method for presenting a color natural image being displayed.

In the present invention, a color natural image is not an image in which the object to be photographed is limited to a specific object, but is generally a natural image such as a landscape photographed by a camera or a television camera.

[0003]

2. Description of the Related Art Conventionally, as a first method for storing natural images photographed by a camera or the like in an image database and retrieving the stored natural images, the natural image generally has a clear contour. Since it is not an image, there is a method of extracting a representative color of the entire screen, or a method of focusing on only the outline that can be clearly extracted.

When the above-mentioned first method cannot be used for the search, the second method is to manually recognize the image content of the natural image stored in the image database, or
Interpret and give a keyword in characters. When searching, the computer interprets the keyword and presents a matching image.

[0005]

However, the second conventional method has a problem that only a strict match search can be realized because a technique for flexibly interpreting the keyword itself by a computer has not been established. Also, when trying to search for an image manually, the search does not mean that the person accurately stores the image information of the natural image, and the search is specified only by the main part that remains in memory. That means
It is necessary to realize a flexible search by using only important image information that constitutes the screen instead of exact matching, but it is not realized at present.

The present invention has been made in view of the above-mentioned problems, and solves the above-mentioned problems of the prior art, and is flexible and natural even for search designation by image information of only the main part constituting the screen from the outside. An object is to provide an image search device and an image search method capable of image search.

[0007]

FIG. 1 is a block diagram showing the principle of the present invention.

The image retrieving apparatus of the present invention uses an image accumulating means 1 for accumulating color natural images and a combination of basic figures for each color natural image accumulated in the image accumulating means 1 to create an index. The index assigning means 2 for assigning, the search designating means 3 for designating the search of the color natural image stored in the image storing means 1 by the combination of the basic figures, and the search designating means from the color natural images of the image storing means 1. 3, a search means 4 for sequentially searching for a color natural image having an index similar to the combination of the basic figures specified by the search specification 3, and an output means 5 for outputting the color natural image searched by the search means 4.
Have and.

Further, the above-mentioned search means 4 is provided between the combination of the basic figures designated by the search and the index of the color natural image stored in the image storage means 1 provided by the combination of the basic figures. Position, area of basic figure on screen,
It includes a similarity calculation means for calculating the similarity between the search target image designated by the search designation means 3 by paying attention to the shape and color and the natural color image stored in the image storage means 1.

Further, the searching means 4 sequentially searches the natural color images accumulated in the image accumulating means 1 from natural images having a high degree of similarity to the retrieval target.

FIG. 2 is a diagram for explaining the principle of the present invention.

According to the image retrieval method of the present invention, color natural images and respective color natural images are indexed and accumulated using a combination of basic figures (step 1), and the basic figures are searched at the time of retrieval. The combination is used to specify the search with the search key (step 2), and the color natural image closest to the content specified by the search key is searched from the plurality of accumulated color natural images and presented (step 3).

Further, in the image search method, when a search is designated (step 2), the position of each basic graphic in the screen between the combination of the basic graphics specified by the search and the index, The degree of similarity between the search target and the color natural is calculated by focusing on the area, shape, and color, and based on the calculation result, the accumulated natural color images are sequentially searched and presented in descending order of similarity. .

[0014]

According to the present invention, an index using a basic figure is given to a large number of natural pictures already stored in the natural picture database, and when the natural picture database is searched, a combination of basic figures is used. By designating and referring to the index, natural images similar to the retrieval designation are sequentially presented in descending order of similarity. With this, at the time of search, by specifying a combination of basic figures as a search key, based on the element of the basic figure on the index side and the element of the basic figure specified by the search,
Similarity is required. The high degree of similarity means that there is a high possibility that it is a desired natural image. Further, since the images are displayed in descending order of similarity, a desired natural image can be searched for in a short time.

[0015]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 3 shows the configuration of an image search apparatus according to an embodiment of the present invention. The image retrieval apparatus shown in FIG. 1 includes a natural image database 13 in which color natural images 11 are stored, an index assigning unit 12 for assigning an index to each of the color natural images stored in the natural image database 13 based on a basic figure, Based on the search information input from the search designation unit 14 and the search designation unit 14 that designates search information using a basic graphic input from the outside, a color natural image having a graphic similar to the input search information is generated. The natural image database 13 is composed of a similarity calculation unit 16 that obtains a similarity for searching and a display unit 15 that displays the searched result (color natural image).

FIG. 4 shows an example of the hardware configuration of the image retrieval apparatus according to the embodiment of the present invention. As shown in FIG. The natural image database 13 stores color natural images obtained by the image scanner 21 and the television camera 22 using a magnetic disk. The keyboard 30 is used for input when the search designation unit 14 designates search information. C as the display unit 15
The RT display 27 is used.

A natural image to be stored is input to the natural image database 13 by using the image scanner 21 and the television camera 22.

The index assigning unit 12 and the search designating unit 14 combine the basic figures by using the tablet 28 and the mouse 29 to add an index or designate a target image.

It is also possible to directly specify and search the internal representation of basic figures stored in the later-described natural image database 13 using the keyboard 30.

A central processing unit (CPU) 25 and a large amount of memory 26 are provided as devices for controlling the entire image retrieval apparatus.
A personal computer or workstation (23) having a computer is used. Further, the similarity calculation unit 16 that extracts a natural image similar to the basic graphic designated by the search designation unit 14 is executed by the CPU 25.

First, the case where the index assigning unit 12 assigns an index to the natural images stored in the natural image database 13 will be described.

When the index is manually added, the index is added by a combination of basic figures of circle, ellipse, triangle and quadrangle while looking at the input image. FIG. 5 is a diagram for explaining indexing according to an embodiment of the present invention. Same figure (A)
Is an input image, and FIG. 7B is an index created by combining basic figures. At this time, the internal representation of the index is an index (triangle, square, circle, triangle).

Each basic figure has not only its shape but also information about its position. In this embodiment, the basic figure is a circle,
An example of the internal representation of the information about the position when using an ellipse, a triangle, and a quadrangle is shown.

The three coordinates representing the three vertices in the case of a triangle are (0.0 0.0) (0.1 0.0) (0.05
0.05) Four coordinates representing four points in the case of a rectangle are (0.0 0.0) (0.1 0.0) (0.1
0.1) (0.0 0.1) When representing a circle, the coordinates of the center and the radius are ((0.05 0.05) 0.025) When representing an ellipse, the coordinates of the center and the radius of the horizontal axis. It is assumed that the coordinate of the radius of the vertical axis is ((0.05 0.05) 0.020 0.01) and that each basic figure exists one by one.

In the above example, the expression in which the vertical and horizontal sizes of the image are each normalized to a maximum of 1 is used, but it is also possible to express it in absolute coordinates obtained from the size of the image.

Next, how the similarity of a natural image is calculated using an index using a basic figure will be shown below.

In the case of determining the similarity between the index of the combination of basic figures given to a natural image and the combination of basic figures designated for retrieval, a procedure applying human visual characteristics is used. To do.

From the fact that, in general, a person can search and specify only an impressive portion even when seeing one natural image,
Even if there is a natural image that perfectly matches the original search purpose, it is not possible to completely specify the search given to it.
Generally, it has a characteristic that it is difficult. That is, as shown in FIG. 6, when the number of stored screens in the natural image database 13 reaches tens of thousands, it is impossible for a person who searches for natural images to remember all the contents. Normally, the index 31 given to a natural image and the search condition 32 using a basic figure designated from the outside are designated only by the part that is impressive because the number of basic figures of the latter is small. That's what it means.

Therefore, in the present embodiment, index creation or
In order to realize the characteristic that the larger the area, the more impressive it is when inputting the search conditions: ・ Emphasize a large basic figure, that is, increase the weight; ・ In a large basic figure, the difference in shape Is particularly heavy,
That is, controlling the weight by the difference in shape; -expressing which area in the natural picture is occupied by the position of the center of gravity of the basic graphic; -coloring the color of the basic graphic in the natural picture Yes; express with various characteristics such as.

FIG. 7 is a diagram showing the relationship between the search designation and the index of the natural image database according to the embodiment of the present invention. When the search designation consists of a combination of n basic figures as shown in FIG. 9A, the n shown in FIG.
Select + m.

Note that, in FIG. 7, for the sake of explanation, the basic figures are arranged in order of decreasing area.

Next, the index R related to the similarity is obtained, and the natural images indexed in descending order of the index R are displayed on the display unit 1.
Present to 5. The above series of flow is shown in the flowchart of FIG.

Step 101) The basic figures searched and designated by the search designation section 14 are arranged in descending order of area (the largest basic figure has a count value i = 1).

Step 102) Natural image database 13
The index basic figure j given to the natural image is associated so that the difference r _ij between the figure accumulated in the above-mentioned and the basic figure is minimized.

[0036]

[Equation 1]

Step 103) Natural image database 13
J is removed from the combination of the basic figures given to the natural image stored in.

Step 104) The process after step 102 is repeated until i becomes n and the process of all the n basic figures designated for retrieval is completed.

Step 105) The similarity count is calculated by the following formula.

[0040]

[Equation 2]

Each of the above items is as follows, and FIG. 9 shows the relationship of the respective graphic characteristics. d _ij : Distance between the centroids of the basic figure i designated for retrieval and the basic figure j of natural image index (FIG. 9A) e _ij : The shape of the basic figure i designated for retrieval and the basic figure j of natural image index Difference (FIG. 9 (B)) s _i : area of basic figure i designated for retrieval f _ij : area difference between basic figure i designated for retrieval and basic figure j of index of natural image (example: same shape → 0 , Square and circle → 0.5, triangle and circle or triangle and square → 1.0)
(FIG. 9C) g _ij : inclination of the longest axis of the basic figure i designated for retrieval and the basic figure j of the index of the natural image (when the length of the longest axis is 1, g =
tan θ) (FIG. 9 (D)) h _ij : Color difference (distance on color solid) between basic figure i specified for retrieval and basic figure j of index of natural image (FIG. 9 (E)) The difference h is obtained by the following method.

H _ij = (x _i −x _j ) ² + (y _i −y _j ).
^{_{2 + (z i -z j)}} 2 _{where, x i = s i cos (} 2πH i) y i = s i sin (2πH i) z i = L i _{and, x j = s j cos (} 2πH j) y _j = s _j sin (2πH _j ) z _j = L _j .

S is the saturation, H is the hue, and L is the lightness, and the color of each basic figure is the average of all the pixels in the area surrounded by the basic figure.

Α, β, γ, δ, ε, η: Constants that give a constant weight to the above-mentioned graphic characteristics. In this embodiment, a constant weight (α to η) is added to the combination of the above-mentioned graphic characteristics. In addition, add. In order to perform the above calculation and search the natural image database 13 for the natural image closest to the search condition input from the search designation unit 14, the smaller the difference between the above calculation results is, the more similar the results are. Search for natural pictures you have.

In order to accurately obtain the above-mentioned similarity index R, _{(n + m)} P _{n is set} between one natural image index (combination of basic figures) and search designation (combination of basic figures _). Although it is necessary to calculate the minimum value for each combination, the calculation requires an enormous amount of time. Therefore, in the present invention, the calculation is approximately performed by the procedure shown in the flowchart of FIG. That is, the calculation time is significantly reduced by comparing the basic figures designated by search in descending order of area.

In the above embodiment, the comparatively simple similarity index R is used for description, but it is also possible to use a more complicated index that can be defined by a combination of characteristics of basic figures. .

The display unit 15 outputs a natural image similar to the combination of the basic figures input by the search designation unit 14 by the similarity calculation unit 16 described above.

As described above, in the indexing unit 12, the natural image is visually recognized by the operator in the indexing unit 12 so that the natural image is indexed by the combination of the basic figures and registered in the natural image database 13. When retrieving an image, the retrieval designating unit 14 designates the retrieval designating information by a combination of basic figures, and the similarity calculating unit 16 makes the natural image similar to the retrieval designating information based on the retrieval designating information. Is retrieved from the natural image database 13 and output to the display unit 15.

In the present embodiment, an example in which the index adding section 12 is performed manually has been described. However, the present invention is not limited to this example, and the feature extraction of a natural image is performed by an existing image recognition technique or the like. Alternatively, an index may be added according to the extraction result.

Further, in the above embodiment, the triangle, the square,
Although the description has been made using a circle or an ellipse, any preset graphic may be used.

The present invention is not limited to the above embodiments, and various modifications and applications are possible within the scope of the claims.

[0052]

As described above, according to the present invention, according to the present technology, an index using a combination of basic figures is added to a color natural image for which image feature extraction such as clear contour extraction cannot be performed. It is possible to express the similarity between the search designation specified by the combination of figures and the graphic index of each natural image in the natural image database, and search only the important part that has an impression in the search specification. By doing, from among the natural images accumulated in the database,
It is possible to sequentially present color natural images with high similarity.

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of the present invention.

FIG. 2 is a diagram for explaining the principle of the present invention.

FIG. 3 is a configuration diagram of an image search apparatus according to an embodiment of the present invention.

FIG. 4 is a diagram illustrating a hardware configuration example of an image search device according to an embodiment of the present invention.

FIG. 5 is a diagram for explaining indexing according to an embodiment of the present invention.

FIG. 6 is a diagram showing a relationship between an index of a natural image database and search designation according to an embodiment of the present invention.

FIG. 7 is a diagram showing a relationship between a search designation and an index of a natural image database according to an embodiment of the present invention.

FIG. 8 is a flowchart showing a procedure for calculating a similarity according to an embodiment of the present invention.

FIG. 9 is a diagram showing a relationship of graphic characteristics according to an embodiment of the present invention.

[Explanation of symbols]

 1 Image Storage Means 2 Indexing Means 3 Search Designating Means 4 Similarity Acquiring Means 5 Output Means 10 Searching Units 11 Color Natural Images 12 Indexing Units 13 Natural Image Database 14 Search Designating Units 15 Displaying Units 16 Search Natural Images 21 Image Scanners 22 TV camera 25 CPU 26 Memory 27 CRT display 28 Tablet 29 Mouse 30 Keyboard 31 Index 32 Search condition

Claims (5)

[Claims] 1. An image accumulating means for accumulating color natural images, and an indexing means for applying an index to each color natural image accumulated in the image accumulating means by using a combination of basic figures. A search designating unit that designates a search for a color natural image stored in the image storing unit by a combination of the basic figures, and a basic search target designated by the search designating unit from the color natural image stored in the image storing unit. An image search apparatus comprising: a search unit that sequentially searches a color natural image having an index similar to a combination of figures, and an output unit that outputs the color natural image searched by the search unit. 2. The basic search between the combination of the basic figures designated by the search and the index of the natural image stored in the image storage means provided by the combination of the basic figures. Similarity degree for calculating the similarity between the search target image designated by the search designation unit and the color natural image stored in the image storage unit by paying attention to the position, area, shape and color of the figure on the screen The image search device according to claim 1, further comprising a calculation unit. 3. The image search apparatus according to claim 2, wherein the search means sequentially searches the color natural images stored in the image storage means from an image having a high similarity to a search target. 4. A color natural image and an index are added to each color natural image by using a combination of basic figures and accumulated, and at the time of search, a search is designated by the combination of the basic figures, and the index is stored. An image retrieval method characterized by retrieving and presenting a color natural image that is closest to the content designated by the search from the stored color natural images. 5. When a search is designated, the search is performed by focusing on the position, area, shape, and color of each basic figure on the screen between the combination of the designated basic figures and the index. The image search method according to claim 4, wherein the similarity between the target and the color natural is calculated, and based on the calculation result, the accumulated color natural images are sequentially searched and presented in descending order of the similarity.