JPH03194655A - Method for accumulating and retrieving similar picture - Google Patents

Abstract

PURPOSE:To attain retrieval which is adjusted to the content of a picture and which is adjusted to the eyesight and the sense of a man by obtaining information which is impervious to express by means of a language from an exemplified picture and area-specifying an accumulation picture which is not apparently similar to the exemplified picture on a characteristic space based on an evaluated scale obtained from plural characteristics and removing the picture. CONSTITUTION:Elements showing the characteristics of respective pictures from plural inputted pictures, a characteristic parameter, a similarity, the similar measure of the whole accumulation picture 14 and an index file 27 are stored. The exemplified picture 34 is inputted from a terminal 3, the picture content to be retrieved is grasped, and space which is effective in the characteristic space constituted by the picture content is precisely converged. Thus, the picture which is not similar to the exemplified picture 34, namely, the picture of a range that cannot be permitted as a candidate picture is specified and removed. Consequently, the picture similar to the content of the exemplified picture 34 is presented from a picture data base to the terminal 3 according to the order of the candidates.

Description

DETAILED DESCRIPTION OF THE INVENTION (Technical field to which the invention pertains) The present invention relates to a method for storing similar images from a terminal into an image database, and a method for searching for similar images from an image database based on example images input from the terminal. .

(Conventional Technique Ill) Conventionally, in systems that search for necessary images from an image database based on a request from a terminal, keywords such as words and symbols are used as search keys, and one image database is determined at the time of construction. It was composed of keyword categories.

(Problem to be Solved by the Invention) However, when using such a keyword-based classification method, when the scale of a single image database becomes large and the content of expression becomes complex, the classification prepared in advance in the image database cannot be used to classify the images sufficiently. It is often stored in a location that does not match the content.

For this reason, it is often not possible to search for the desired image in one go using keywords such as words and symbols, and it is necessary to reconsider the keywords such as words and symbols that are thought to contain the desired image, or to search multiple times. First, there is a problem in that the search takes time and it is not possible to perform a search based on the content of the image that matches the user's intention.

(Objective of the Invention) An object of the present invention is to solve the above-mentioned conventional problems and provide an efficient method for storing and retrieving similar images.

(Means for Solving the Problems) In order to achieve the above object, the present invention provides a method for storing similar images according to claim (1), which extracts a plurality of elements representing the characteristics of each image from a plurality of input images. However, in order to ensure consistency between the process of calculating the properties commonly obtained from the plurality of elements representing the characteristics of the plurality of stored images as a plurality of feature parameters and the subjective data representing the similarity measure of the stored images. A process of weighting each feature parameter to determine the degree of similarity of each stored image and constructing a similarity measure for the entire stored image, and elements and feature parameters obtained in each of the above steps,
The present invention is characterized by a process of creating an index file based on the degree of similarity, and storing the elements, feature parameters, degree of similarity, similarity measures of the entire stored images, and the index file.

In addition, I [similar image search method according to claim (2)] uses a similarity measure created by the similar image accumulation method to select a plurality of elements representing characteristics of the accumulated image from exemplary images input at the time of search. a step of extracting and calculating a plurality of feature parameters in the same manner as at the time of storage according to claim (1); and a step of applying a similarity measure of the entire stored image to the example image to obtain a degree of similarity as an estimated value. In the feature space characterized by the plurality of feature parameters obtained in the above process, a range as a candidate of the example image is specified according to the position in the feature space determined from the plurality of feature parameters of the example image, their weights, and similarities. and a step of determining the ranking of candidates according to the degree of similarity determined by the above process and the specified range, and outputting and displaying the ranking based on the ranking.

(Function) The present invention uses the above-mentioned means to obtain elements and characteristic parameters representing the characteristics of each image from a plurality of input images.

Store the degree of similarity, the similarity measure of all accumulated images, and an index file, input example images from the terminal, understand the image content you want to search, and accurately find an effective space in the feature space composed of the image content. By narrowing down the search to images that are not similar to the example images, that is, images that are not acceptable as candidate images, are identified and eliminated.

By presenting images similar to the example image content to the terminal from the image database according to the candidate ranking, it is possible to realize a search that matches human vision, sense, etc. in accordance with the image content.

The method of the present invention will be explained below with reference to drawings, but the database targeted by the present invention does not include everything in an encyclopedic manner, but is a specialized database, such as trademarks, designs, medical products, etc. However, the database for the method of the present invention is not limited to specific ones.

(Embodiment) FIG. 1 is a block diagram of an embodiment of an apparatus for carrying out the method of the present invention, in which 1 is an image database creation apparatus, 2 is an image database system, and 3 is a terminal, 11 of the database creation device l is an input section;
12 is a control unit, 13 is a display unit, 14 is an image for storage, and 21 of the 1-image database system 2 is a control unit;
22 is an input data storage section, 23 is a feature extraction section, 24 is a similarity calculation section, 25 is a similarity determination section, 26 is an index creation section, 27 is an index file, 28 is an image feature storage section, and 29 is an image data storage section. 210 is an area determination section, and 211 is an output data storage section. Also, 31 of terminal 3
32 is an input section, 32 is a control section, 33 is a display section, and 34 is an input example image.

FIG. 2 shows an example of region identification means (3
(a) is an example in which the quadrant to which the input example image belongs is the selection area, and the area where X n > O (n =
3), (b) is an example in which the adjacent region of (a) is added to (a), where X n > O and the adjacent region (n+1 quadrants).

(C) is an example in which the selection area is the quadrant to which the input example image belongs and the quadrant with the opposite sign of each axis.
> Area other than O (2”-1 quadrants), (d) is an example where the inside of a hypersphere with radius r is selected as the selected area, (e) is a cross section of a circle with radius r at the position of the input example image. An example is shown in which the selection area is the inside of a cone whose apex is the origin.
1 indicates the feature vector of the input example image.

First, when storing images, the image database creator inputs the storage image 14 into the input section of the image database creation device 1]1
Enter. The control unit 12 is an image database system 2
, and sends the input image from the input section 11 to the control section 2.
1 to the input data storage section 22. After that, the control unit 21 activates the feature extraction unit 23. This feature extraction unit 23 extracts elements ai (i=1.2.3...
...; i indicates the identifier of the image feature), and when the extraction is completed, the control unit 21 is notified of this fact.

Upon receiving the notification, the control unit 21 activates the similarity calculation unit 24. This similarity calculation unit 24 calculates the properties that a plurality of accumulated images that have been accumulated so far have in common into elements ai representing image characteristics.
Calculated as multiple feature parameters based on As a weighting method, a means for associating a physical quantity obtained from one image with a human psychological quantity is used.

Here, there are various means for making the above-mentioned correspondence, but for example, each feature parameter is weighted by statistical linear means such as multivariate analysis, or non-linear means such as a neural network, using human subjective data as training data.

This corresponds to determining the position of each accumulated image in a psychological feature space in which each dimension is defined by each feature parameter.

By establishing the correspondence between physical quantities and psychological quantities as described above, the weight of feature parameters that reflect subjective data becomes large, and the weight of feature parameters that do not reflect subjective data becomes small.

In this way, the feature parameters are weighted, a similarity measure is constructed, and the degree of similarity of each stored image is determined. When this series of operations is completed, the similarity calculation unit 24 notifies the control unit 21 to that effect.

When the control unit 21 receives the notification, it controls the index creation unit 26.
Start. This index creation unit 26 generates elements at representing characteristics of images obtained from the storage images 14.

An index for the image feature storage section 28 and image data storage section 29 is created based on the calculated feature parameters and similarity, and stored in the index file 27. When the storage is completed, the control section 21 is notified of this fact.

The control unit 21 follows the contents of the index file 27,
The image data, an element ai representing its characteristics, a plurality of characteristic parameters, a degree of similarity, and a similarity measure are stored in the image data storage section 29 and the image feature storage section 28, respectively. At the same time, the control unit 21 displays the index file 2 on the display unit 13 via the control unit 12 of the image database creation device 1.
7, the contents of the image feature storage section 28, and the image data.

The database creator looks at the contents of the display section 13 and issues the next instruction (confirmation, re-input, cancellation, etc.). For example, when a confirmation instruction is input into the input section 11, the control section 12 sends the information to the control section 21 of the image database 2. Notify. When the control unit 21 receives the confirmation instruction, the series of processing ends.

Note that the feature parameters are elements a representing features of multiple images.
Calculated by combinations of i. However, this does not prevent the element ai representing the image feature from being directly used as the feature parameter of the present invention. This also applies to the next search.

Next, searching for similar images will be explained. First, during a search, the user inputs the input example image 34 into the input unit 31 of the terminal 3. The control unit 32 transfers the input example image 34 to the input data storage unit 22 via the control unit 21 of the image database system 2. At the same time, the control unit 21 activates the feature extraction unit 23. The feature extraction unit 23 extracts the element ai (i=1.2.3...) representing the feature of the image from the input data storage unit 22 via the control unit 21 in the same manner as when the extraction is completed. The controller 21 is notified of this fact.

When the control unit 21 receives the notification, it activates the similarity calculation unit 24. The similarity calculation unit 24 refers to the index file 27 via the control unit 21 and acquires a similarity measure from the image feature storage unit 28 based on the element ai representing the image feature obtained from the input example image 34. .

Element a representing the feature of the image acquired from the input example image 34
A feature parameter is calculated from i and weighted, and an evaluation scale (similarity degree) Sj (j=1, 2, 3...j is the similarity of each candidate image) indicating the similarity state between the input example image 34 and each candidate image data Calculate the difference.

Here, various types of equations can be applied as the similarity sj, but for example, weights suitable for the input example image 34 and each candidate image -jiF = 1 t 2 e 3...; i = 1.2° 3 ...) (determined by multivariate analysis method, learning, etc.), and sj=Sonojlal+mj2a2+-(
j = 1.

2.3...). The degree of similarity sj is expressed numerically from most similar to least similar (for example, from 0 to 1), and is used to rank the degree of similarity.

When the similarity calculation is completed, the controller 21 is notified.

Upon receiving the notification, the control unit 21 arranges the input example image 34 in a feature space characterized by each calculated feature parameter of the input example image 34. By dividing the feature space into regions according to the position where the input example image 34 belongs in the feature space using a space dividing means such as a quadrant, hypersphere, hyperplane, etc. or a combination thereof, images in a range that is not acceptable as a candidate image are obtained. The area determining unit 210 is activated in order to exclude the area to which the calculated similarity belongs and identify the area where the calculated similarity is valid.

The region determining unit 210 determines an effective region on an n-dimensional vector space whose dimensions are elements ai representing image characteristics using region specifying means as shown in the example of FIG.

For example, in a feature space characterized by feature parameters calculated based on elements ai representing the characteristics of the input example image 34 as shown in FIG. shall be.

Alternatively, as shown in FIG. 2(b), an area including the area of FIG. 2(a) and its adjacent quadrants is set as an effective area.

Furthermore, as shown in FIG. 2(d), if the norm r of the vectors constituted by the input example image 34 is less than or equal to a certain threshold th, the area within the hypersphere with radius r is also considered to be a valid area. Use area determination means such as

A plurality of the above-mentioned area determining means can be used in combination. The area determination unit 210 notifies the control unit 21 when the area determination is completed.

When the control unit 21 receives the notification, it activates the similarity determination unit 25. The similarity determination unit 25 determines that the image is similar to the input example image 34 based on the result calculated by the similarity calculation unit 24 within the valid region on the n-dimensional vector space determined by the region determination unit 210. Determine the candidate ranking of candidate images.

According to the determined candidate order, the index file 27 is referred to via the control section 21, candidate images are sequentially taken out from the image data storage section 29, arranged in an output format, and then transferred to the output data storage section 211. When the transfer is completed, the control unit 21 outputs the output data according to the request from the terminal 3, for example, display only the first candidate, display up to the nth candidate (n=1.2, 3, etc.), display all candidates, etc. Accumulation section 211
The necessary image data is extracted from the terminal 3 and transferred to the control unit 32 of the terminal 3. When the control section 32 receives the image data, it displays it on the display section 33.

After the above processing, the user finds the desired image among the candidate images, and by changing the input example image 34, the user can extract the desired image and similar images.

Note that the input example image 34 may include not only images but also words and symbols to limit the information.

(Effects of the Invention) As explained above, the method of the present invention uses example images when searching image data from a terminal, acquires information that is difficult to express in words from the example images, and evaluates the information obtained from multiple characteristics. Since stored images that are clearly not similar to the example image are eliminated based on the scale by identifying regions in the feature space, the degree of similarity can be determined within a limited group of images, improving accuracy. It has the advantage of being able to quantitatively classify all images using evaluation scales, and realizing searches that suit human vision and senses in accordance with the content of the images.

[Brief explanation of drawings]

FIG. 1 is a block configuration diagram of an embodiment of an apparatus for carrying out the method of the present invention, and FIG. 2 is an example of area identification means in an n-dimensional vector space in which each dimension is each feature extracted from an input example image. FIG. DESCRIPTION OF SYMBOLS 1... Image database creation device, 2... Image database system, 3... Terminal, 11... Input section, 12... Control section, 13... Display section. 14... Image for storage, 21... Control unit, 22...
- Input data storage section, 23... Feature extraction section. 24... Similarity calculation section, 25... Similarity determination section. 26... Index creation unit, 27... Index file, 28... Image feature storage unit, 29...
・Image data storage unit, 210... area determination unit, 2
11... Output data storage section, 31... Input section,
32...Control unit, 33...Display unit. 34... Input example image.

Claims (2)

[Claims] (1) Extract multiple elements representing the characteristics of each image from the input multiple images, and calculate properties commonly obtained from the multiple elements representing the characteristics of these multiple stored images as multiple feature parameters. and a step of determining the degree of similarity of each stored image by weighting each feature parameter to ensure consistency with subjective data representing a similarity measure of the stored images, and configuring a similarity measure of the entire stored images; The process includes the steps of creating an index file based on the elements, feature parameters, and degrees of similarity obtained in each process, and storing the elements, feature parameters, degrees of similarity, similarity measure of the entire accumulated image, and the index file. A method for accumulating similar images as features. (2) A plurality of elements representing the characteristics of the stored image are extracted from the example image input at the time of search using the similarity measure created in claim (1), in the same manner as at the time of storage according to claim (1). a step of calculating a plurality of feature parameters; a step of applying a similarity measure of all stored images to the example image to obtain a degree of similarity as an estimated value; and a step of calculating a plurality of feature parameters obtained in the above process as each dimension. A process of specifying a range as a candidate of an example image according to a position in a feature space determined from a plurality of feature parameters of the example image, their weights, and degrees of similarity in the feature space, and a degree of similarity and identification determined by said process. A method for searching for similar images, comprising the steps of: determining a candidate ranking according to the determined range; and outputting and displaying the ranking based on the ranking.