JPH0573625A - Similar scale constitution processing method - Google Patents

Abstract

PURPOSE:To execute retrieval matching with the eyesight and the sense of man by providing a psychological non-similarity or similarity expression part, a feature extraction part, a mapping generation part and an evaluation part for the goodness of fit. CONSTITUTION:The psychological non-similarity or similarity expression part 101 expresses a picture by a multidimensional scale showing non-similarity or similarity by using data showing psychological non-similarity or similarity between the pictures. The feature extraction part 102 extracts the elements of the plural features. The mapping generation part 103 obtains mapping from the elements of the plural features extracted from the picture to the multidimensional scale showing non-similarity or similarity. The evaluation part for the conformity degree 104 evaluates the conformity degree of data between data showing psychological non-similarity or similarity between the pictures and mapping by a square error, and it decides the appropriate number of dimensions of the multidimensional scale expressing the picture. Thus, the similar scale based on the psychological similar relation can be constituted and retrieval which agrees with the eyesight and the sense of man can be executed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a similar scale composition processing method for searching a desired or similar image by giving an example image from a database in which images are accumulated.

[0002]

2. Description of the Related Art Conventionally, when an example image is given to a database in which images are stored and a desired or similar image is searched from the database, the physical feature amount acquired from the example image is stored. The comparison was performed by using a similarity measure composed of the physical feature amount acquired in advance from the acquired image, and the selected image was presented to the user.

[0003]

However, the search result does not correspond to the human sense because the similarity measure that a person is satisfied with cannot be constructed only from the physical feature amount acquired from the image. There was a problem.

It is an object of the present invention to solve the above-mentioned conventional problems and to make it possible to perform a search that matches a person's vision, sense, etc., by performing a search using an exemplary image.

[0005]

FIG. 1 is a block diagram showing the principle of the present invention. In the figure, reference numeral 101 is a psychological dissimilarity or similarity expression unit, 102 is a feature extraction unit, 103 is a mapping generation unit,
Reference numeral 104 represents a fitness evaluation unit.

Psychological dissimilarity or similarity expression unit 101
Expresses images on a multidimensional scale representing dissimilarity or similarity using data representing psychological dissimilarity or similarity between images.

The feature extraction unit 102 extracts a plurality of feature elements from the image. The mapping generation unit 103 obtains a mapping from a plurality of feature elements extracted from an image to the above-described multi-dimensional scale indicating dissimilarity or similarity.

The goodness-of-fit evaluation unit 104 evaluates the goodness-of-fit of the data representing the psychological dissimilarity or similarity between the aforementioned images and the above-mentioned mapping by the square error, and is a multidimensional scale expressing the images. Determine the optimal dimensionality of.

[0009]

[Operation] An image is expressed in a multidimensional scale by psychological judgment, and on the other hand, a feature element is extracted from each image. Then, the two are associated with each other, and the square error means determines the optimum number of dimensions.

[0010]

FIG. 2 is a block diagram showing an embodiment of the present invention.
Is a subjective evaluation data input unit, 2 is a subjective evaluation data analysis unit,
3 is an image data input unit, 4 is a feature extraction unit, 5 is a similarity measure configuration unit, 6 is an image storage unit, 7 is an index file,
Reference numeral 8 is a similarity measure storage unit, 9 is a similarity calculation unit, 10 is a search unit, and 11 is a display unit.

FIG. 3 is a flow chart of the similarity measure processing in the embodiment. FIG. 4 is a flowchart of the search process in the embodiment. FIG. 5 and FIG. 6 together constitute one figure, and the accumulated images (butterfly image 2
It is an example of subjective evaluation data showing the (non) similarity between five examples).

FIG. 7 shows an example of coordinate values when a stored image (an example of 25 images of butterflies) which constitutes a search system is represented in a two-dimensional Euclidean space. FIG. 8 is a conceptual diagram showing a mapping f from the physical space to the psychological space.

FIG. 9 is an example of a diagram showing the relationship between the stress and the square error of a butterfly and the number of dimensions. First, in order to investigate the psychological similarity between images of the accumulated images (n images) that make up the search system selected from the accumulated images in the image database, a subjective evaluation by a plurality of people was performed, and each accumulated image was stored. Get the similarity between images. The accumulated images forming the search system used at this time are the same as all accumulated images or a subset thereof. It is desirable that the accumulated images that make up this search system be a set that reflects the contents of the population of this image database. As an example, there is a method of performing grouping in which all stored images are similar to each other and then selecting an average image of each group. These procedures correspond to “perform subjective evaluation 21 on accumulated images that constitute the search system” in FIG. There are various methods for subjective evaluation, and for example, a method such as a multiple ratio determination method or a ranking method can be used.

The multiple ratio determination method is a method of measuring the dissimilarity to the reference image by a physical distance with reference to one image among the accumulated images that constitute the search system, and is a physical equivalent to the dissimilarity. Put another image at a certain distance. The same procedure is repeated until all images become reference images.

In the ranking method, one image among the stored images forming the search system is used as a reference, one image that is most similar to the reference image is selected, the next image is removed, and then the most similar to the reference image. The entire ranking is performed by an iterative procedure of selecting one image. The same procedure is repeated until all images become reference images.

5 and 6 show examples of subjective evaluation data representing the (non) similarity between the obtained accumulated images. At this time, the dissimilarity with oneself is considered to be zero. Subjective evaluation data indicating the (non) similarity between the obtained accumulated images is input from the subjective evaluation data input unit 1 shown in FIG. Person's (non)
Regarding the similarity judgment, the subjective evaluation data analysis unit 2 analyzes the input subjective evaluation data. If a person views the (non) similarity of images on a p-dimensional scale, it is desirable that the subjective evaluation data can be described from a p-dimensional viewpoint. Therefore, the dimensions that can be configured without breaking the order relation of the subjective evaluation data are examined in order from the lowest dimension, and the optimum dimension that does not distort the (non) similarity between the accumulations as much as possible is adopted. At this time, if n accumulated images are represented as n points in a p-dimensional Euclidean space (coordinate values are represented by an n × p matrix X), the Young-Householder transformation P is determined to be satisfied. There is a need. The element is the square of the Euclidean distance between n accumulated images.

[0017]

[Equation 1]

Is expressed as follows.

[0019]

[Equation 2]

Here,

[0021]

[Equation 3]

Is an n-dimensional vector whose elements are all 1,

[0023]

[Equation 4]

Is

[0025]

[Equation 5]

It is a diagonal matrix whose diagonal elements are diagonal elements of. this

[0027]

[Equation 6]

Young-Householder transformation using

[0029]

[Equation 7]

Is

[0031]

[Equation 8]

It is represented by here,

[0033]

[Equation 9]

Is

[0035]

[Equation 10]

It is defined as this

[0037]

[Equation 11]

Has the following properties:

[0039]

[Equation 12]

At this time,

[0041]

[Equation 13]

In the sense of least squares approximation to

[0043]

[Equation 14]

The method of obtaining This procedure is a multidimensional scale approach and corresponds to the process of "analyze subjective evaluation data and decompose into a multidimensional scale 22 representing psychological factors" in FIG. Further, here, the value in each scale obtained by dividing the relationship of the obtained subjective evaluation data into multiple dimensions corresponds to the coordinate value of each axis in the multidimensional space, where each scale is an axis in the multidimensional space. .. FIG. 7 shows an example of coordinate values when the accumulated images (25 images of butterflies) forming the search system are represented in a two-dimensional Euclidean space. There is a scale called stress as a scale showing how well these coordinate values are applied. It is defined by the following equation.

[0045]

[Equation 15]

Where s is stress,

[0047]

[Equation 16]

Is a value determined so as to minimize s under the condition that the degree of dissimilarity between images i and j obtained by a subjective evaluation experiment and the like has the same order, and d _ij is p It is the degree of dissimilarity between the image i and the image j when expressed in dimensions. It is better to have less stress, but how many or less is sufficient is an arbitrary matter. Generally, 10% or less is acceptable, and 5% or less is good.

On the other hand, all stored images are stored in the image data input unit 3
Is input to the system using an input device such as a camera or a scanner, and the feature extraction unit 4 acquires various physical feature amounts for each image. This procedure corresponds to "feature extraction 23 from accumulated image forming search system" in FIG. Note that the physical feature amount is a feature amount acquired from an image (for example,
Mesh features, Fourier features, etc.), and there is no limit to the type and number. At this time, since the mapping is obtained by a statistical linear method or a non-linear method such as a neural network,
The sum q of physical features and the number n of stored images are at least q.
The relationship of ≦ n is necessary, and it is desirable that n is as large as possible compared to q.

Furthermore, the mapping f from the physical space represented by the characteristic amount acquired from the accumulated image to the multidimensional psychological space obtained by analyzing the subjective evaluation data is statistically calculated in the similarity measure construction unit 5. It is obtained using a linear method or a non-linear method such as a neural network. This mapping f is defined by one or more. This mapping f makes it possible to construct a similarity measure corresponding to human senses. FIG. 8 shows a conceptual diagram showing this mapping f from the physical space to the psychological space. For example, the mapping f can be obtained by performing multiple regression analysis using the coordinate value of each axis obtained by the multidimensional scaling method as the objective variable and the feature amount obtained from the image as the explanatory variable. At this time, the degree of fit of the mapping f is evaluated by the square error for each dimension to be expressed, and the stress value has a threshold value t.
If it can be considered that it is converged below h (it is asymptotic to a certain fixed value), the dimension that is more than the number of dimensions at that time and has the smallest square error is determined as the optimum dimension.

[0051]

[Equation 17]

Here,

[0053]

[Equation 18]

Is an estimated distance between the image i and the image j in the p-dimensional psychological space calculated from the physical feature quantity. FIG. 9 shows an example of a diagram showing the relationship between the stress and the square error of the butterfly and the number of dimensions. In this example, 7 dimensions are the optimal dimensions. It can be said that this 7-dimensional multidimensional psychological space reflects human subjectivity.

Thereafter, for the optimum dimension, the stored image is stored in the image storage unit 6, the coordinate values on each axis of the multidimensional psychological space are stored in the index file 7 in correspondence with each stored image,
The function representing the mapping created by the mapping f and the values of various parameters are stored in the similarity measure storage unit 8, respectively. Further, accumulated images other than accumulated images constituting the search system and additional images are input to the system from the image data input unit 3 using an input device such as a camera or a scanner, and the feature extraction unit 4 extracts each image. Then, the same various physical feature quantities as when the similar scale is constructed are acquired. Next, the similarity calculator 9
At, the mapping value of each axis in the multidimensional psychological space is calculated from the acquired physical feature amount by using the similarity measure extracted from the similarity measure accumulation unit 8. The image is stored in the image storage unit 6, and the coordinate value on each axis of the multidimensional psychological space is stored in the index file 7 corresponding to each stored image.

These procedures correspond to "24 which calculates the mapping from the image feature elements to the multidimensional psychological space and constitutes the similarity measure" in FIG. At the time of search, the user inputs an example image from the image data input unit 3 to the system using an input device such as a camera or a scanner, and the feature extraction unit 4
The various physical characteristics that are the same as when the similar scale is constructed are acquired with. This procedure corresponds to “presentation image 31” and “feature extraction 32 from example image” in FIG.

Next, in the similarity calculation unit 9, the mapping value of each axis in the multidimensional psychological space of the example image is calculated by using the similarity measure extracted from the similarity measure accumulation unit 8 from the acquired physical feature amount. To calculate. This procedure corresponds to “calculation 33 of similarity” in FIG. Further, by the minimum distance identification method, the search unit 10 calculates the distance from the coordinate value of each axis in the multidimensional space of the example image to the position in the multidimensional space of the accumulated image obtained from the index file 7, The image storage unit 6 searches for a plurality of candidate images in ascending order of the distance from the example image and displays them on the display unit 11. In this way, images with a short distance in the multidimensional psychological space represent that they are psychologically similar. This procedure corresponds to “showing to the user 34 by comparing the accumulated images and ordering from the similar images” in FIG.

The user can thus obtain a desired or similar image. Further, even if the image is not displayed on the screen, it can be viewed in an order similar to the example image by designating the display screen.

[0059]

As described above, according to the present invention, the subjective evaluation is performed on the images accumulated in the image database, and the psychological similarity between the images is acquired. There is an advantage that a similarity measure based on a psychological similarity, which cannot be obtained by a similarity measure based only on the characteristic features, can be configured, and a similarity search corresponding to a human sense can be realized.

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of the present invention.

FIG. 2 is a configuration diagram showing an embodiment of the present invention.

FIG. 3 is a flowchart of similarity measure processing in the embodiment.

FIG. 4 is a flow chart of search processing in the embodiment.

FIG. 5 is a part of an example of subjective evaluation data representing a (non) similarity between accumulated images (an example of 25 images of a butterfly) forming a search system.

FIG. 6 is a part of an example of subjective evaluation data representing a degree of (non) similarity between accumulated images (an example of 25 images of a butterfly) forming a search system.

FIG. 7 is an example of coordinate values when an accumulated image (an example of 25 images of a butterfly) that constitutes a search system is represented in a two-dimensional Euclidean space.

FIG. 8 is a conceptual diagram showing a mapping f from a physical space to a psychological space.

FIG. 9 is an example of a diagram showing the relationship between the stress and the square error of a butterfly and the number of dimensions.

[Explanation of symbols]

 101 Psychological dissimilarity or similarity expression unit 102 Feature extraction unit 103 Mapping generation unit 104 Fitness evaluation unit 1 Subjective evaluation data input unit 2 Subjective evaluation data analysis unit 3 Image data input unit 4 Feature extraction unit 5 Similarity scale configuration unit 6 Image storage unit 7 Index file 8 Similarity scale storage unit 9 Similarity degree calculation unit 10 Search unit 11 Display unit

Claims (2)

[Claims] 1. A means for expressing an image on a multidimensional scale representing dissimilarity or similarity using data representing psychological dissimilarity or similarity between images, and a plurality of feature elements from the image. Means for extracting, means for obtaining a mapping from a plurality of feature elements extracted from an image to the multidimensional scale representing the dissimilarity or similarity, and data representing the psychological dissimilarity or similarity between the images A means for determining the optimal dimensionality of a multidimensional scale expressing an image by evaluating the degree of fit of the data with the mapping by a square error, and the parameter of the mapping and the mapping value of the image at the optimal dimensionality. And a means for storing the original image in association with the original image. 2. A means for extracting elements of a plurality of features described above from an image input for retrieval using the similarity measure created in the preceding paragraph, and a multidimensional image using the parameters of the mapping described in the preceding paragraph. A means for obtaining a mapping value, a means for presenting the mapping value and the mapping value of the image accumulated in the previous section in the multidimensional scale space using the minimum distance identification method in order from the one with the shortest distance, The similarity scale construction processing method according to claim 1, further comprising: