JPH11345342A - Method and device for retrieving time series image, and recording medium having recorded time series image retrieval program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make an image sequence matching with user's evaluation of similarity highly precisely retrievable by calculating the weighted sum of distances as non- similarity between image sequences and outputting the image sequence small in non- similarity among image sequences stored in a database as a retrieval result. SOLUTION: This device consists of a retrieving part 207 which uses a locus on proper space of an image sequence given as a key and an auxiliary image characteristic and retrieves the image sequence from an intra-proper space image sequence storing part 206, a characteristic quantity summation adjusting part 208 which evaluates the similarity of an image sequence retrieved according to the scaling of a distance between loci on the proper space based on evaluation reference given from a similarity evaluation offering part 104 about the selected image sequence in an image sequence storing part 201 and adjusts summation at the time of performing non-similarity calculation so as to coincide with the evaluation reference and a characteristic quantity summation storing means 209 which stores adjusted summation. An outputting part 300 outputs an image sequence of a retrieval result which is outputted from a processing part 200.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a time series for searching a time series image similar to an arbitrary time series image input as a key from a large amount of time series images stored in a database. The present invention relates to an image search method and apparatus.

[0002]

2. Description of the Related Art A technique using an eigenspace has been known as a technique for retrieving a time-series image. As an example,
Literature: Yasushi Fujimoto, Hidehiko Iwasa, Naokazu Yokoya, Haruo Takemura “Moving Image Retrieval Based on Similarity of Trajectories in Eigenspace”, IEICE Technical Report, PRMU96-110, pp. 139-143. 49
-56, 1996. Is mentioned. In this method, for each frame of a time-series image, the gray value of a pixel in the image is expressed as a one-dimensional vector, and this vector calculated for each frame of the image sequence forming the time-series image database is defined as a column vector. A matrix is created, a covariance matrix is obtained for a matrix obtained by subtracting an average value of the feature vectors of all image sequences from the matrix, and an eigenvector based on an eigenvector corresponding to a large eigenvalue obtained by solving an eigenvalue problem of the matrix is obtained. A space is created, and the trajectory of each image sequence projected on the eigenspace is calculated. The distance between the trajectory in the eigenspace corresponding to the image sequence of the retrieval key and the trajectory in the eigenspace of the image sequence stored in the database is used as a measure of dissimilarity to search for an image sequence with a small distance. As a result.

By expressing an image sequence as a trajectory in an eigenspace, an image pattern that changes over time can be easily expressed.
Further, since the search can be executed in the eigenspace having a smaller dimension than the number of dimensions of the original image feature, there is an advantage that the time required for the search is short.

[0004]

However, in the above-mentioned prior art, the distance between the trajectories in the eigenspace does not always match the evaluation of the similarity of the user, and the distance between the trajectories is the same. Also in this case, there is a problem that the degree of similarity recognized by the user changes depending on the image sequence.
Further, there is a problem that only spatial distribution features such as grayscale values of an image used for forming an eigenspace are insufficient as feature amounts for search.

Accordingly, it is an object of the present invention to provide a time-series image search method and apparatus capable of performing a high-accuracy search for an image series that matches the evaluation of similarity of a user.

[0006]

A time-series image retrieval method according to the present invention provides a time-series image sequence for retrieving a similar time-series image sequence portion from a time-series image database by giving a time-series image sequence as a key. A search method, inputting a time-series image sequence, storing the image sequence in a time-series image database, and calculating a spatial distribution feature amount vector of an image at each unit time from the input image sequence;
Calculating an eigenspace of the spatial distribution feature from a sequence of spatial distribution feature vectors calculated from a set of image sequences in a time-series image database; and calculating a trajectory when projecting the image sequence on the eigenspace. Calculating and storing them in a database; calculating an auxiliary feature amount per unit time from a set of image sequences; storing the auxiliary feature amount in a database; Inputting an image sequence of a sequence, calculating the trajectory of the input image sequence when projected onto the eigenspace calculated earlier, and storing the trajectory and the image stored in the database. Calculating a distance from the trajectory of the sequence, calculating an auxiliary feature amount of the image sequence input as a key for the search, and Calculating the distance between the quantity and the auxiliary feature of the image sequence stored in the database; and calculating the weighted sum of the distance between the trajectories on the eigenspace and the distance between the auxiliary features. Calculated as dissimilarity between
Outputting an image sequence having a low degree of dissimilarity from the image sequences stored in the database as a search result.

According to the present invention, a time-varying image pattern can be easily represented by expressing an image sequence as a trajectory in the eigenspace, and the eigenspace in the eigenspace having a smaller dimension than the original image feature can be expressed. Since the search can be executed by using, there is an advantage that the time required for the search is short.

Further, in the present invention, an auxiliary feature is introduced in addition to a similarity measure of a distance between trajectories in an eigenspace constituted by spatial distribution features of an image, and a plurality of features are introduced. The sum weighted to the distance is used as a measure of dissimilarity between the image sequences, and a search is performed. Therefore, as compared with the conventional method using the eigenspace of the spatial distribution feature of the image, a wide range of image feature amounts can be used, and a highly accurate search that matches the user's request can be performed.

According to the embodiment of the present invention, the step of calculating the spatial distribution feature amount vector of the image from the image sequence for each unit time comprises the step of arranging the grayscale values of the pixels on the image plane in order. Calculating as a vector.

According to the embodiment of the present invention, the step of calculating the spatial distribution feature amount vector of the image from the image sequence for each unit time comprises dividing the image plane into meshes and calculating the grayscale value of the image included in each mesh. Calculating a mean at each time interval and obtaining a spatial distribution feature vector as a one-dimensional feature vector at each time.

According to the embodiment of the present invention, the step of calculating the spatial distribution feature vector of the image from the image sequence for each unit time includes obtaining the speed vector at each time, and calculating the speed component of each point on the image plane. Are arranged in a line to obtain a spatial distribution feature vector as a one-dimensional feature vector.

According to an embodiment of the present invention, the step of calculating an auxiliary feature amount per unit time from a set of image sequences includes a texture feature or a color feature of an image density distribution.
Alternatively, the method includes a step of calculating an auxiliary feature amount as a one-dimensional vector for each unit time, with a part or all of the shape features of the image pattern as a feature amount.

According to the embodiment of the present invention, the step of calculating the eigenspace includes forming a matrix by combining feature amounts of one-dimensional vectors obtained at certain time intervals with respect to the image sequence for the entire image sequence. Obtaining a covariance matrix for a matrix obtained by subtracting the average value of the feature amount vectors of all image sequences from the matrix, and obtaining an eigenspace based on an eigenvector corresponding to a large eigenvalue obtained by solving an eigenvalue problem of the matrix. Including.

According to an embodiment of the present invention, the step of calculating a trajectory when a sequence of spatial distribution feature amounts corresponding to an image sequence is projected onto an eigenspace includes: The method includes a step of sequentially converting the feature amounts of the one-dimensional vector obtained for each interval into points on the eigenspace, and obtaining a sequence of the points as a trajectory on the eigenspace for the image sequence.

According to the embodiment of the present invention, the step of calculating the distance between the trajectory in the eigenspace of the image sequence input as a search key and the trajectory of the image sequence stored in the database, For each trajectory, there is included a step of calculating the distance between the point sequences on the trajectory corresponding to the feature amount vector at each time, by observing the time order of each point and summing them.

According to the embodiment of the present invention, the dissimilarity between the image sequence input as a search key and the image sequence stored in the database is determined by the distance between the trajectories in the eigenspace and the auxiliary value. The step of calculating as a weighted sum of the distances between the various feature amounts includes a step of adjusting in advance the magnitude of the weight multiplied by the distance so that the search result matches the evaluation of the user.

According to the embodiment of the present invention, the weight of the distance between individual feature values when calculating the dissimilarity between the image sequence input as a search key and the image sequence stored in the database is calculated. Is a step in which the user selects a representative image sequence from the image sequences stored in the database, and for each, searches for an image sequence in which the distance between the trajectories in the eigenspace is short. Whether or not the searched image sequence is similar to the image sequence of the search key is evaluated by an evaluation scale given by the user, and the individual feature amounts are set so that the degree of dissimilarity to the similar image sequence is reduced. Determining the weighting of the distance between them.

According to the embodiment of the present invention, for a representative image sequence selected by the user, an image sequence having a short distance between trajectories in the eigenspace is searched, and whether or not the image sequence is similar is evaluated. Given, the step of determining the weighting of the distance of the individual feature amount so that the dissimilarity to a similar image sequence is reduced, for each representative image sequence, a set of searched image sequences, The image sequence is classified into two classes, an image sequence evaluated to be similar to the representative image sequence and an image sequence evaluated to be dissimilar, so that the variation in the distance between the feature amounts in each class is reduced. And a step of calculating the weight of the distance between the feature values by discriminant analysis so that the variation between the classes becomes large.

According to the embodiment of the present invention, for the representative image sequence selected by the user, the obtained weights for the dissimilarity calculation are respectively assigned to the individual image sequences corresponding to the respective representative image sequences. And storing the information in association with a position on a trajectory in space.

According to the embodiment of the present invention, the dissimilarity between the image sequence input as a search key and the image sequence stored in the database is determined by the distance between the trajectories in the eigenspace and the auxiliary value. Calculating the weighted sum of the distances between the characteristic features corresponds to the eigenspace corresponding to the representative image sequence previously selected by the user, which is closest to the trajectory on the eigenspace corresponding to the image sequence of the search key. Searching for the upper trajectory and calculating the dissimilarity between the search key and the image sequence in the database using the weighted sum calculated for the representative image sequence.

Since the weighting of the distance between a plurality of feature amounts can be adjusted to match the similarity measure of the user, the similarity measure for each user can be reflected in the search.

Further, the weighting of the distance between a plurality of feature values can be set for each representative image sequence selected by the user from the database, and the weights set for each of the representative image sequences can be set in the trajectory on the eigenspace. Can be stored in association with the position. Therefore, it is possible to consider the difference in the similarity scale due to the difference in the image sequence, and it is possible to search a wider range of the image sequence.

Further, in the search stage, a weight between feature values for a representative image sequence set in advance, which is closest to the locus of the image sequence given as a search key on the eigenspace, is searched, and this weight is determined. A search is performed based on the similarity measure used. Therefore, a search can be performed using the most suitable similarity measure for each image sequence, and a more accurate search that meets the user's requirements can be performed.

A time-series image search device according to the present invention is a time-series image search device which gives a time-series image sequence as a key and searches a similar time-series image sequence portion from a time-series image database. Means for inputting a time-series image sequence and accumulating it in a time-series image database; means for calculating a spatial distribution feature vector of an image at each unit time from the input image sequence; Means for calculating the eigenspace of the spatial distribution feature from a sequence of spatial distribution feature vectors calculated from a set of image sequences, and a trajectory when the sequence of images is projected onto the eigenspace is calculated and stored in a database Means for calculating an auxiliary feature amount for each unit time from a set of image sequences, means for storing the auxiliary feature amount in a database, and a search key. Means for inputting a time-series image sequence, means for calculating the distance between the input image sequence and the trajectory when projected on the eigenspace calculated previously, and storing the trajectory and the database Means for calculating the distance from the trajectory of the image sequence present, means for calculating an auxiliary feature amount for the image sequence input as a key for the search, and means for storing the auxiliary feature amount and the data stored in the database. Means for calculating the distance to the auxiliary feature of the image sequence being present, and calculating the weighted sum of the distance between the trajectories in eigenspace and the distance between the auxiliary features as the dissimilarity between the image sequences, Means is provided for outputting, as a search result, an image sequence having a low degree of dissimilarity from among the image sequences stored in the database.

[0025]

Next, embodiments of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 1, a time-series search device according to an embodiment of the present invention includes an input unit 100, a processing unit 200, and an output unit 300.

The input unit 100 includes a database image sequence input unit 101 for inputting an image sequence to be stored in a database, a search key image sequence input unit 102 for inputting an image sequence serving as a search key, and a similarity measure. Learning image sequence selection unit 103 for selecting a representative image sequence in a database for adjusting
Similarity evaluation providing unit 10 for evaluating the similarity of the representative image sequence in the database given by
Consists of four.

The processing unit 200 includes an image sequence storage unit 201 that stores the image sequence input by the database image sequence input unit 101, and an image space distribution feature calculation unit 202 that calculates the feature amount of the image space distribution from the image sequence. For the image sequence in the image sequence storage unit 201, an eigenspace construction unit 203 that constructs an eigenspace of the feature calculated by the image space distribution feature calculation unit 202, and a trajectory in the eigenspace corresponding to the image sequence are obtained. An eigenspace projection unit 204, an auxiliary image feature calculation unit 205 that calculates auxiliary image features for the image sequence, and a trajectory and an auxiliary image feature on the eigenspace corresponding to the image sequence in the image sequence storage unit 201. Using the eigenspace image sequence storage unit 206 to store the image sequence given as a key in the eigenspace and auxiliary image features, the eigenspace A retrieval unit 207 for retrieving an image sequence from the image sequence storage unit 206 and an image retrieved from the image sequence in the image sequence storage unit 201 selected by the image sequence selection unit 103 using a measure of the distance between trajectories in eigenspace. The similarity of the series is evaluated based on the evaluation criterion provided by the similarity evaluation providing unit 104, and the feature amount weight adjustment unit 20 adjusts the weight at the time of the dissimilarity calculation so as to match the evaluation criterion.
8 and a feature amount weight storage unit 20 for storing the adjusted weight.
Consists of nine.

The output unit 300 outputs the search result image sequence output from the processing unit 200 to a display device, a file device, or the like.

FIGS. 2, 3 and 4 are flow charts showing an example of the processing of this embodiment shown in FIG.

As shown in FIG. 2, FIG. 3, and FIG. 4, the processing of this embodiment includes three steps of an eigenspace construction step, a feature weight adjustment step, and a search step, and is executed in order.

In the eigenspace construction stage of step 1, an image sequence of the database is input (step 401), and the spatial distribution characteristics of the image sequence are calculated for each unit time (step 402). An eigenspace is constructed (step 403), a trajectory in the eigenspace corresponding to the image sequence is calculated (step 404), and an auxiliary feature amount of the image sequence is calculated for each unit time (step 405). The trajectory on the eigenspace and the auxiliary feature are stored (step 406).

In the feature amount weight adjustment step of step 2,
A representative image sequence for adjusting the feature amount weight is selected (step 410), and the spatial distribution feature of the image sequence is calculated for each representative image sequence (step 41).
1) Calculate the trajectory in the eigenspace, search for a plurality of image sequences having a short distance (step 412), and evaluate the similarity between the searched image sequence and a representative image sequence input as a key. (Step 413) Further, an auxiliary image feature amount is calculated for a representative image sequence input as a key (Step 414), the feature amount weight is adjusted (Step 415), and the representative value input using the weight as a key is calculated. The image sequence is stored in association with the position of the trajectory on the eigenspace of the typical image sequence (step 416).

In the search step of step 3, an image sequence serving as a search key is input (step 420), a trajectory in the eigenspace is calculated (step 421), and an auxiliary image feature value is calculated (step (420)). 422), a feature weight suitable for the image sequence of the search key is searched (step 423), a similar image sequence is searched from the image sequences in the database (step 424), and the searched image sequence is output (step 425). ).

Hereinafter, the processing of each unit will be specifically described.

The image sequence storage unit 201 is a database that stores the image sequences input by the database image sequence input unit 101. In the eigenspace configuration stage of stage 1, all the stored image sequences are stored in the image space. Output to the distribution feature calculation unit 202 and the auxiliary image feature calculation unit 205, and in the feature amount weight adjustment stage of stage 2, the image sequence specified by the learning image sequence selection unit 103 is converted to the image space distribution feature calculation unit 202 and the auxiliary image The image sequence is output to the feature calculation unit 205, and an image sequence is output to the search unit 207 in response to a request from the search unit 207.

As this image sequence, a set of image sequences that continue without a break in time and an image sequence that is continuous in an arbitrary section can be input and stored.

The image space distribution feature calculation unit 202 inputs the image sequence stored in the image sequence storage unit 201 and the image sequence input by the search key image sequence input unit 102, and outputs the image sequence on an image plane for each unit time. Is calculated as a one-dimensional vector. In the eigenspace construction stage of stage 1, the image sequence storage unit 2
The feature amount is calculated for the image sequence input from 01,
Output to eigenspace construction section 203 and eigenspace projection section 204. In steps 2 and 3, the feature amounts calculated for the input image sequence are stored in the eigenspace projection unit 20.
4 is output.

Here, as an example of the spatial distribution feature of an image,
A method of dividing an image plane into meshes, calculating an average of gray values of images included in each mesh at certain time intervals, obtaining a one-dimensional feature vector at each time, and a velocity vector field at each time. A method for obtaining a feature amount using one or both of the methods for obtaining the feature amount vector and arranging them in a line at each time and obtaining the feature amount vector will be described.

In the case of the former feature, the image I (i, j, t) obtained at a certain time t as shown in the example of FIG.
As in, delimiting the mesh N _X × N _y, and gray value I of the pixels in each mesh (i, j, t) of the mesh an average value of the values D _t (x, y). Raster scan of the values of each mesh as a feature value, as a one-dimensional vector

[0041]

[Outside 1] It expresses.

The characteristic of the latter velocity field is that the latest image I (i, j, t) at time t and the image I (i, j, ta) one to several frames earlier are shown in FIG. In this manner, the mesh is divided, and each partial image is subjected to matching using the cross-correlation coefficient between the images described in the related art, and a method of obtaining a moving speed of the pattern from a shift width in which the cross-correlation coefficient is maximum. , Can be calculated as a velocity component for each mesh. As a result, the speed field

[0043]

[Outside 2] Can be requested. This is also a one-dimensional vector

[0044]

[Outside 3] Is expressed as

Any one of such spatial distribution features can be used independently, and some feature amounts can be integrated into a one-dimensional feature vector per time.

Note that features other than those described above can also be used, such as a simple one-dimensional vector in which the gray values of each pixel on the image plane are simply arranged in order.

In the eigenspace construction stage 203 in the eigenspace construction stage of stage 1, the image space distribution feature calculation unit 2 for all image sequences stored in the image sequence storage unit 201
The sequence of image space distribution features calculated in step S 02 is input, an eigenspace is formed from a set of feature amount vectors, and a base vector extending the eigenspace is output to the eigenspace projection unit 204.

The feature amount of the image sequence obtained by the image space distribution feature calculation unit 202 is one set of feature vectors per time.

[0049]

[Outside 4] It is represented as Here, the average of the feature amount vectors of all learning image sequences

[0050]

[Outside 5] To

[0051]

(Equation 1) Calculate as follows. Here, Z is the image sequence storage unit 201
Is the number of feature amount vectors calculated by the image space distribution feature calculation unit 202 for all image sequences stored in.

Next, feature amount vectors of all learning image sequences

[0053]

[Outside 6] The mean vector from

[0054]

[Outside 7] And one matrix

[0055]

[Outside 8] Put together.

[0056]

(Equation 2) Next, the covariance matrix of the feature set of the image sequence for learning

[0057]

[Outside 9] To

[0058]

(Equation 3) Eigen equation for the image sequence for learning

[0059]

(Equation 4) Solve for k large eigenvalues

[0060]

[Outside 10] Eigenvector corresponding to

[0061]

[Outside 11] To obtain an eigenspace with 基底 as a base vector. This basis vector

[0062]

[Outside 12] Are output to the eigenspace projection unit 204.

The eigenspace projection unit 205 inputs the feature amount of the image sequence obtained by the image space distribution feature calculation unit 202 and converts the image sequence into an eigenspace on which the basis vectors obtained by the eigenspace construction unit 203 span. To calculate the trajectory obtained by projecting the trajectory. In the eigenspace construction stage of stage 1,
The trajectory in the eigenspace is calculated for all the image sequences stored in the image sequence storage unit 201, and the sequence of points constituting each trajectory is stored in the eigenspace image sequence storage unit 206. In steps 2 and 3, the trajectory of the target image sequence in the eigenspace is calculated, and
Output to

Sequence of input feature amount vectors

[0065]

[Outside 13] In eigenspace corresponding to

[0066]

[Outside 14] Is

[0067]

(Equation 5) And the image sequence is expressed as a trajectory consisting of a sequence of points on the eigenspace.

The auxiliary image feature calculation unit 205 calculates an auxiliary image feature amount for each unit time with respect to the image sequence input by the image sequence storage unit 201 and the search key image sequence input unit 102. Is stored in the eigenspace image sequence storage unit 206, and in steps 2 and 3,
Output to the search unit 207.

Examples of the auxiliary image feature amount include a texture feature or a color feature of an image density distribution, a shape feature of an image pattern, and the like. These features are, for each type of feature, a one-dimensional feature vector

[0070]

(Equation 6) Is calculated as

Note that N-1 is the number of auxiliary feature amounts.

The search unit 207 inputs a trajectory in the eigenspace corresponding to an image sequence serving as a key for inspection and a sequence of auxiliary image feature amounts, and searches for the image sequence stored in the image sequence storage unit 201. , A search for an image sequence similar to the image sequence of the search key is performed.

In the feature amount weight adjustment step of step 2,
The trajectory in the eigenspace for the image sequence in the image sequence storage unit 201 selected by the learning image sequence selection unit 103 is input, and the trajectory stored in the in-eigenspace image sequence storage unit 206 is input. A search is made for a trajectory having a small distance from the obtained trajectory, and a corresponding image sequence is stored in the image sequence storage unit 20.
1 together with the trajectory of the image sequence and the auxiliary feature amount, and output the feature amount weight adjustment unit 208.

In the search step of step 3, the trajectory in the eigenspace calculated for the image sequence of the search key input by the search key image sequence input unit 102 and the auxiliary image feature amount are input. Of the image sequence stored in the eigenspace image sequence storage unit 206 to determine the degree of dissimilarity with the image sequence stored in the eigenspace image sequence storage unit 206. Calculated as a weighted sum of the distances, an image sequence with a low degree of dissimilarity is taken as a search result, the original image sequence is extracted from the image sequence storage unit 201, and output to the output unit 300.

Here, the following method will be described as an example of the method of steps 2 and 3. In addition, methods other than the following can also be used.

First, in step 2, in order to reflect the similarity evaluation between the image sequences possessed by the user in the search, a weighting coefficient for calculating the distance between some image features is determined. Therefore, first, in the learning image sequence selection unit 103, from among the image sequences in the image sequence storage unit 201,
The user selects a representative image sequence and projects the trajectory when the image sequence is projected onto the eigenspace by eigenspace projection unit 204.
Is calculated by A point sequence forming one of the trajectories of this representative image sequence (referred to as a representative image sequence)

[0077]

[Outside 15] It expresses. Here, M is the number of times when the feature amount is calculated for the image sequence to be predicted. This point sequence is called an input image locus. Here, the input image trajectory and the image sequence stored in the eigenspace image sequence storage unit 206 are stored.

[0078]

[Outside 16] Is calculated as follows.

[0079]

(Equation 7) However, in general, it is considered that the smaller the distance D is, the more similar the two image sequences are.

Therefore, such a distance D is calculated for all image sequences stored in the eigenspace image sequence storage unit 206, and one or a plurality of partial image sequences having a small distance D are extracted.

FIG. 8 shows the trajectory in the eigenspace corresponding to the representative image sequence and the candidate image sequence. The trajectory of the image sequence thus searched, the auxiliary image feature, and the image sequence are output to the feature weight adjustment unit 208.

Next, in the search step of Step 3, the trajectory on the eigenspace calculated by the eigenspace projection unit 204 is applied to the image sequence of the search key input by the search key image sequence input unit 102, and Auxiliary image feature calculator 20
5 is input. Here, the point sequence forming the trajectory corresponding to the search key image sequence is

[0083]

[Outside 17] It expresses. Here, M is the number of times when the feature amount is calculated for the image sequence to be predicted. In addition, N-1 types of auxiliary image feature

[0084]

(Equation 8) It expresses.

The dissimilarity between the search key image sequence and the image sequence in the eigenspace image sequence storage unit 206 is calculated as a weighted sum of the distance between the trajectories in the eigenspace and the distance between the auxiliary feature amounts. In this case, the weighting is performed in the feature amount weighting adjustment step of step 2 by selecting the trajectory in the eigenspace corresponding to the image sequence of the retrieval key from the trajectory in the eigenspace of the selected representative image sequence. The closest one is a feature weighted storage unit 2
09 and uses the weight calculated for the searched image sequence. Here, one point of the trajectory in the eigenspace for the search key image sequence

[0086]

[Outside 18] The weights found for

[0087]

[Outside 19] It expresses.

Using these, the dissimilarity between the search key image sequence and the image sequence in the eigenspace image sequence storage unit 206 is:

[0089]

(Equation 9) Can be calculated. However, the image sequence stored in the eigenspace image sequence storage unit 206 for which the distance is to be calculated is

[0090]

[Outside 20] And N-1 types of auxiliary image feature amounts

[0091]

[Outside 21] It expresses.

From the image sequence in the eigenspace image sequence storage unit 206, those having a small degree of dissimilarity are extracted and output as search results.

In the feature amount weighting adjustment step of step 2, the feature amount weighting adjustment unit 208 performs the representative image sequence (representative image selection) by the user in the learning image sequence selection unit 103 in step 2 of the search unit 207. Sequence), a plurality of image sequences having a small distance between the searched trajectories from among the trajectories stored in the eigenspace image sequence storage unit 206 are input, and the retrieved image The similarity evaluation providing unit 104 evaluates whether the sequence is similar to the representative image sequence using an evaluation scale given by the user,
The weighting of the distance between the feature values is calculated so that the degree of dissimilarity to the similar image sequence is reduced, and the feature value weight storage unit 209 stores the feature value in association with the trajectory of the representative image sequence.

Here, the following method will be described. In addition, methods other than the following can also be used.

First, the user selects a plurality of representative image sequences from the image sequences in the image sequence storage unit 201. Subsequently, for the selected representative image sequence, a plurality of image sequences in which the distance of the trajectory in the eigenspace in the eigenspace image sequence storage unit 206 is small are searched and input. This searched image sequence is called a candidate image sequence. The similarity between each representative image sequence and the candidate image sequence is determined by the similarity evaluation providing unit 104.
The evaluation is performed using a scale given by the user to determine whether each of the candidate image sequences is similar or not similar to the representative image sequence. One of the methods is
A method of presenting the representative image sequence and the candidate image sequence to the user, visually judging the similarity, and giving the result through the similarity evaluation providing unit 104 can be used. Note that a method other than this method can be used.

Next, using this result, the candidate image sequence is classified into two classes, an image sequence which is a key for retrieval and an evaluated image sequence class similar thereto, and an image sequence evaluated as dissimilar. The weighting of the distance between the feature values is calculated by discriminant analysis so that the variation in the feature value within each class is small and the variation between the classes is large.

Here, as the definition of the degree of dissimilarity between the representative image sequence and the candidate image sequence, a weighted sum of the distance between the trajectories in the eigenspace and the distance between the auxiliary feature amounts is used.

Representative image sequence and candidate image sequence S_i Between
The distance between the trajectories in the eigenspace is d₁, Between auxiliary features
Is represented as dj (dj = 2, 3,..., N). Distance
Separation d ₁ Can use D in equation (7),
Distance dj (j = 2,3, ..., N)

[0099]

(Equation 10) Can be calculated as follows.

Also, the weighting of the distance of the feature quantity is represented by w ₁ , w ₂ ,
..., Expressed as w _N. Then, a weighted sum (dissimilarity) y of the distance between the feature amounts between the representative image sequence and the candidate image sequence is:

[0101]

[Equation 11] It can be expressed as. Here, the class of the candidate image sequence similar to the representative image sequence is C ₁ , and the class of the dissimilar candidate image sequence is C ₂ , to classify the candidate image sequence, and the distance between the corresponding representative image sequence and the candidate image sequence.

[0102]

[Outside 22] Is an element of the set of each class. If images belonging to the same class can be arranged as close as possible and images belonging to different classes can be arranged as far as possible, only similar images can be easily searched. Therefore, in the converted space (distance) Y, the transformation matrix A is determined so that the variation within each class is small and the variation between classes is large. Therefore, the within-class covariance matrix

[0103]

[Outside 23] , Interclass covariance matrix

[0104]

[Outside 24] Is introduced.

[0105]

(Equation 12) However, P _i is the a priori probability of class _i, n i is the number of samples in class i. Also,

[0106]

[Outside 25] Is the mean vector of class i,

[0107]

[Outside 26] Is the average vector of all elements.

In the converted space Y, the same amount

[0109]

[Outside 27] To

[0110]

(Equation 13) Can be sought. Here, in the converted feature space, it is desirable that the intra-class variance is as small as possible and the inter-class variance is as large as possible. Therefore, an evaluation function J (A) representing the degree of separation between the classes after the conversion is set. I do. As the evaluation function J (A), the ratio between the intra-class variance and the inter-class variance in the space after the conversion is:

[0111]

[Equation 14] Which can be considered as a maximization problem for finding A that maximizes the evaluation function J (A). This maximization problem is

[0112]

(Equation 15) Is equivalent to maximizing the numerator of equation (19) under the normalization condition

[0113]

(Equation 16) To come back to. Therefore,

[0114]

[Outside 28] A matrix in which the eigenvector corresponding to the largest eigenvalue of is a row vector is the transformation matrix A to be obtained, and its element is the weight of the distance between the feature amounts to be obtained.

The above processing is independently performed on each representative image sequence, and the weight is obtained. Then, the obtained weights are stored in the feature amount weight storage unit 209 at the positions of the respective points on the trajectory in the eigenspace of each representative image sequence.

Referring to FIG. 9, a time-series image search device according to another embodiment of the present invention includes an input device 501 and a storage device 50.
2, 503, 504, output device 505, and recording medium 506
And a data processing device 507.

The input device 501 is used by the user to input an image sequence serving as a key for an image sequence search to be stored in the database, to select an image sequence in the database, and to input evaluation criteria for similarity evaluation. It is for. The storage devices 502, 503, and 504 are respectively an image sequence storage unit 201, an eigenspace image sequence storage unit 206,
It corresponds to the feature amount weight storage unit 209. The output device 505 is a display device, a file device, or the like to which search results are output. The storage medium 506 stores the image sequence input from the input device 501 and the image sequence serving as a search key in the storage device 502, and stores the image space distribution feature calculation unit 20 in FIG.
2. A CD that records a time-series image search program including the steps of an eigenspace construction unit 203, an eigenspace projection unit 204, an auxiliary image feature calculation unit 205, a search unit 207, a feature weight adjustment unit 208, and an output unit 300. -A recording medium such as a ROM, a floppy disk, a magneto-optical disk, and a semiconductor memory. The data processing device 507 is a CPU that reads a time-series image search program from the recording medium 506 and executes the program.

[0118]

As described above, the present invention introduces auxiliary features in addition to the distance between the trajectories on the eigenspace based on the spatial distribution features of the image, and introduces those features. The weighted sum of the distances between the images is used as a measure of dissimilarity between image sequences, and the weight is adjusted to match the measure of similarity between users. A high-precision search for an image sequence that matches the evaluation can be performed. Also, for any image sequence on the eigenspace, the weight of the distance between the feature values can be calculated, and in order to store the weight of the distance between the feature values in association with the position of the trajectory of the image sequence on the eigenspace, It is possible to flexibly cope with a difference in similarity scale due to a difference in image sequence, and it is possible to apply the present invention to a search for a wider range of image sequences.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a time-series image search device according to an embodiment of the present invention.

FIG. 2 is a flowchart of a process of a stage 1 in the embodiment of FIG. 1;

FIG. 3 is a flowchart of a process of a stage 2 in the embodiment of FIG. 1;

FIG. 4 is a flowchart of a process of Step 3 in the embodiment of FIG. 1;

FIG. 5 is a diagram illustrating an example of an input weather image in the embodiment of FIG. 1;

FIG. 6 is a diagram illustrating a spatial distribution feature of a pattern used in the embodiment of FIG. 1;

FIG. 7 is a diagram illustrating a velocity field of a pattern used in the embodiment of FIG. 1;

FIG. 8 is a diagram illustrating a trajectory in an eigenspace corresponding to a representative image sequence and a candidate image sequence in the embodiment of FIG. 1;

FIG. 9 is a configuration diagram of a time-series image search device according to another embodiment of the present invention.

[Explanation of symbols]

REFERENCE SIGNS LIST 100 input unit 101 database image sequence input unit 102 search key image sequence input unit 103 training image sequence selection unit 104 similarity evaluation providing unit 200 processing unit 201 image sequence storage unit 202 image space distribution feature calculation unit 203 eigenspace construction unit 204 unique Space projection unit 205 Auxiliary image feature calculation unit 206 Eigenspace image sequence storage unit 207 Search unit 208 Feature weight adjustment unit 209 Feature weight storage unit 300 Output unit 401 to 406, 410 to 416, 420 to 425
Step 501 Input device 502, 503, 504 Storage device 505 Output device 506 Recording medium 507 Data processing device

Claims (39)

[Claims] 1. A time-series image search method for providing a time-series image sequence as a key and searching for a portion of a similar time-series image sequence from a time-series image database, wherein a time-series image sequence is input. Accumulating the image sequence in the time-series image database, calculating the spatial distribution feature amount vector of the image at each unit time from the input image sequence, and calculating from the set of image sequences in the time-series image database. Calculating the eigenspace of the spatial distribution feature from the sequence of the spatial distribution feature vector; calculating the trajectory when the image sequence is projected onto the eigenspace; storing the locus in a database; A step of calculating an auxiliary feature amount from the set at each unit time; a step of storing the auxiliary feature amount in a database; A step of inputting a time-series image sequence; a step of calculating a trajectory of the input image sequence when projected on the previously calculated eigenspace; an image stored in the database with the trajectory Calculating a distance from a sequence trajectory; calculating an auxiliary feature amount for an image sequence input as a key for the search; and storing the auxiliary feature amount and an image stored in the database. Calculating the distance between the auxiliary features of the sequence and calculating the weighted sum of the distance between the trajectories in eigenspace and the distance between the auxiliary features as dissimilarity between the image sequences, Outputting as a search result an image sequence having a low degree of dissimilarity from among the stored image sequences. 2. The method according to claim 1, wherein the step of calculating a spatial distribution feature amount vector of the image at each unit time from the image sequence comprises the step of calculating the spatial distribution feature amount as a one-dimensional vector that can sequentially arrange grayscale values of pixels on an image plane. 2. The method of claim 1, further comprising the step of calculating a vector. 3. A step of calculating a spatial distribution feature amount vector of an image at each unit time from the image sequence, dividing the image plane into a mesh, and calculating an average of grayscale values of images included in each mesh at a certain time interval. 2. The time-series image search method according to claim 1, further comprising the step of calculating the spatial distribution feature vector as a one-dimensional feature vector at each time. 4. A step of calculating a spatial distribution feature amount vector of an image at each unit time from the image sequence includes obtaining a velocity vector field at each time, arranging velocity components at respective points on an image plane in a line, and performing one-dimensional processing. 2. The time-series image search method according to claim 1, further comprising the step of obtaining the spatial feature vector as the feature vector of (a). 5. The method according to claim 1, wherein the step of calculating an auxiliary feature amount for each unit time from the set of image sequences comprises: a texture feature, a color feature, or a part or all of a shape feature of an image pattern. 2. The time-series image search method according to claim 1, further comprising the step of calculating the auxiliary feature amount as a one-dimensional vector for each unit time. 6. The step of calculating the eigenspace includes forming a matrix by combining feature amounts of a one-dimensional vector obtained at a certain time interval with respect to the image sequence for the entire image sequence, and calculating the matrix of the entire image sequence from the matrix. 2. The time-series image according to claim 1, including a step of obtaining a covariance matrix for a matrix obtained by subtracting an average value of the feature amount vectors, and obtaining an eigenspace based on an eigenvector for a large eigenvalue obtained by solving an eigenvalue problem of the matrix. How to search. 7. A step of calculating a trajectory when a sequence of spatial distribution feature amounts corresponding to the image sequence is projected onto an eigenspace, wherein the trajectory is obtained at a certain time interval with respect to the image sequence. 2. The method according to claim 1, further comprising the step of sequentially converting the feature values of the dimensional vector into points on the eigenspace, and obtaining the point sequence as a trajectory on the eigenspace for the image sequence.
Time series image search method. 8. A step of calculating a distance between a trajectory in an eigenspace of an image sequence input as a key of the search and a trajectory of an image sequence stored in the database, wherein: 2. The time-series image search method according to claim 1, further comprising a step of obtaining a distance between a series of points on a trajectory corresponding to the feature amount vector of the time by observing a time order of each point and summing them. 9. The dissimilarity between an image sequence input as a key for the search and an image sequence stored in a database is calculated by calculating a distance between trajectories in eigenspace and a distance between auxiliary feature amounts. 2. The step of calculating as a weighted sum of includes adjusting the magnitude of the weight multiplied by the distance in advance so that the search result matches the user's evaluation.
Time series image search method. 10. A step of adjusting a weight of a distance between individual feature amounts when calculating a dissimilarity between an image sequence input as a search key and an image sequence stored in a database. , The user selects a representative image sequence from the image sequences stored in the database, and for each, searches for an image sequence in which the distance between the trajectories in the eigenspace is short. Is evaluated by an evaluation scale given by the user as to whether or not the image sequence is similar to the image key of the search key, and the weighting of the distance between the individual feature amounts is performed so that the degree of dissimilarity to the similar image sequence is reduced. 10. The time-series image search method according to claim 9, further comprising the step of: 11. A typical image sequence selected by the user is searched for an image sequence with a short distance between trajectories in the eigenspace, and the user gives an evaluation as to whether or not they are similar. Determining the weight of the distance between the individual feature amounts so that the degree of dissimilarity with respect to the sequence is small; and for each representative image sequence, a set of searched image sequences is represented by a representative image sequence. Are classified into two classes: an image sequence evaluated to be similar to the image sequence, and an image sequence evaluated to be dissimilar to the class. The variation in the distance between the feature amounts in each class is reduced. 11. The time-series image search method according to claim 10, further comprising the step of calculating the weight of the distance between the feature values by discriminant analysis so as to increase the variation. 12. For each of the representative image sequences selected by the user, the obtained weights for the dissimilarity calculation are calculated on the trajectory in the eigenspace corresponding to each representative image sequence. 10. The time-series image search method according to claim 9, further comprising a step of storing the image in association with the position. 13. The dissimilarity between an image sequence input as a key for the search and an image sequence stored in a database is calculated by calculating a distance between trajectories in eigenspace and a distance between auxiliary feature amounts. The step of calculating as a weighted sum of the search for the trajectory in the eigenspace corresponding to the representative image sequence previously selected by the user, which is closest to the trajectory in the eigenspace corresponding to the image sequence of the search key, 11. The time-series image search method according to claim 10, further comprising the step of calculating a dissimilarity between a search key and an image series in a database using a weighted sum calculated for the representative image series. 14. A time-series image sequence is given as a key,
What is claimed is: 1. A time-series image search apparatus for searching a time-series image database for a portion of a similar image series, comprising: a means for inputting a time-series image series and storing the input image series in a time-series image database; Means for calculating a spatial distribution feature amount vector of an image at each unit time from, and eigenspace of the spatial distribution feature from a sequence of spatial distribution feature vectors calculated from a set of image sequences in a time-series image database. Means for calculating a trajectory when the image sequence is projected onto the eigenspace and storing the calculated trajectory in a database; means for calculating an auxiliary feature amount per unit time from a set of image sequences; Means for storing a characteristic amount in a database, means for inputting a time-series image sequence serving as a search key, and a unique characteristic value previously calculated for the input image sequence. Means for calculating a trajectory when projected on space; means for calculating the distance between the trajectory and the trajectory of the image sequence stored in the database;
Means for calculating an auxiliary feature for an image sequence input as a key for the search, and calculating a distance between the auxiliary feature and an auxiliary feature of the image sequence stored in the database; Means for calculating the weighted sum of the distance between the trajectories on the eigenspace and the distance between the auxiliary features as the dissimilarity between the image sequences, and calculating the dissimilarity from the image sequences stored in the database. A means for outputting an image sequence having a small size as a search result. 15. A means for calculating a spatial distribution feature vector of an image from the image sequence at each unit time, wherein the spatial distribution feature vector is a one-dimensional vector that can be obtained by sequentially arranging gray values of pixels on an image plane. 15. The time-series image search device according to claim 14, further comprising: means for calculating. 16. A means for calculating a spatial distribution feature vector of an image at each unit time from the image sequence, divides an image plane into meshes, and averages grayscale values of images included in each mesh at a certain time interval. 15. The time-series image retrieval device according to claim 14, further comprising means for calculating the spatial distribution feature vector as a one-dimensional feature vector at each time. 17. A means for calculating a spatial distribution feature amount vector of an image at each unit time from the image sequence, finds a velocity vector field at each time, arranges velocity components of each point on an image plane in a line, and performs one-dimensional 15. The time-series image search device according to claim 14, further comprising means for obtaining the spatial distribution feature amount vector as the feature amount vector. 18. A means for calculating an auxiliary feature amount for each unit time from the set of image sequences, wherein a part or all of a texture feature, a color feature, or a shape feature of an image pattern of an image density distribution is featured. 15. The time-series image search device according to claim 14, further comprising means for calculating the auxiliary feature amount as a one-dimensional vector for each unit time. 19. The means for calculating the eigenspace forms a matrix by combining feature amounts of one-dimensional vectors obtained at certain time intervals with respect to an image sequence for all image sequences,
Includes means for obtaining a covariance matrix for a matrix obtained by subtracting the average value of feature vectors of all image sequences from this matrix, and obtaining an eigenspace based on eigenvectors for large eigenvalues obtained by solving an eigenvalue problem of this matrix Claim 14
Time-series image search device. 20. A means for calculating a trajectory when a sequence of spatial distribution feature amounts corresponding to the image sequence is projected onto an eigenspace field, wherein the trajectory is obtained at a certain time interval with respect to the image sequence. 15. The time-series image search device according to claim 14, further comprising means for sequentially converting the feature values of the dimensional vector into points on the eigenspace, and obtaining the point sequence as a trajectory in the eigenspace for the image sequence. 21. A means for calculating a distance between a trajectory in an eigenspace for an image sequence input as a key for the search and a trajectory of an image sequence stored in the database, wherein each of the trajectories is calculated at each time 15. The time-series search device according to claim 14, further comprising means for obtaining a distance between a series of points on a trajectory corresponding to the feature amount vector of each of the points by observing the time order of each point and summing them. 22. The degree of dissimilarity between an image sequence input as a key for the search and an image sequence stored in a database is determined by a distance between trajectories in eigenspace and a distance between auxiliary feature amounts. 15. The time-series image search device according to claim 14, wherein the means for calculating the sum of the weights includes a means for adjusting the magnitude of the weight multiplied by the distance in advance so that the search result matches the evaluation of the user. 23. A means for adjusting a weight of a distance between individual feature amounts when calculating a dissimilarity between an image sequence input as a key for the search and an image sequence stored in a database. The user selects a representative image sequence from the image sequences stored in the database, searches for an image sequence with a short distance between the trajectories in the eigenspace for each, and searches for the searched image sequence. Is evaluated by an evaluation scale given by the user as to whether or not the image sequence is similar to the key image sequence of the search, and the weighting of the distance between the individual feature amounts is performed so that the degree of dissimilarity to the similar image sequence is reduced. 23. The time-series search device according to claim 22, further comprising: means for determining 24. With respect to a representative image sequence selected by the user, an image sequence with a short distance between trajectories in the eigenspace is searched, and the user gives an evaluation as to whether or not they are similar. The means for determining the weighting of the distance between the individual feature amounts so that the degree of dissimilarity to the sequence is reduced, for each representative image sequence, a set of searched image sequences is converted into a representative image sequence. The image sequence is classified into two classes, an image sequence evaluated to be similar and an image sequence evaluated to be dissimilar, so that the variation in the distance between the feature amounts in each class is reduced. 24. The time-series image search device according to claim 23, further comprising means for calculating the weight of the distance between the feature amounts by discriminant analysis so that the distance becomes large. 25. For each of the representative image sequences selected by the user, the obtained weight for the dissimilarity calculation is calculated by using the trajectory on the eigenspace corresponding to each representative image sequence. 23. The time-series image search device according to claim 22, further comprising means for storing in association with a position. 26. The degree of dissimilarity between an image sequence input as a search key and an image sequence stored in a database is calculated by calculating a distance between trajectories in eigenspace and a distance between auxiliary feature amounts. Means for calculating a weighted sum of the search for the locus on the eigenspace corresponding to the representative image sequence selected by the user in advance, which is closest to the locus on the eigenspace corresponding to the image sequence of the search key, 3. A means for calculating a dissimilarity between a search key and an image sequence in a database using a weighted sum calculated for the representative image sequence.
3. A time-series image search device. 27. A means for inputting a time-series image sequence and storing the image sequence in a time-series image database, a procedure for calculating a spatial distribution feature vector of an image at each unit time from the input image sequence, A procedure of calculating an eigenspace of the spatial distribution feature from a sequence of spatial distribution feature vectors calculated from a set of image sequences in the sequence image database;
Calculating a trajectory when the image sequence is projected onto the eigenspace and storing the trajectory in a database; calculating an auxiliary feature amount for each unit time from a set of image sequences; Storing a feature amount in a database, inputting a time-series image sequence serving as a search key, and calculating a trajectory of the input image sequence when projected on a previously calculated eigenspace A procedure for calculating a distance between the trajectory and the trajectory of the image sequence stored in the database; a procedure for calculating an auxiliary feature amount for the image sequence input as a key for the search; Calculating the distance between the auxiliary feature and the auxiliary feature of the image sequence stored in the database; and weighting the distance between the trajectories and the auxiliary feature in the eigenspace Sum between image sequences Calculated as dissimilarity, the time-series image retrieval program storage medium storing for executing the steps of outputting a search result dissimilarity is smaller image sequences from the image sequence stored in the database in the computer. 28. A procedure for calculating a spatial distribution feature amount vector of an image from the image sequence at each unit time, wherein the spatial distribution feature amount vector is a one-dimensional vector formed by sequentially arranging gray values of pixels on an image plane. 28. The recording medium of claim 27, comprising the step of: 29. A procedure for calculating a spatial distribution feature vector of an image at each unit time from the image sequence includes dividing an image plane into meshes, and calculating an average of grayscale values of images included in each mesh at a certain time interval. 28. The recording medium according to claim 27, further comprising a step of calculating the spatial distribution feature vector as a one-dimensional feature vector at each time. 30. A step of calculating a spatial distribution feature amount vector of an image at each unit time from the image sequence includes obtaining a velocity vector field at each time, arranging velocity components of each point on an image plane in a line, and performing one-dimensional processing. 28. The recording medium according to claim 27, further comprising a step of obtaining said spatial distribution feature vector as said feature vector. 31. A method of calculating an auxiliary feature amount for each unit time from the set of image sequences, wherein a part or all of a texture and a feature of an image density distribution, a color feature, or a shape feature of an image pattern is determined. 28. The recording medium according to claim 27, comprising a step of calculating the auxiliary feature amount as a one-dimensional vector for each unit time. 32. The step of calculating the eigenspace, comprising: combining a feature amount of a one-dimensional vector obtained for each image sequence at a certain time interval for all image sequences to form a matrix;
Procedure for obtaining a covariance matrix for a matrix obtained by subtracting the average value of the feature amount vectors of all image sequences from this matrix, and obtaining an eigenspace based on eigenvectors corresponding to large eigenvalues obtained by solving an eigenvalue problem of this matrix Claim 2 including
7. The recording medium of item 7. 33. A step of calculating a trajectory when a sequence of spatial distribution feature amounts corresponding to the image sequence is projected onto an eigenspace, wherein the trajectory is obtained at a certain time interval with respect to the image sequence. 28. The recording medium according to claim 27, further comprising a step of sequentially converting the feature values of the dimensional vectors into points on the eigenspace, and obtaining a sequence of the points as a trajectory on the eigenspace for the image sequence. 34. The procedure for calculating the distance between the trajectory in the eigenspace of the image sequence input as a key for the search and the trajectory of the image sequence stored in the database is as follows. 28. The recording medium according to claim 27, further comprising a step of determining a distance between a series of points on a trajectory corresponding to a feature amount vector at a time while observing a time order of each point. 35. The degree of dissimilarity between an image sequence input as a search key and an image sequence stored in a database is determined by a distance between trajectories in eigenspace and a distance between auxiliary feature amounts. 28. The recording medium according to claim 27, wherein the step of calculating as a weighted sum of the steps includes a step of adjusting in advance the magnitude of the weight to be weighted by the distance so that the search result matches the evaluation of the user. 36. A procedure for adjusting the weight of the distance between individual feature amounts when calculating the degree of dissimilarity between an image sequence input as a search key and an image sequence stored in a database. The user selects a representative image sequence from the image sequences stored in the database, searches for an image sequence with a short distance between the trajectories in the eigenspace for each, and searches for the searched image sequence. Is evaluated by an evaluation scale given by the user as to whether or not the image sequence is similar to the key image sequence of the search, and the weighting of the distance between the individual feature amounts is performed so that the degree of dissimilarity to the similar image sequence is reduced. 36. The recording medium according to claim 35, comprising a step of determining 37. With respect to a representative image sequence selected by the user, an image sequence with a short distance between trajectories in the eigenspace is searched, and the user gives an evaluation as to whether or not they are similar. The procedure of determining the weighting of the distance of each feature amount so that the degree of dissimilarity with respect to the sequence is reduced, for each representative image sequence, a set of searched image sequences is converted into a representative image sequence. The image sequence is classified into two classes, an image sequence evaluated to be similar and an image sequence evaluated to be dissimilar, so that the variation in the distance between the feature amounts in each class is reduced. 37. The recording medium according to claim 36, further comprising a step of calculating a weight of the distance between the feature amounts by a discriminant analysis so that is larger. 38. For each of the representative image sequences selected by the user, the obtained weights for the dissimilarity calculation are calculated on the trajectory in the eigenspace corresponding to each representative image sequence. The recording medium according to claim 35, further comprising a step of storing the information in association with the position. 39. The degree of dissimilarity between an image sequence input as a key for the search and an image sequence stored in a database is calculated as a distance between trajectories in eigenspace and a distance between auxiliary feature amounts. The procedure of calculating as a weighted sum of the search for the trajectory in the eigenspace corresponding to the representative image sequence previously selected by the user, which is closest to the trajectory in the eigenspace corresponding to the image sequence of the search key, 37. A method for calculating a dissimilarity between a search key and an image sequence in a database using a weighted sum calculated for the representative image sequence.
Recording medium.