JP5481710B2 - Video management apparatus, method, and program - Google Patents

Description

  The present invention relates to a video management apparatus, method, and program, and more particularly, to a video management apparatus, method, and program for recognizing or searching for a video based on the relationship between the video and additional information added to the video.

  Conventionally, video search technology for searching for a desired video based on given language information, and video recognition technology for automatically giving language information that describes the video to the given video have been developed. ing. In recent years, with the spread of imaging devices such as digital video cameras and mobile phones, and the generalization of video sharing on the Internet, such video search technology and video recognition technology have become very important technologies. .

  In addition, when the target signal is limited to still images, many image recognition search technologies have been developed that realize image search and image recognition under the same framework. For example, learning a topic model, which is a statistical model that links two observation information using latent variables, and automatically assigning an appropriate text label to a given image using this topic model A technique has been proposed that uniformly handles image acquisition for finding an appropriate image from a given text label (see, for example, Non-Patent Documents 1 and 2).

Nakayama, Harada, Kuniyoshi, Otsu “Ultra-high-speed image recognition / retrieval method using stochastic structure learning corresponding to concept between images and words”, IEICE Technical Report, PRMU2007-147, December 2007 Kimura, Nakano, Kameoka, Sugiyama, Maeda, Sakano “SSCDE: Semi-Teacher Canonical Density Estimation Method for Image Recognition Retrieval”, Image Recognition and Understanding Symposium MIRU2010, OS8-1, July 2010

  However, if the techniques described in Non-Patent Documents 1 and 2 are applied to video recognition and search techniques, the process proceeds in a way that completely ignores time information, which is one of important information of video. It will be. Considering that image information and audio information included in video has temporal continuity, it is difficult to properly model video signals, and as a result, the accuracy of video recognition and video search decreases. There is a problem.

  The present invention has been made in view of the above problems, and can perform video recognition or video search considering time information with high accuracy based on the relationship between video and additional information added to the video. It is an object to provide a management apparatus, method, and program.

  In order to achieve the above object, the video management apparatus of the present invention provides the storage extracted from each of the stored video sections obtained by time-dividing each of the stored videos as elements of the stored video set stored in advance in a predetermined section. Each of the accumulated video section features indicating the characteristics of the video sections, and each of the accumulated additional information features indicating the characteristics of the accumulated additional information extracted from each of the accumulated additional information added corresponding to each of the accumulated video sections. And a stored video segment obtained using a stored latent variable derivation function based on a stored video segment feature set that is a set of stored video segment features and a stored additional information feature set that is a set of stored additional information features And a cumulative state variable distribution that is a set of accumulated state variables indicating the distribution of accumulated latent variables that describe the relationship between the accumulated additional information and the accumulated state variable distribution. A video including a topic trajectory model storage means storing a topic trajectory model composed of the accumulated state variable set or the accumulated state variable distribution constructed by the trajectory model learning means and the accumulated latent variable derivation function; and a video recognition means. The management device, wherein the video recognition means extracts each of the input video section features indicating the characteristics of the input video section from each of the input video sections obtained by time-dividing the input video in a predetermined section. An input latent variable corresponding to the input video section feature using an extraction latent function and an input latent variable derivation function based on an input video section feature set that is a set of input video section features extracted by the input video section feature extracting means Input latent variable extracting means for extracting an input latent variable set which is a set of the input latent variables, and extracting the input latent variable by the input latent variable extracting means. An input state variable set that is a set of input state variables that are variables that describe the distribution of the input latent variables based on the set of input latent variables and the accumulated state variable set or the accumulated state variable distribution, or the input Input state variable extraction means for extracting an input state variable distribution that is a distribution of state variables, and input latent variable extraction based on the input state variable set or the input state variable distribution extracted by the input state variable extraction means Input latent variable correction means for extracting a new corrected input latent variable set obtained by correcting the input latent variable set extracted by the means, corrected input latent variable set extracted by the input latent variable correction means, and the input video section Based on the input video segment feature set extracted by the feature extraction means, it is a set of output additional information features related to the input video segment feature. Output additional information feature estimation means for estimating the output additional information feature set, and output additional information estimation means for estimating output additional information corresponding to the feature indicated by the output additional information feature set estimated by the output additional information feature estimation means And.

  Thus, the topic describing the relationship between the input state variable obtained from the input latent variable based on the input video section feature describing the relationship between the input video section and the input additional information, and the stored video set and the stored additional information set The time information is used to correct the input latent variable based on the trajectory model, and to estimate the output additional information feature that is likely to be added to the video section based on the corrected input latent variable and the input video section feature. Image recognition can be performed with high accuracy.

  Also, the video management apparatus of the present invention inputs each of the stored video as an element of the stored video set as the input video, and outputs the additional stored additional information as additional information corresponding to the stored video set by the video recognition means. Stored video recognition means for outputting an output accumulated additional information set that is a set of the above, input additional information inputted, accumulated additional information set corresponding to the inputted accumulated video set, and output outputted by the video recognition means Based on the accumulated additional information set, it may further comprise output video estimating means for estimating an image suitable for the input additional information from the accumulated video set as an output video.

  In this way, the input additional information is compared by comparing the output additional information set corresponding to the stored video set with the output additional information set for the stored video set that is accurately estimated in consideration of the time information. Therefore, the video search can be performed with high accuracy in consideration of the time information.

  The video management apparatus according to the present invention may further include the topic trajectory model learning means.

  In addition, the video management method of the present invention shows the characteristics of the stored video section extracted from each of the stored video sections obtained by time-dividing each of the stored videos that are elements of the stored video set stored in advance in a predetermined section. Based on each of the stored video section features and each of the stored additional information features indicating the characteristics of the stored additional information extracted from each of the stored additional information added corresponding to each of the stored video sections, the storage A stored video segment obtained by using a storage latent variable derivation function based on a stored video segment feature set that is a set of video segment features and a stored additional information feature set that is a set of the stored additional information features. An accumulated state variable set that is a set of accumulated state variables indicating a distribution of accumulated latent variables that describe a relationship, or the accumulated state variable that is constructed by extracting an accumulated state variable distribution that is a distribution of the accumulated state variable Or a topic trajectory model consisting of the accumulated state variable distribution and the accumulated latent variable derivation function is stored, and the characteristics of the input video section are obtained from each of the input video sections obtained by time-dividing the input video in a predetermined section. Each of the input video segment features is extracted and an input latent variable derivation function based on the input video segment feature set that is a set of the extracted input video segment features is used to obtain an input latent variable corresponding to the input video segment feature. An input latent variable set that is a set of the input latent variables is extracted, and based on the extracted input latent variable set and the accumulated state variable set or the accumulated state variable distribution, the distribution of the input latent variables is obtained. An input state variable set that is a set of input state variables that are variables to be described or an input state variable distribution that is a distribution of the input state variables is extracted, and the extracted input state variable set Alternatively, a new corrected input latent variable set obtained by correcting the input latent variable set is extracted based on the input state variable distribution, and the input is calculated based on the extracted corrected input latent variable set and the input video section feature set. This is a method of estimating an output additional information feature set that is a set of output additional information features related to a video section feature, and estimating output additional information corresponding to the feature indicated by the estimated output additional information feature set.

  Also, the video management method of the present invention inputs each of the stored videos as elements of the stored video set as the input video, and outputs the output storage as a set of output stored additional information as additional information corresponding to the stored video set. The additional information set is output, and the input additional information is input from the stored video set based on the input additional information input, the stored additional information set corresponding to the input stored video set, and the output output additional stored information set. An image suitable for information can be estimated as an output image.

  The video management program of the present invention is a program for causing a computer to function as each means constituting the video management apparatus.

  As described above, according to the video management apparatus, method, and program of the present invention, video recognition or video search considering time information is accurately performed based on the relationship between video and additional information added to the video. The effect that it can perform well is acquired.

It is a block diagram which shows the functional structure of the video recognition apparatus of 1st Embodiment. It is a block diagram which shows the detailed functional structure of the video recognition apparatus of 1st Embodiment. It is a flowchart which shows the content of the recognition process routine in the video recognition apparatus of 1st Embodiment. It is a block diagram which shows the functional structure of the video recognition apparatus of 2nd Embodiment. It is a block diagram which shows the detailed functional structure of the video recognition apparatus of 2nd Embodiment. It is a flowchart which shows the content of the search process routine in the video recognition apparatus of 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, the first embodiment will be described. In the first embodiment, a case where the video management apparatus of the present invention is applied to a video recognition apparatus that adds additional information corresponding to an input video will be described.

  The video recognition apparatus 10 according to the first embodiment is composed of a computer including a CPU, a RAM, and a ROM storing a program for executing a recognition processing routine described later. As shown in FIG. 1, it can be expressed by a configuration including a topic locus model learning unit 1 and a video recognition unit 2.

  As shown in FIG. 2, the topic trajectory model learning unit 1 further includes an accumulated video section feature extracting unit 11, an accumulated additional information feature extracting unit 12, an accumulated latent variable extracting unit 13, and an accumulated state variable extracting unit 14. Can be expressed in a configuration including The topic trajectory model learning unit 1 inputs a stored video set that is a set of pre-stored video signals and a stored additional information set that is a set of additional information corresponding to each of the stored videos. A topic trajectory model, which is a statistical model for video recognition, describing the relationship between signals and additional information is constructed and output.

  The stored video section feature extraction unit 11 inputs a stored video set, divides each of the stored video that is each element along the time axis, and extracts a plurality of stored video sections. For example, the accumulated video can be divided evenly by a predetermined time length, or a cut point can be calculated from the accumulated video, and the cut point can be divided as a boundary.

  Next, a key frame representing the stored video section is extracted from the frames included in the stored video section. As a key frame extraction method, for example, a frame at a predetermined location in the accumulated video section (a first frame, a center frame, a last frame, or the like) can be selected. Further, an average of video signals of arbitrary frames included in the accumulated video section may be created as a key frame.

  Subsequently, an image feature expressing the characteristic is calculated from the key frame, and this is extracted as an accumulated video section feature. Image feature calculation methods include, for example, color histograms, arbitrary components of digital cosine transform, arbitrary components of Haar wavelets, higher order local autocorrelation features (N. Otsu and T. Kurita “A new scheme for practical flexible and intelligent vision systems, ”Proc. IAPR Workshop on Computer Vision, pp.431-435, 1988.), Bag of Features representation of feature points selected by any method (Li Fei-Fei et al.,“ A Bayesian Hierarchical Model Methods such as for Learning Natural Scene Categories Li Fei-Fei and Pietro Perona, "2005.), and any combination of these methods can be used. Feature selection methods include SIFT (D. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, Vol. 60, No. 2, pp. 91-110, 2004.), See SURF (H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (surf),” CVIU, vol. 110, no. 3, pp. 346-359, 2008. ), Harris, Harris-Laplace, random selection, etc., and any combination of these methods. Further, as a method for extracting feature amounts from feature points, methods such as SIFT and SURF can be used.

  In addition, a window having a predetermined length is assigned to the stored video, and the stored video section is extracted while shifting the window by one frame. For the extracted stored video section, TAS (see Japanese Patent No. 30653314) or BAM is extracted. (See Japanese Patent No. 4358229) The feature amount extracted from the stored video section itself as a representative example may be calculated.

  The accumulated additional information feature extraction unit 12 extracts an accumulated additional information feature that is a vector representing the characteristic from each of the accumulated additional information that is additional information added to the accumulated video section. Here, the additional information is information describing the video, and general additional information includes text describing the content and structure of the video, information on the time and place where the video was shot, and the like.

  For example, if the additional information is text, particularly a word, a binary vector representing the presence or absence of a word (language label) included in the text label can be used as the accumulated additional information feature. More specifically, the accumulated additional information feature is a vector having the same number of dimensions as the total number of words to be considered, and can be a vector in which each dimension of the vector corresponds to the word to be considered. For example, when the stored additional information includes the word i, it is possible to extract a stored additional information feature that sets the i-th dimension of the vector to 1 and does not include the word i.

  Further, a word co-occurrence vector may be used as the accumulated additional information feature. More specifically, a vector having the d-th element of the word co-occurrence vector as the number of the d-th word included in the text label can be extracted as the accumulated additional information feature.

  The accumulated latent variable extraction unit 13 relates the accumulated video section and the accumulated additional information based on the accumulated video section feature set that is a set of accumulated video section features and the accumulated additional information feature set that is a set of accumulated additional information features. A storage latent variable derivation function, which is a function for deriving a vector (accumulation latent variable) describing sex, is obtained, and a latent variable is obtained from each of the stored video section features and each of the stored additional information features using the stored latent variable derivation function And the accumulated latent variable set, which is a set of accumulated latent variables, is extracted using the latent variables as accumulated latent variables.

  The accumulated latent variables are, for example, a method using canonical correlation analysis described in Non-Patent Document 1, a method using probabilistic canonical correlation analysis (Nakayama et al. “Image annotation / retrieval method for large-scale Web images Web set Toward autonomous image knowledge acquisition from knowledge ”, Image Recognition / Understanding Symposium MIRU2009, OS2-4, July 2009), a method using semi-supervised canonical correlation analysis described in Non-Patent Document 2 Or the like.

  For example, in the method using probabilistic canonical correlation analysis, accumulated latent variables as shown in the following equation (1) can be extracted.

  Where Λ is a diagonal matrix whose diagonal elements are singular values obtained by the canonical correlation analysis using the stored video section feature set and the stored additional information feature set as input, and Wx and Wy are the same canonical correlation analysis. Is a matrix with the left singular vector obtained by

  The accumulated state variable extraction unit 14 obtains, from the accumulated latent variable set, an accumulated state variable set that is a set of accumulated state variables that is a variable describing the distribution of accumulated latent variables, or an accumulated state variable distribution that is a distribution of accumulated state variables. Extract. A topic trajectory model is constructed from the accumulated state variable set or accumulated state variable distribution extracted here and the accumulated latent variable derivation function.

  The method of extracting the accumulated state variable set or accumulated state variable distribution is not particularly limited. For example, considering a hidden Markov model in which the output distribution is defined as a mixed normal distribution, the Viterbi algorithm is used as the E-step of the EM algorithm. Viterbi learning (see Japanese Patent Application Laid-Open No. 2009-259035) or a method in which a Forward-backward algorithm is incorporated in the E-step of the EM algorithm (see Japanese Patent Application Laid-Open No. 2009-259035) can be used.

  As shown in FIG. 2, the video recognition unit 2 further includes an input video section feature extraction unit 21, an input latent variable extraction unit 22, an input state variable extraction unit 23, an input latent variable correction unit 24, The output additional information feature estimation unit 25 and the output additional information estimation unit 26 can be represented by a configuration. The video recognition unit 2 inputs an input video set which is a set of input videos to which additional information is added by video recognition, and uses the topic trajectory model constructed by the topic trajectory model learning unit 1 to input the input video. Additional information corresponding to each is estimated and output.

  The input video section feature extraction unit 21 divides each of the input videos, which are each element of the input video set input, along the time axis by a process similar to that of the stored video section feature extraction unit 11, and outputs a plurality of inputs. Extract video segment. Then, a key frame is extracted from the frames included in the input video section, an image feature expressing the characteristic is calculated from the key frame, and this is extracted as the input video section feature.

  Based on an input latent variable derivation function based on an input video section feature set that is a set of input video section features, the input latent variable extraction unit 22 is an input latent variable set that is a set of input latent variables corresponding to the input video section features. To extract. Unlike the accumulated latent variable extraction unit 13, since the latent variable is calculated without additional information, for example, an input latent variable as shown in the following equation (2) described in Non-Patent Document 2 can be extracted.

The input state variable extracting unit 23 is an input state variable set that is a set of input state variables that are variables describing the distribution of the input latent variables based on the input latent variable set and the accumulated state variable set or accumulated state variable distribution. Or an input state variable distribution that is a distribution of the input state variables. This is done by using the Viterbi search or the Forward-backward algorithm, from the input latent variable series to the input state variable ~ s _i series ~ s = {~ s ₁ , ~ s ₂ , ...}, or its probability distribution You can ask for.

  The input latent variable correction unit 24 corrects the input latent variable set based on the input state variable set or the input state variable distribution, and newly extracts this as a corrected input latent variable set. When the input state variable ~ s is obtained by the input state variable extraction unit 23, the latent variable given in a form corresponding to the input state variable is obtained by using the topic trajectory model constructed by the topic trajectory model learning unit 1. The mixed normal distribution is determined. The corrected input latent variable ^ z is calculated from this distribution itself or its average vector.

  The output additional information feature estimation unit 25 estimates an output additional information feature that is a vector corresponding to the output additional information, based on the corrected input latent variable set and the input video section feature. When the input latent variable ^ z after correction is obtained by the input latent variable correcting unit 24, for example, the following equation (3) that is an equation for estimating additional information from the latent variable in the stochastic canonical correlation analysis, or the above ( 1) The output additional information feature is calculated based on the input latent variable and the input video section feature by the following equation (4) obtained by transforming the equation.

  Since there are a plurality of corrected input latent variables ^ z in the form of an average vector or a distribution, a plurality of output additional information features having a high possibility are obtained based on these values.

  The output additional information estimation unit 26 performs processing reverse to that of the accumulated additional information feature extraction unit 12 and estimates each of the output additional information from each of the output additional information feature sets estimated by the output additional information feature estimation unit 25. Thus, an output additional information set that is a set of output additional information is output.

  Next, with reference to FIG. 3, a recognition processing routine executed in the video recognition apparatus 10 according to the first embodiment will be described.

  In step 100, a stored video set and a corresponding stored additional information set input by input means (not shown) are acquired.

  Next, in step 102, each of the stored videos that are the elements of the stored video set acquired in step 100 is divided into a plurality of stored video sections along the time axis, and from the key frames included in the stored video sections, An image feature expressing the characteristic is calculated, and this is extracted as an accumulated video section feature.

  Next, in step 104, a stored additional information feature that is a vector representing the characteristic is extracted from each of the stored additional information that is an element of the stored additional information set acquired in step 100.

  Next, in step 106, based on the accumulated video section feature set that is a set of accumulated video section features extracted in step 102 and the accumulated additional information feature set that is a set of accumulated additional information features extracted in step 104. A stored latent variable is obtained using the obtained stored latent variable derivation function, and a stored latent variable set that is a set of stored latent variables is extracted.

  Next, in step 108, from the accumulated latent variable set extracted in step 106, the accumulated state variable set which is a set of accumulated state variables which are variables describing the distribution of accumulated latent variables, or the accumulated state variable distribution. The accumulated state variable distribution is extracted and a topic trajectory model is constructed.

  Next, in step 110, an input video set to be recognized input by an input unit (not shown) is acquired.

  Next, in step 112, each input video that is an element of the input video set is divided into a plurality of input video segments along the time axis by the same processing as in step 102, and the input video segments are divided into the input video segments. An image feature expressing the characteristic is calculated from the included key frame, and this is extracted as an input video section feature.

  Next, in step 114, the input latent variable corresponding to the input video segment feature is determined based on the input video segment feature set that is the set of input video segment features extracted in step 112 and the input latent variable derivation function. Then, an input latent variable set that is a set of input latent variables is extracted.

  Next, in step 116, the variable describes the distribution of the input latent variables based on the input latent variable set extracted in step 114 and the accumulated state variable set or accumulated state variable distribution extracted in step 108. An input state variable set that is a set of input state variables or an input state variable distribution that is a distribution of the input state variables is extracted.

  Next, in step 118, the input latent variable set extracted in step 114 is corrected from the input state variable set or the input state variable distribution extracted in step 116, and this is newly extracted as a corrected input latent variable set. .

  Next, in step 120, based on the corrected input latent variable set extracted in step 118 and the input video section feature extracted in step 112, an output additional information feature that is a vector corresponding to the output additional information is obtained. Thus, an output additional information feature set that is a set of output additional information features is estimated.

  Next, in step 122, a process reverse to that in step 104 is performed to estimate each of the output additional information from each of the output additional information feature sets estimated in step 120, and this is a set of output additional information. Output as output additional information set.

  As described above, according to the video recognition apparatus of the first embodiment, the input state variable obtained from the input latent variable based on the input video section feature describing the relationship between the input video section and the input additional information, The input latent variable is corrected based on the topic trajectory model describing the relationship between the stored video set and the stored additional information set, and added to the video section based on the corrected input latent variable and the input video section feature. Therefore, video recognition can be performed with high accuracy in consideration of time information.

  Next, a second embodiment will be described. In the second embodiment, a case will be described in which the video management apparatus of the present invention is applied to a video search apparatus that searches for a desired video based on input additional information. Note that, in the video search device of the second embodiment, the same components as those of the video recognition device of the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The video search apparatus 20 according to the second embodiment is composed of a computer including a CPU, a RAM, and a ROM storing a program for executing a search processing routine described later. As shown in FIG. 4, it can be expressed by a configuration including a topic locus model storage unit 15, a video recognition unit 2, and a video search unit 3.

  The topic trajectory model storage unit 15 stores a topic trajectory model constructed by the topic trajectory model learning unit 1 of the video recognition apparatus 10 according to the first embodiment.

  As shown in FIG. 5, the video search unit 3 can be represented by a configuration that further includes an accumulated video recognition unit 31 and an output video estimation unit 32. The video search unit 3 searches for a video that matches the input additional information (input additional information) using the topic trajectory model constructed by the video recognition device 10 of the first embodiment.

  The stored video recognition unit 31 inputs each of the stored videos as elements of the input stored video set to the video recognition unit 2 as input video, and operates each unit of the video recognition unit 2 in the same manner as in the first embodiment. Then, an output accumulation additional information set that is a set of output accumulation additional information that is the output is output.

  The output video estimation unit 32 inputs the input additional information input, the storage additional information set corresponding to the stored video set input to the stored video recognition unit 31, and the output storage additional information set output from the video recognition unit 2. By comparing them, an output video suitable for the input additional information is estimated and output.

  Next, with reference to FIG. 6, a search processing routine executed in the video search apparatus 20 of the second embodiment will be described.

  In step 200, input additional information and a stored video set input by an input unit (not shown) are acquired.

  Next, in step 202, an output accumulation additional information set output process is executed. In the output additional information set output process, steps 110 to 122 in the recognition process (FIG. 3) of the first embodiment are performed using each of the stored videos that are elements of the stored video set input in step 200 as input videos. Similar processing is executed.

  Next, in step 204, the input additional information input in step 200, the storage additional information set corresponding to the stored video set input in step 200, and the output storage additional information set output in step 202 are obtained. By comparison, an output video that matches the input additional information is estimated from the stored video set and output. The output video to be output may be one or plural. Further, ranking may be given and output according to the degree of matching with the input additional information.

  As described above, according to the video search apparatus of the second embodiment, the output accumulation additional information set for the accumulated video set estimated accurately in consideration of the time information and the accumulation addition corresponding to the accumulated video set. Since the information set and the input additional information are compared to estimate the output video that matches the input additional information, the video search can be performed with high accuracy in consideration of the time information.

  The first embodiment describes the video recognition device, and the second embodiment describes the video search device. However, in the second embodiment, the topic trajectory model storage unit is the same as that of the first embodiment. By replacing with a topic trajectory model learning unit, it can be configured as a video recognition search device capable of executing both video recognition and search.

  Further, the present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, although the above-described video recognition device and video search device have a computer system inside, if the “computer system” uses a WWW system, a homepage providing environment (or display environment) Shall also be included.

  Further, in the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 1 Topic locus model learning part 2 Image | video recognition part 3 Image | video search part 10 Image | video recognition apparatus 15 Topic locus | trajectory model memory | storage part 20 Image | video search apparatus

Claims (6)

Each of the stored video section features indicating the characteristics of the stored video section extracted from each of the stored video sections obtained by time-dividing each of the stored videos that are elements of the stored video set stored in advance in a predetermined section, and the storage Based on each of the stored additional information features indicating the characteristics of the stored additional information extracted from each of the stored additional information added corresponding to each of the video sections, the stored video section that is a set of the stored video section features The distribution of stored latent variables describing the relationship between the stored video section and stored additional information obtained using a stored latent variable derivation function based on the feature set and the stored additional information feature set that is a set of the stored additional information features. The accumulated state variable constructed by the topic trajectory model learning means for extracting the accumulated state variable set that is the set of accumulated state variables or the accumulated state variable distribution that is the distribution of the accumulated state variables. Collection or the accumulation state variable distribution, and the topic trajectory model storage means for storing a topic trajectory model consisting of the storage latent variable derivation function, an image management apparatus comprising a video recognition unit,
The image recognition means includes
Input video section feature extraction means for extracting each of the input video section features indicating the characteristics of the input video section from each of the input video sections obtained by time-dividing the input video in a predetermined section;
Using an input latent variable derivation function based on an input video segment feature set that is a set of input video segment features extracted by the input video segment feature extracting means, obtaining an input latent variable corresponding to the input video segment feature, Input latent variable extracting means for extracting an input latent variable set which is a set of the input latent variables;
This is a set of input state variables which are variables describing the distribution of the input latent variables based on the input latent variable set extracted by the input latent variable extracting means and the accumulated state variable set or the accumulated state variable distribution. An input state variable extracting means for extracting an input state variable set or an input state variable distribution which is a distribution of the input state variables;
A new corrected input latent variable set obtained by correcting the input latent variable set extracted by the input latent variable extracting unit based on the input state variable set or the input state variable distribution extracted by the input state variable extracting unit. Input latent variable correction means to be extracted;
Based on the corrected input latent variable set extracted by the input latent variable correction means and the input video section feature set extracted by the input video section feature extraction means, the output additional information feature related to the input video section feature Output additional information feature estimation means for estimating an output additional information feature set that is a set;
Output additional information estimation means for estimating output additional information corresponding to the feature indicated by the output additional information feature set estimated by the output additional information feature estimation means;
Including video management device. Each of the stored videos that are elements of the stored video set is input as the input video, and an output storage additional information set that is a set of output storage additional information is added as additional information corresponding to the stored video set by the video recognition means. Accumulated image recognition means for outputting;
Based on the input additional information input, the stored additional information set corresponding to the input stored video set, and the output stored additional information set output by the video recognition unit, the stored video set changes to the input additional information. Output video estimation means for estimating a suitable video as an output video,
The video management apparatus according to claim 1, further comprising:   The video management apparatus according to claim 1, further comprising a topic trajectory model learning unit. Each of the stored video section features indicating the characteristics of the stored video section extracted from each of the stored video sections obtained by time-dividing each of the stored videos that are elements of the stored video set stored in advance in a predetermined section, and the storage Based on each of the stored additional information features indicating the characteristics of the stored additional information extracted from each of the stored additional information added corresponding to each of the video sections, the stored video section that is a set of the stored video section features The distribution of stored latent variables describing the relationship between the stored video section and stored additional information obtained using a stored latent variable derivation function based on the feature set and the stored additional information feature set that is a set of the stored additional information features. An accumulated state variable set or accumulated state variable distribution constructed by extracting an accumulated state variable set that is a set of accumulated state variables or an accumulated state variable distribution that is a distribution of the accumulated state variables, Stores the topic trajectory model of microcrystal said storage latent variable derivation function,
Extracting each input video section feature indicating the characteristics of the input video section from each of the input video sections obtained by time-dividing the input video in a predetermined section,
An input latent variable derivation function based on an input video segment feature set, which is a set of extracted input video segment features, is used to obtain an input latent variable corresponding to the input video segment feature, and is a set of the input latent variable Extract the input latent variable set,
Based on the extracted input latent variable set and the accumulated state variable set or the accumulated state variable distribution, the input state variable set that is a set of input state variables that are variables describing the distribution of the input latent variables, or Extract the input state variable distribution, which is the distribution of the input state variables,
Based on the extracted input state variable set or the input state variable distribution, a new corrected input latent variable set obtained by correcting the input latent variable set is extracted;
Based on the extracted corrected input latent variable set and the input video segment feature set, an output additional information feature set that is a set of output additional information features related to the input video segment feature is estimated;
A video management method for estimating output additional information corresponding to a feature indicated by an estimated output additional information feature set. Each of the stored videos that are elements of the stored video set is input as the input video, and an output stored additional information set that is a set of output stored additional information is output as additional information corresponding to the stored video set,
Based on the inputted input additional information, the accumulated additional information set corresponding to the inputted accumulated video set, and the output output accumulated additional information set, an image suitable for the input additional information is output from the accumulated video set. The video management method according to claim 4, wherein the video management method is estimated as a video.   The video management program for functioning a computer as each means which comprises the video management apparatus of any one of Claims 1-3.

Images