JP2020035149A - Moving image data processing device, and program - Google Patents

Abstract

To provide a moving image data processing device capable of acquiring evaluation of moving image data with a method suppressing cost, and a program.SOLUTION: A moving image data processing device extracts the feature quantity of each still image data with each of N (N is an integer of two or more) pieces of still image information obtained by sampling moving image information included in moving image data being a processing object as an input, uses a machine learning device undergoing machine learning of image relation weight information related to the extracted feature quantity corresponding to each of the N pieces of still image information, multiplies the image relation weight information obtained by undergoing the machine learning to accumulate each feature quantity, and outputs a result of the accumulation as a feature quantity of the moving image information included in the moving image data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a moving image data processing device and a program.

  Conventionally, in order to evaluate the effect of moving image data, for example, an advertisement image (commercial film: CF), it is necessary to actually present the image to a viewer and obtain the impression by a questionnaire. Did not get.

  In addition, for example, in an advertisement video, the audience rating of a program being broadcast at the time of presenting an advertisement or the number of advertisements may be used as an evaluation index in order to simply represent the effect.

Z. Hussain, et.al., “Automatic understanding of image and video advertisements”, In CVPR, 1100-1110, 2017.2, 4

  However, in the above-mentioned conventional method, for example, in the method using a questionnaire, it is costly to obtain information from viewers who are subject to a questionnaire. I was

  Further, there is a problem that a simple evaluation index does not always have a large correlation with an actual evaluation result.

  Under such a background, there is a demand for a technique for obtaining evaluation of moving image data by a cost-saving method before presenting the evaluation to a viewer. Non-Patent Document 1 discloses that topic analysis of an advertisement image is performed.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a moving image data processing apparatus and a program that can obtain evaluation of moving image data in a cost-effective manner. .

  One embodiment of the present invention for solving the problems of the above-described conventional example is a moving image data processing device, which includes N moving image data obtained by sampling from moving image information included in moving image data to be processed. Receiving means for receiving each of the still image information (N is an integer of 2 or more), feature extracting means for receiving the received N pieces of still image information as input, and extracting a feature amount of each still image information; Using the machine learning device in a state where the extracted image relation weight information relating to the feature amount corresponding to each of the extracted N still image information obtained by sampling is machine-learned, the machine learning is performed. Accumulating means for accumulating each feature amount by multiplying the obtained image relation weight information, and output means for outputting the result of the accumulation as a feature amount of moving image information included in the moving image data, The movie Feature amount information is obtained by the fact to be subjected to a predetermined processing related to the evaluation of the moving image data as the object of processing.

  ADVANTAGE OF THE INVENTION According to this invention, evaluation of moving image data can be obtained by the method which suppressed cost.

FIG. 1 is a block diagram illustrating a configuration example of a moving image data processing device according to an embodiment of the present invention. 1 is a functional block diagram illustrating an example of a moving image data processing device according to an embodiment of the present invention. It is a functional block diagram showing another example of the moving image data processing device according to the embodiment of the present invention.

  An embodiment of the present invention will be described with reference to the drawings. A moving image data processing device 1 according to an embodiment of the present invention includes a control unit 11, a storage unit 12, an operation unit 13, an output unit 14, and an interface unit 15, as illustrated in FIG. .

  Here, the control unit 11 is a program control device including at least one CPU, a GPU (Graphics Processing Unit), and the like, and operates according to a program stored in the storage unit 12. In the present embodiment, the control unit 11 receives an input of moving image data to be processed, and obtains N (N is 2 or more) obtained by sampling from moving image information included in the received moving image data. ) Of still image information.

  Specifically, the control unit 11 extracts and samples the still image information at predetermined timings from the moving image information (consisting of still image information forming a series of frames). Here, the predetermined timing may be, for example, to extract one frame of still image information per second.

  The control unit 11 extracts each feature amount of the sampled N pieces of still image information. This feature amount may be, for example, the output itself of the neural network when still image information corresponding to the neural network that has been machine-learned in advance is input.

  The control unit 11 further multiplies the image relationship weight information obtained by the machine learning by using a machine learning device in which the image relationship weight information related to the feature amount of the still image information obtained here is machine-learned. To calculate each feature amount. Then, the control unit 11 outputs the result of the accumulation as a feature amount of the moving image information included in the moving image data.

  In one aspect of the present embodiment, the feature amount of the moving image information obtained here is subjected to predetermined processing relating to the evaluation of the input moving image data. The operation of the control unit 11 will be described later in detail.

  The storage unit 12 is a memory device or the like, and stores a program executed by the control unit 11. This program may be stored in a non-transitory computer readable recording medium and provided, and may be copied to the storage unit 12. The storage unit 12 also operates as a work memory of the control unit 11.

  The operation unit 13 is a keyboard, a mouse, or the like, and accepts a user's instruction operation and outputs it to the control unit 11. The output unit 14 is a display or the like, and outputs information according to an instruction input from the control unit 11.

  The interface unit 15 is a serial interface such as a USB (Universal Serial Bus), a network interface, or the like, and receives moving image data from an external device or the like and outputs the moving image data to the control unit 11.

  Next, the operation of the control unit 11 will be described. Functionally, the control unit 11 of the present embodiment includes a still image information receiving unit 21 and a plurality of feature amount extractors 22-1, 22-2,..., 22-N as illustrated in FIG. , An image relation weight multiplying unit 23, an accumulating unit 24, an audio feature amount extracting unit 25, a metadata feature amount extracting unit 26, a second weight multiplying unit 27, and an output control unit 29. Is done.

  The still image information receiving unit 21 receives and holds N pieces of still image information obtained by sampling the moving image data to be processed at predetermined timings (for example, every second), and stores the N pieces of still images. The i-th still image information among the information is output to the corresponding feature amount extractor 22-i. In the present embodiment, the moving image data includes moving image information and audio information.

  Each of the feature amount extractors 22-i (i = 1, 2,... N) extracts the feature amount of the i-th still image information from the N pieces of still image information received by the still image information receiving unit 21. . In an example of the present embodiment, the feature amount extractor 22-i is a neural network having at least one hidden layer, and is configured to output a feature amount using a plurality of predetermined image data in advance. It is assumed that

  Specifically, the feature amount extractor 22-i performs resnet (Kaiming He, et. Al.) Machine-learned using ImageNet (http://www.image-net.org/) which is a set of predetermined image data. .al., Deep Residual Learning for Image Recognition, arXiv: 1512.03385) or distilling the resnet machine-learned using imagenet (the output when the same data as resnet was input was machine-learned using the resnet output as a teacher) It may be a fully connected neural network with one hidden layer. It should be noted that each feature amount extractor 22-i in this example uses the same neural network (the same machine learning result).

  In this example, the input of the feature amount extractor 22-i is an image of a predetermined size (for example, an image of 224 × 224 pixels), and the output is, for example, 256-dimensional vector information. This output corresponds to the feature amount of the still image information of the present invention.

  The image relation weight multiplying unit 23 outputs the feature amounts fi (i = 1, 2,...) Of the N pieces of still image information output from the N feature amount extractors 22-i (i = 1, 2,..., N). , N), the image relation weight information α is obtained using a machine learning device in a state in which the method of calculating the image relation weight information αi (i = 1, 2,..., N) is machine-learned.

  Further, the image relation weight multiplying unit 23 applies the feature amount fi of the N pieces of still image information output from the N feature amount extractors 22-i (i = a, b,... The value αifi obtained by multiplying the corresponding image relation weight information αi obtained by using the learning device is output. A method of machine learning the calculation method of the image relation weight information of the image relation weight multiplication unit 23 will be described later.

  The accumulating unit 24 accumulates the weighted feature amounts αifi output from the image-related weight multiplying unit 23, and sets the accumulation result 累 αifi as the feature amount Fframe of the still image information included in the moving image information. Output. The feature amount Fframe of the still image information is represented by 256-dimensional vector information in an example of the present embodiment.

  The audio feature amount extraction unit 25 receives the input of audio information included in the moving image data from the moving image data to be processed, and extracts the characteristic amount of the audio information. In one example of the present embodiment, the feature amount of the voice information is an output when the voice information is input to a neural network that has been machine-learned in advance. Specifically, the audio feature amount extraction unit 25 converts the soundnet (Yusuf Aytar, et. /projects/urbansounddataset/urbansound8k.html) and other neural networks obtained by machine learning using a predetermined data set. Note that the audio information does not necessarily have to be used, and in that case, the audio feature amount extraction unit 25 is not necessarily required.

  In this example, the audio feature amount extraction unit 25 outputs an output when audio information included in moving image data to be processed is input to the neural network as a feature amount Fsound of the audio information.

  The metadata feature extraction unit 26 extracts the feature of the additional information (metadata) separately input by the user from the moving image data to be processed. In an example of the present embodiment, this additional information is, for example, information in which a survey date, an age group and a gender (of a main assumed viewer of moving image data), a title, and a narration voice are written as characters, Information indicating the category of the product being advertised (if it is an ad), information indicating whether it is a series or not, the provider (for the moving image data) (the advertiser for the ad), (for the ad, the ad Product name), service name, and other information. However, these are merely examples, and the metadata may not always be used. In that case, the metadata feature extraction unit 26 is not necessarily required.

  The metadata feature extraction unit 26 can also be realized using a neural network (neural network for metadata) machine-learned in advance by a predetermined method. Specifically, the metadata feature extracting unit 26 inputs the vector information representing the metadata to the metadata neural network, and outputs the output as the metadata feature Fmeta.

  In one example of the present embodiment, it is assumed that the feature amount Fsound of the audio information output by the audio feature amount extraction unit 25 is 256-dimensional vector information, and the metadata feature amount Fmeta is also 256-dimensional vector information ( Vector information having the same dimension as the feature amount Fframe of the still image information).

  The second weight multiplication unit 27 outputs the feature amount Fframe of the still image information output by the accumulation unit 24, the feature amount Fsound of the audio information output by the audio feature amount extraction unit 25, and the metadata feature amount extraction unit 26. A value F = β1Fframe + β2Fsound + β3Fmeta obtained by multiplying each of the metadata weights Fmeta and the second weight information β1, β2, β3 by each of them is output as the estimated feature quantity for the moving image data.

  In one example of the present embodiment, the second weight multiplying unit 27 calculates the second weight of each of the feature amount Fframe of the still image information, the feature amount Fsound of the audio information, and the metadata feature amount Fmeta. The method of calculating the information β1, β2, β3 is machine-learned in advance, and the information β1, β2, β3 of the second weight is calculated each time according to the input information (moving image data including audio information or metadata). It may be calculated. The machine learning method of calculating the second weight information β1, β2, β3 will be described later.

  The output control unit 29 obtains predetermined result information (score) S based on the feature amount F of the moving image information output from the second weight multiplication unit 27, and outputs the obtained result information to the output unit 14. Here, the output control unit 29 receives, for example, vector information having the same dimension as the dimension of the vector of the feature amount F of the moving image information, and outputs vector information having dimensions equal to the number of result information (scores) S. It can be realized using a network. This neural network may be a fully connected neural network. The method of machine learning of this neural network will be described later.

  In an example of the present embodiment, the score (result information) output by the output control unit 29 is the ratio (recognition) of the moving image data to be processed by the subject, the moving image to be processed. The percentage of data that have a favorable impression (favorableness), the percentage of consumers who want to purchase the advertised product or service when the moving image data that was processed is an advertisement (the degree of purchase arousal), It contains four values of the ratio (interest / interest level) at which the changed moving image data has interest / interest. These values do not necessarily need to be represented as ratios (values of 0 or more and 1 or less), but a larger numerical value means that the ratio represented by each value is higher. In this case, the dimensions of the vector output by the neural network used by the output control unit 29 are four dimensions.

  Next, a method of machine learning of a neural network provided in each unit will be described. In the present embodiment, as in the example described above, the plurality of feature amount extractors 22 are in a state in which machine learning is performed in advance using a data set of predetermined still image data. Similarly, the audio feature amount extraction unit 25 is also set to a state in which machine learning is performed using a data set of predetermined audio data. The result of the machine learning may not be updated in the following processing, or may be updated.

  That is, each feature amount extractor 22 outputs the result information output by the moving image data processing device 1 when the moving image data whose result information is known (or can be set) is input to the moving image data processing device 1. And may be updated and set by back propagation based on the difference between the input moving image data and the result information known or set in advance (hereinafter referred to as teacher result information for distinction). . The weight of the neural network of each feature amount extractor 22 may be common.

  On the other hand, as for the weights to be multiplied by the image relation weight multiplying unit 23 and the second weight multiplying unit 27, and for the neural networks included in each of the metadata feature amount extracting unit 26 and the output control unit 29, the result information is known in advance. When moving image data (or metadata) that is (or can be set) is input to the moving image data processing device 1, the result information output by the moving image data processing device 1 and the input moving image data are known in advance. This is updated and set by back propagation based on the difference from the existing (or set) result information (hereinafter referred to as teacher result information for distinction). The accumulating unit 24 performs accumulation, and the image-related weight multiplying unit 23 and the second weight multiplying unit 27 only perform weight multiplication and accumulation. Gating processing can be performed.

  That is, in an example of the present embodiment, a set of a plurality of pieces of moving image data for which result information (teacher result information) is known in advance is prepared. Here, for example, the recognition degree, the favorable impression, the purchase arousal degree, and the interest / interest degree, which are examples of the teacher result information on the plurality of moving image data, are determined in advance by a known test (for example, so-called A / A comparative test such as a B test may be used).

  The user inputs each of the moving image data included in the set of moving image data to the moving image data processing apparatus 1 as teacher data, and outputs the corresponding result information as the corresponding output and the moving image data as the input teacher data. And a method for calculating a weight to be multiplied by the image-related weight multiplying unit 23 and the second weight multiplying unit 27 by a known back propagation using a difference from the teacher result information set corresponding to Machine learning is performed on the neural networks provided in each of the quantity extraction unit 26 and the output control unit 29. Thereby, the calculation method of the weight multiplied by the image-related weight multiplying unit 23 and the second weight multiplying unit 27 and the weight of the neural network of each unit are machine-learned.

  Thereafter, the user inputs the moving image data to be evaluated to the moving image data processing device 1 as a processing target. Then, the result information as the output is obtained, and information on the degree of recognition, liking, purchase arousal, and interest / degree of interest of the moving image data to be processed is obtained.

  The moving image data processing apparatus 1 according to the embodiment of the present invention has the above configuration and operates as follows. In the following example of the present embodiment, a plurality of feature amount extractors 22-1, 22-2,..., 22-N are all neural networks in which the same weight is set. That is, each feature amount extractor 22 performs the same output when the same data is input.

  It is also assumed that the speech feature extraction unit 25 has been machine-learned in advance using a widely known data set such as UrbanSound8K.

  In this state, each of the moving image data included in the set of a plurality of moving image data for which the result information (teacher result information) is known in advance is input to the moving image data processing apparatus 1 as teacher data, and the corresponding output is output. Using the difference between certain result information and the teacher result information set corresponding to the input moving image data as the teacher data, the image-related weight multiplying unit 23 and the second weight multiplying unit are formed by known back propagation. 27, a neural network representing a calculation method of weights αi (i = 1, 2,..., N), β1, β2, and β3; a feature amount extractor 22, a metadata feature amount extraction unit 26, and an output control unit Machine learning is performed on the neural network provided in each of 29 and 29.

  Next, the user inputs the moving image data to be actually evaluated into the moving image data processing device 1 as a processing target. When the moving image data processing device 1 reproduces the moving image information included in the moving image data to be processed, the reproduction time is displayed at the positions of 0 second, 1 second, 2 seconds,. The extracted N pieces of still image information are extracted (sampling).

  The control unit 11 of the moving image data processing apparatus 1 accepts the N pieces of still image information and assigns the corresponding still image information to each of the plurality of feature amount extractors 22-i (i = 1, 2,... N). Enter For example, the still image information displayed at the position where the reproduction time is i seconds is input to the feature amount extractor 22-i.

  Then, the feature quantity extractor 22-i outputs an output fi (i = 1, 2) of a neural network (here, ResNet trained by ImageNet or a neural network obtained by distilling it) corresponding to the input still image information. ,..., N) are output. This output may use a vector value output to the classifier.

  In addition, the moving image data processing device 1 extracts audio information included in the moving image data to be processed and inputs the audio information to the audio feature amount extraction unit 25.

  Then, the audio feature extraction unit 25 outputs an output when the input audio information is input to a neural network that has been machine-learned in advance, as an audio information feature Fsound. For this output, a vector value output to the classifier may be used.

  Further, the user inputs metadata about the moving image data to be processed. Here, the metadata is, for example, a survey date, the age group and gender of the main assumed viewer of the moving image data, the title, the narration voice described as characters, information indicating the category of the product advertised by the moving image data, The information may be information indicating whether or not it is a series, information specifying a poster of an advertisement, which is a provider of moving image data, and the name of a commercialized product or service.

  When receiving the input of the metadata, the control unit 11 of the moving image data processing device 1 functions as a metadata feature amount extraction unit 26, and converts the metadata into a neural network (neural network for metadata) that has been machine-learned by a predetermined method in advance. The output when the metadata is input is output as the metadata feature amount Fmeta.

  The control unit 11 further functions as an image relation weight multiplying unit 23, and the feature amounts fi (N) of the N still image information output from the N feature amount extractors 22-i (i = 1, 2,..., N). The image relation weight information αi (i = 1, 2,..., N) corresponding to each of i = 1, 2,..., N) is obtained by a calculation method obtained by machine learning. αifi, which is a value obtained by multiplying αi by the corresponding feature quantity fi, is output. The control unit 11 accumulates the respective feature amounts αifi multiplied by the weights here as the accumulation unit 24, and outputs the accumulation result Σαifi as the feature amount Fframe of the still image information included in the moving image information.

  The control unit 11 also functions as a second weight multiplication unit 27, and calculates a second weight of each of the feature amount Fframe of the still image information, the feature amount Fsound of the audio information, and the metadata feature amount Fmeta. Information β1, β2, β3 is obtained by a calculation method obtained by machine learning, and values β1, Fframe, β2Fsound, and β3Fmeta are obtained by multiplying the second weight information β1, β2, β3 by respective corresponding feature amounts. . Then, the control unit 11 calculates β1Fframe + β2Fsound + β3Fmeta as a result of the accumulation of the output. The control unit 11 outputs the result of the accumulation as the feature value F of the moving image information.

  The control unit 11 further functions as an output control unit 29, inputs the feature amount F of the moving image information to the machine-learned neural network, and outputs the output of the neural network as result information (score) S. .

  The score S, which is the output, corresponds to the teacher result information used when the neural network or the like of the output control unit 29 has been machine-learned. Is a vector value having as an element.

  The user obtains the output of the recognition level, the preference level, the purchase arousal level, and the interest / degree of interest as the information on the evaluation of the moving image data to be processed.

[Evaluation of each frame included in video information]
Furthermore, the moving image data processing device 1 of the present embodiment may evaluate at least a part of the N pieces of still image information obtained by sampling the moving image data to be processed. Here, it is assumed that the influence on the result information output from the moving image data processing device 1, such as the degree of recognition, liking, degree of purchase arousal, and degree of interest, is evaluated.

  In this example, the still image information receiving unit 21 of the control unit 11 further includes a selection instruction unit 31, a feature amount comparison unit 32, and a comparison output unit in addition to the functional configuration of FIG. 33.

  In the control section 11 of this example, the still image information receiving section 21 receives and holds N pieces of still image information obtained by sampling the moving image data to be processed at predetermined timings (for example, every 1 second). I do.

  Then, the still image information receiving unit 21 outputs the i-th still image information to the corresponding feature amount extractor 22-i. The feature amount comparison unit 32 stores the score S output from the output control unit 29 at this time as the score S of the moving image information.

  Next, the selection instructing unit 31 selects at least one piece of still image information to be evaluated from among the N pieces of still image information held by the still image information receiving unit 21 (hereinafter, M pieces of information (M is M <M < An instruction is output to select (N is one or more integers) still image information). Here, the selection of the still image information to be evaluated may be performed according to an instruction from the user, or M pieces (for example, one by one from the first) different from each other may be selected in a predetermined order. The following processing may be repeated.

  The still image information receiving unit 21 converts the designated M still image information (M is an integer of 1 or more where M <N) into a predetermined test still image in accordance with the instruction output from the selection instruction unit 31. Replace with information. Here, the test still image information may be still image information in which the entire surface is filled with a predetermined color such as black. At this time, the weights α, β1, β2, and β3 are not changed (fixed) from the values when the stored score S is calculated.

  Then, the still image information receiving unit 21 outputs the i-th still image information to the corresponding feature amount extractor 22-i. At this time, for the still image information for which the above selection has not been made among the feature amount extractors 22, the feature amount of the still image information sampled from the moving image data to be processed is output. . In the feature amount extractor 22, the still image information selected above is replaced by the still image information sampled from the moving image data to be processed, and the feature of the test still image information is changed. Will output the quantity. The feature amount comparison unit 32 extracts the score S of the moving image information output by the output control unit 29 at this time as a temporary score S ′, and calculates a difference ΔS = S′−S from the stored score to perform comparison. Output to the output unit 33.

  The comparison output unit 33 uses the difference ΔS obtained here as the evaluation value of the selected still image information, and information for specifying the still image information selected at that time (the number of still image information that has been selected). Is output to the output unit 14 and presented to the user.

  According to this example of the present embodiment, the difference between the provisional score S ′ when the selected still image information is not included and the score S when it is included, that is, by including the selected still image information The increase (or decrease) amount ΔS of the evaluation is indicated as the evaluation value.

  Also, according to the example of selecting one by one in order from the first and repeating the processing, a provisional score S ′ when each of the N pieces of still image information sampled from the moving image information is not included, and a score when each is included, The difference from S, that is, for each still image information, the increase (or decrease) amount ΔS of the evaluation due to the inclusion of the still image information is indicated as the evaluation value.

[Configuration of control unit]
In the present embodiment, a plurality of feature amount extractors 22-1, 22-2,..., 22-N, an image relation weight multiplying unit 23, an accumulating unit 24, an audio feature amount extracting unit 25, Each unit such as the metadata feature amount extraction unit 26, the second weight multiplication unit 27, and the output control unit 29 may be realized by one control unit 11 sequentially performing corresponding processing, The unit 11 may include a plurality of GPUs and the like, and each GPU may operate in parallel to perform processing.

[Effects of Embodiment]
According to the present embodiment, for example, when moving image data for advertisement is generated, it is possible to know the degree of advertisement effect without actually broadcasting it and without requiring work such as a questionnaire. . Therefore, evaluation of moving image data can be obtained in a cost-reduced manner.

  Further, since it is possible to estimate which scene image of the moving image information is effective in a preferable direction and effective in an unfavorable direction, it is possible to easily examine the moving image information.

REFERENCE SIGNS LIST 1 moving image data processing device, 11 control unit, 12 storage unit, 13 operation unit, 14 output unit, 15 interface unit, 21 still image information receiving unit, 22 feature quantity extractor, 23 image relation weight multiplication unit, 24 accumulation Unit, 25 audio feature amount extraction unit, 26 metadata feature amount extraction unit, 27 second weight multiplication unit, 29 output control unit, 31 selection instruction unit, 32 feature amount comparison unit, 33 comparison output unit.

Claims (5)

Receiving means for receiving each of N (N is an integer of 2 or more) still image information obtained by sampling from moving image information included in moving image data to be processed;
A feature extraction unit that receives the received N pieces of still image information as input, and extracts a feature amount of each piece of still image information;
Using the machine learning device in a state in which the image relation weight information relating to the feature amount corresponding to each of the extracted still image information obtained by the sampling is machine-learned, Accumulating means for accumulating each feature amount by multiplying the obtained image relation weight information;
Output means for outputting the result of the accumulation as a feature amount of moving image information included in the moving image data,
Including
A moving image data processing device, wherein the feature amount of the moving image information is used for a predetermined process related to evaluation of the moving image data to be processed. The moving image data processing device according to claim 1,
The processing target moving image data further includes audio information,
Audio feature amount extraction means for extracting a feature amount of audio information included in the moving image data subjected to the processing,
Using a second machine learning device in a state in which information of a second weight corresponding to each of the feature amount of the moving image information and the feature amount of the audio information is machine-learned; Second accumulating means for accumulating each feature amount by multiplying the information of the second weight obtained through
Further comprising
A moving image data processing device in which a result of the accumulation is subjected to predetermined processing relating to evaluation of the moving image data to be processed. The moving image data processing device according to claim 2,
Means for receiving an input of metadata regarding the moving image data that has been processed,
Metadata feature amount extraction means for extracting a feature amount of metadata related to the moving image data subjected to the processing,
The second accumulating means sets a state in which information of a second weight corresponding to each of the feature amount of the moving image information, the feature amount of the audio information, and the feature amount of the metadata is machine-learned. Using a certain second machine learning device, multiplying information of the second weight obtained through the second machine learning device to accumulate each feature amount,
A moving image data processing device in which a result of the accumulation is subjected to predetermined processing relating to evaluation of the moving image data to be processed. The moving image data processing device according to any one of claims 1 to 3,
Further, M still image information (M is an integer of 1 or more where M <N) selected from the N pieces of still image information obtained by sampling is converted into a predetermined test still image. Means for inputting to the accumulating means, multiplying by the image relation weight information obtained by the machine learning to obtain a provisional feature amount by accumulating each feature amount,
Means for calculating and outputting an evaluation value of the selected still image information based on a difference between the feature amount of the moving image information and the provisional feature amount,
A moving image data processing device further including: Computer
Receiving means for receiving each of N (N is an integer of 2 or more) still image information obtained by sampling from moving image information included in moving image data to be processed;
A feature extraction unit that receives the received N pieces of still image information as input, and extracts a feature amount of each piece of still image information;
Using the machine learning device in a state in which the image relation weight information relating to the feature amount corresponding to each of the extracted still image information obtained by the sampling is machine-learned, Accumulating means for accumulating each feature amount by multiplying the obtained image relation weight information;
Output means for outputting the result of the accumulation as a feature amount of moving image information included in the moving image data,
Function as
A program in which a feature amount of the moving image information is subjected to predetermined processing related to evaluation of the moving image data that has been processed.

Images