JP2015207038A - Video image evaluation device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To evaluate a video image system taking into consideration of video image contents.SOLUTION: A video image evaluation device 100 comprises: subject evaluation similarity calculation means 113 that calculates similarity between a subject evaluation vector yabout all items of a plurality of evaluation axes about a plurality of subjects and a subject evaluation similarity vector yhaving only a specific evaluation item of the evaluation axis selected and calculates a subject evaluation similarity vector s; mapping selection means 120 that selects mapping information associating bio-signal similarity between the subjects with subject evaluation similarity between the subjects; and subject evaluation corresponding scene estimation means 114 that obtains an estimation value of the vector xso as to establish a relationship expression associating a bio-signal similarity vector sserving as an integration of a time-series bio-signal vector of the plurality of subjects and an unknown time-series bio-signal vector xand the subject evaluation similarity sby the mapping information, and determines a scene of a video image on the basis of the estimation value.

Description

  The present invention relates to a video evaluation apparatus that objectively evaluates a video system, and more particularly to a video evaluation apparatus that evaluates a video content presentation apparatus and a program thereof.

  Conventionally, in order to evaluate a video system, subjective evaluation has been performed in which an evaluator who has viewed video content presented using the video system gives an evaluation value to one or a plurality of evaluation items after viewing. . However, as the quality of video systems increases, the result of subjective evaluation tends to be more strongly dependent on the content of video content. Therefore, it is very difficult to separately handle whether the result of the subjective evaluation is an evaluation result for the video system or an evaluation result for the video content displayed using the video system.

  In particular, since the subjective evaluation of the video system is mainly performed after viewing the video content, the evaluation value tends to be for the entire video content rather than a specific scene of the video content. Therefore, it is in principle difficult to specify which scene of the video content has a great influence on the subjective evaluation. For example, in order to identify which scene has a large influence on the subjective evaluation, a method of performing a subjective evaluation for a short time by segmenting video content in time can be considered. However, with this method, it is possible to search for a scene that causes subjective evaluation, but there are the following two problems. First, performing subjective evaluation every short time is a great effort. Second, the video content cannot be viewed in a natural manner because the video content is cut into pieces for each evaluation item or evaluation question. As described above, it is difficult to evaluate a video system as a video content presentation device. The video content refers to content such as a broadcast program or a movie. When an image other than video content, for example, a still image or moving image sample (for example, a sample image of a natural plant or landscape) for confirming image quality, is simply displayed, it is called an image display device and is distinguished from the video content presentation device. .

Conventionally, as one of subjective evaluation methods, a continuous quality evaluation method that is performed simultaneously with viewing of video content is known. In this method, the evaluation axis (evaluation scale) is often a simple one that is not related to the content of the video content itself, such as whether the image quality is good or bad. That is, it is used for evaluation of a video system mainly as an image display device.
On the other hand, when the video system is evaluated mainly as a video content presentation device, assuming that the continuous quality evaluation method is used, it is necessary to change the evaluation axis as follows. In other words, it is necessary to change to an evaluation axis that has further entered into the content of the video content, for example, an evaluation axis such as sensitivity, presence, three-dimensionality, and powerfulness. However, these evaluation axes are more complex than evaluation axes such as image quality, and it is very difficult for the subject to continuously perform appropriate evaluations.

  In addition, in order to evaluate the video system under more natural viewing conditions without processing or operating the video content into pieces, the viewer's biological signals (such as brain waves and center of gravity fluctuations) are measured. There is a method for examining whether or not a predetermined pattern appears in the biological signal being viewed (see, for example, Patent Documents 1 to 9).

Japanese Patent No. 4370209 Japanese Patent No. 4441345 Japanese Patent No. 4686299 Japanese Patent No. 3991066 Japanese Patent No. 4189440 Japanese Patent No. 5119375 Japanese Patent No. 3048918 JP 2012-73350 A Japanese Patent No. 2772413

  However, the conventional technique using a biological signal pattern cannot be applied to a wide variety of video contents because the biological signal pattern is fixed. In general, there are various types of video content, and it is composed of various scenes. Therefore, various evaluation axes should be provided. It is difficult to preset characteristic biosignal patterns corresponding to such various evaluation axes. Even if only biosignals are used as in the prior art, evaluation of video systems considering the contents of video content It was difficult. In other words, the conventional method cannot evaluate the video system mainly as a video content presentation device.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a video evaluation apparatus capable of evaluating a video system in consideration of the contents of video content and a program thereof.

  In order to solve the above-mentioned problem, the video evaluation apparatus according to the present invention measures the video content presented in the video system simultaneously from a plurality of subjects who are viewing each of a plurality of measurement channels indicating any part of the body. Evaluation on a predetermined evaluation axis based on the biometric signal data of each subject in the series and the evaluation result data of each subject that is a response that the plurality of subjects subjectively evaluated the video content using a plurality of evaluation axes A video evaluation apparatus that evaluates the video system by estimating a scene of video content that has contributed to a subjective evaluation vector creation means, a subjective evaluation similarity calculation means, a mapping selection means, and a subjective evaluation corresponding scene estimation Means.

In such a configuration, the video evaluation device accepts selection of a specific evaluation item on the predetermined evaluation axis among the plurality of evaluation axes in order to estimate a scene corresponding to the predetermined evaluation axis by the subjective evaluation vector creation unit, A subjective evaluation vector is generated that numerically represents that only the specific evaluation item has been selected among all items of the plurality of evaluation axes.
Then, the video evaluation apparatus calculates the similarity between the evaluation result data regarding all items of the plurality of evaluation axes and the subjective evaluation vector for N subjects by the subjective evaluation similarity calculation means, and calculates an N-dimensional vector. A subjective evaluation similarity vector is calculated.
And the mapping memory | storage means provided in the inside or the exterior of an image evaluation apparatus is the said biosignal similarity between test subjects calculated for every said measurement channel using the data of each biosignal of said N test subjects, Mapping information calculated in advance as a matrix for associating the subjective evaluation similarity between the subjects calculated by combining the evaluation result data for the N subjects using the data of the biological signal for each measurement channel. I remember it. Then, the video evaluation apparatus selects mapping information for at least one measurement channel from the mapping storage unit by the mapping selection unit. The mapping information for the selected measurement channel is used by the subjective evaluation corresponding scene estimation means.
Then, the subjective evaluation corresponding scene estimation means is an N-dimensional product that is a product of a matrix that is data of time-series biosignals of each of the N subjects and an unknown vector whose elements are predicted time-series biosignals. The biometric signal similarity vector consisting of the vector and the subjective evaluation similarity vector obtained from the subjective evaluation vector in which only the specific evaluation item is selected are related by mapping information about the selected measurement channel A time series estimation value of the biological signal is obtained by performing an operation for determining each element of the unknown vector so that a relational expression is established, and the obtained value is obtained as a scene contributing to the evaluation on the received predetermined evaluation axis. A video scene is determined based on the time-series estimated value of the biological signal.

  The video evaluation apparatus according to the present invention is a video evaluation program for causing a computer of the video evaluation apparatus to function as subjective evaluation vector creation means, subjective evaluation similarity calculation means, mapping selection means, and subjective evaluation corresponding scene estimation means. Can be operated.

The present invention has the following excellent effects.
According to the present invention, mapping information (correspondence relationship) that associates the similarity between biological signal time-series data between multiple subjects and the similarity between subjective evaluation patterns between multiple subjects is used to relate to a specific subjective evaluation value. The scene of the video content that contributed to the evaluation on the predetermined evaluation axis of the subjective evaluation can be estimated by finding the pattern that becomes the basis of the time series of the biological signal to be performed. That is, it is possible to estimate what scene of the video content the subject gave the evaluation value to. Thereby, it is possible to objectively understand how the video content is received by the subject. In addition, it is possible to grasp a scene in which the features of the video system that presents video content effectively appear. Therefore, according to the present invention, the video system can be evaluated in consideration of the content of the video content.

It is a block block diagram which shows the structure of the image | video evaluation apparatus which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the outline | summary of the image | video evaluation apparatus which concerns on embodiment of this invention. It is an example of the evaluation axis | shaft used with the image | video evaluation apparatus which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the image | video evaluation apparatus which concerns on embodiment of this invention.

Embodiments of the present invention will be described below with reference to the drawings.
[Outline of video evaluation system]
First, an overview of the video evaluation apparatus 100 according to the embodiment of the present invention will be described with reference to FIGS. 1, 2, and 3.
The video evaluation apparatus 100 includes time-series biometric signals (observation data) 101 obtained by simultaneously measuring video content presented by a video system (not shown) from a plurality of viewing subjects, and these subjects perform a plurality of evaluations. Based on the subjective evaluation result 102 data of each subject, which is an answer obtained by subjectively evaluating the video content using the axis, the video system (invalidity) is estimated by estimating the scene of the video content that contributed to the evaluation on the predetermined evaluation axis (Shown) is evaluated.

  The video evaluation apparatus 100 is used to evaluate the video system mainly as a video content presentation apparatus in order to evaluate how the video is received by the subject. Here, the video system is, for example, a direct irradiation type video presentation method that is presented on a display monitor such as a liquid crystal display or an organic EL, or a reflection projection type video presentation method that uses, for example, a screen or a projector. This is a system that comprehensively includes a video presentation method (necessary / unnecessary for dedicated glasses), various hardware to be used, an audio system, video content to be presented (type, content, purpose, production effect) and the like. The video content is content such as a broadcast program or a movie.

  When the subject is viewing the video content, the biological signal 101 is simultaneously measured for each of a plurality of measurement channels indicating any part of the body by a biological signal measuring device (not shown). The biological signal 101 shown in FIGS. 1 and 2 is a signal reflecting the state of the biological tissue of the subject and information when the tissue is functioning. In the present embodiment, as an example, it is assumed that the biological signal is measurement data of the amount of oxygen in the cerebral blood by functional nuclear magnetic resonance imaging (fMRI). The measurement data of the cerebral blood oxygen amount by fMRI can be expressed as luminance data of an image measured by the fMRI apparatus.

  A measurement channel distinguishes biological signals measured simultaneously from one subject, and corresponds to a measurement electrode on the measurement device side such as a probe such as a measurement electrode or a measurement site on the subject side such as the brain. doing. In the measurement data of the cerebral blood oxygen amount by fMRI, the measurement channel corresponds to a portion of the subject's brain expressed in pixel units (more precisely, voxel units because they are three-dimensional images). In the fMRI apparatus, the standard number of voxels is 122880 (= length 64 × width 64 × height 30), of which the number of voxels corresponding to the gray matter of the brain (where the nerve cells are located) is empirical. It is known that there are about 60000-70000 pieces. Furthermore, since analysis is performed by focusing on a voxel with information or a specific brain region, the number of measurement channels may be set to 10000 or less (lower limit is, for example, several hundreds).

The evaluation axis is composed of, for example, two different adjective pairs, one positive adjective and its negative expression. For example, it may be a multi-stage impression evaluation as used in the SD (Semantic Differential) method. FIG. 3 shows 22 evaluation axes as examples of evaluation axes applicable to the evaluation of the video system. For example, the evaluation axis described at the top in FIG. 3 represents the evaluation axis used when the subject answers the questionnaire such as the next question Q1 after viewing the video content.
Question Q1: Please rate whether this video content feels “hard” or “soft”.
("Hard": 0, "soft": 1)

  The video evaluation apparatus 100 in FIG. 1 estimates how video content and scene contributed to the evaluation of a specific evaluation item on a predetermined evaluation axis, so that the video is accepted. It can be evaluated whether it is.

In the mathematical expression described below, the biological signal (observation data) 101 of a certain measurement channel c of a subject i illustrated in FIG. 2 is represented as a vector x having T elements in time series corresponding to the video content scale. and in association with the ^c _i.
Further, in the mathematical expression in the description below, the subjective evaluation result 102 of a subject i illustrated in FIG. 2 is associated with a vector y _i having the same number of elements as the number N _{q of} evaluation axes. The subjective evaluation result 102 illustrated in FIG. 2 includes an evaluation axis a having an evaluation item “bright” and an evaluation item “not bright”, an evaluation item “classy”, and an evaluation item “not elegant”. An evaluation axis b having an evaluation item "open" and an evaluation item "not open", an evaluation item "dynamic" and an evaluation item "not dynamic" The answer results for a total of four evaluation axes of the evaluation axis d are shown.

The inventor of the present application assumes that there is some correspondence between the measured biological signal 101 of each subject in time series and the subjective evaluation result 102 of each subject, and brain activity is generated based on this assumption. If so, we thought that related scenes could be estimated. In addition, as a result of preliminary experiments, a result that this assumption was valid was obtained.
2 schematically shows that the subjective evaluation result 102 shown in FIG. 2 and the biological signal (observation data) 101 shown in FIG. 2 have some correspondence.
The “scene 115 corresponding to the evaluation axis” schematically shown in the dashed-line frame 201 in FIG. 2 corresponds to the “scene 115 corresponding to the evaluation axis (see FIG. 1)” output from the video evaluation apparatus 100. In the example of FIG. 2, the subject must answer either “Y / N” on each of the four evaluation axes. However, if only the response “Y” of the item “bright” is selected on the evaluation axis a, the time change of the scene (time change of the relevance) related to the answer of the selected item is indicated by the broken line in FIG. If the degree of relevance A changes as shown in 201, the time change α of the characteristic brain activity 210 appears corresponding to the time change A of the degree of relevance.
Further, since the response of the item “classy” on the evaluation axis b is “N”, the temporal change B of the relevance level does not appear in the temporal change β of the characteristic brain activity.
Similarly, the response “Y” of the item “open” on the evaluation axis c can be associated with the temporal change C of the degree of association and the temporal change γ of the characteristic brain activity. It is also possible to make the response “Y” of the item “dynamic” on the axis d correspond to the temporal change D of the relevance and the temporal change δ of the characteristic brain activity.
Here, it is assumed that the biological signal actually observed is generated as a signal in which the temporal changes α, β, γ, δ of the characteristic brain activity 210 and the residue 230 such as noise are all superimposed. By collecting the subjective evaluation result 102 and the biological signal (observation data) 101 from a plurality of subjects, and searching for the generation rule of the biological signal (= time change of the relevance) based on the assumption of the correspondence between the subjective evaluation result and the biological signal. The scene 115 corresponding to each evaluation axis can be estimated.

  Further, when a plurality of subjects view the same video content, the correspondence between the biological signal 101 of each subject and the subjective evaluation result 102 differs depending on the subject i and also differs depending on the measurement channel c. Therefore, in the learning stage, the video evaluation apparatus 100 performs mapping that associates the similarity of the time series data of the biological signal 101 between the plurality of subjects and the similarity of the pattern of the subjective evaluation result 102 between the plurality of subjects by machine learning. First, information (correspondence) is obtained.

  After this learning stage, the video evaluation apparatus 100 further includes a time-series basis of biological signals related thereto when a specific subjective evaluation value (for example, “bright” in FIG. 2) is input in the estimation stage. A scene related to the subjective evaluation (for example, the temporal change A of the degree of association in FIG. 2) is estimated by finding a pattern (for example, the temporal change A in the degree of association in FIG. 2). Hereinafter, the configuration and operation of the video evaluation apparatus 100 that performs such learning and estimation will be described in detail.

[Configuration of video evaluation equipment]
As shown in FIG. 1, the video evaluation apparatus 100 includes a biological signal measuring unit 103, a subjective evaluation grasping unit 104, a biological signal accumulating unit 105, a subjective evaluation accumulating unit 106, a biological signal similarity calculating unit 107, Subjective evaluation similarity calculation means 108, similarity mapping calculation means 109, mapping storage means 110, mapping selection means 120, subjective evaluation vector creation means 112, subjective evaluation similarity calculation means 113, and subjective evaluation correspondence Scene estimation means 114.

<Biological signal measuring means 103>
The biological signal measuring means 103 receives the biological signal 101 of the subject measured by a biological signal measuring device (not shown) at the same time when the subject is viewing the video content, and receives a predetermined signal from the biological signal 101. The biological signal data is created by performing signal processing.

  In this embodiment, the biological signals 101 of a plurality of subjects measured for each measurement channel of an external biological signal measuring apparatus (for example, an fMRI apparatus) (not shown) are sequentially input to the biological signal measuring means 103. The biological signal measuring unit 103 easily performs preprocessing such as noise removal processing on a biological signal that is a time-series electric signal or the like depending on the type and specification of the biological signal measuring device, and easily analyzes the biological signal. Thus, time-series biological signal data is created for each measurement channel of the subject by performing numerical conversion as described above or associating with position data of the measurement channel. The created biological signal data is stored in the biological signal storage means 105 in association with the position information of the measurement channel and the identification information of the subject.

  The fMRI apparatus outputs the amount of oxygen in the brain as a cross-sectional image of the brain, but the image differs in size, shape, etc. depending on the subject. Therefore, the biological signal measuring unit 103 compresses, expands, and rotates the cross-sectional image by affine transformation or the like in order to align the parts indicated as a common measurement channel among a plurality of subjects, and the size of the template brain image called a standard brain It is preferable to perform normalization processing to match the shape and the like.

<Biological signal storage means 105>
The biological signal storage unit 105 stores the biological signal input from the biological signal measuring unit 103. The biological signal storage means 105 can be configured by a general storage device such as a semiconductor memory or a hard disk.
The biological signal storage unit 105 stores biological signals of a plurality of subjects as time series signals associated with the subjects and measurement channels. For example, the biological signal storage means 105 stores biological signals measured by 10,000 measurement channels for 10 subjects. Here, it is assumed that the number of subjects who have measured the biological signals stored in the biological signal storage means 105 is M (for example, 10). As will be described later, in order to obtain mapping information (correspondence), data for N (N <M) people (for example, 9 people) is used, and the accuracy of mapping information is obtained with the data for (MN) people. Perform verification.
The biological signal stored in the biological signal storage unit 105 is referred to by the biological signal similarity calculation unit 107, the mapping selection unit 120, and the subjective evaluation corresponding scene estimation unit 114.

<Biological signal similarity calculation means 107>
The biological signal similarity calculating unit 107 calculates the biological signal similarity between subjects for the biological signals of a plurality of subjects who viewed the same video content stored in the biological signal storage unit 105. In addition, in the biosignal similarity between subjects, in addition to a pair of a subject a and another subject b, a pair of the subject a and the subject a is also considered. Here, the pair of the subject a and the subject a becomes a diagonal component of the similarity matrix when analyzing the similarity between the plurality of subjects in a matrix format.
The biological signal similarity calculating means 107 uses the biological signal data of each of N (N <M) subjects in order to obtain matrix-format mapping information (correspondence relationship), and the biological signal among N × N subjects. Similarity is calculated, and a biological signal similarity matrix having these calculation results as elements is calculated for each measurement channel.

  Specifically, the biological signal similarity calculation unit 107 calculates a biological signal similarity matrix as follows. Here, for example, there are only N × N pairs of combinations of N subjects when a pair of the subjects is allowed. One subject of each pair is identified by an integer i (1 ≦ i ≦ N), the other subject is identified by an integer j (1 ≦ j ≦ N), and a measurement channel is identified by an integer c. Further, when the time-series biological signal data of the subject i stored in the biological signal storage unit 105 has T elements corresponding to the video content viewing time (1 ≦ t ≦ T), the biological signal Data can be represented by a T-dimensional vector as shown in the following equation (1).

In the equation (1), the subscript t represents time, and the superscript t is a mathematical symbol indicating transposition (the same applies to the following equations). In the right side of the equation (1), for example, x ^ci _{, 1} is a biological signal first measured in the measurement channel c while the subject i is viewing the video content.

The biological signal similarity matrix M ^c _x between subjects calculated for each measurement channel c in the biological signal similarity calculation means 107 is represented by an N × N-dimensional matrix as shown on the right side of the following equation (2). It is defined as At this time, the i-row and j-column components of the matrix on the right side of Equation (2) (the ij element of the matrix = M ^c _{x, i, j} ) are the biological signal vector for the measurement channel c of the subject i, and the subject j's It becomes the similarity with the biological signal vector for the measurement channel c.
For example, M ^c _{x, 1, N} = M ^c _{x, N, 1} . Therefore, the biological signal similarity matrix M ^c _x is an N × N-dimensional symmetric matrix. There are N diagonal components (i = j components) of this symmetric matrix, but when the correlation coefficient is used as a means for measuring the degree of similarity, all are 1 because they are biological signals of the same person.

Furthermore, the ij element M ^c _{x, i, j} (similarity) of the biological signal similarity matrix between subjects is the biological signal vector for the measurement channel c of the subject i and the biological signal vector for the measurement channel c of the subject j. Is represented by the right side of the following equation (3a).

  The denominator on the right side of the equation (3a) represents the product of vector norms. The vector on the left side of the equation (3b) is a vector having the average value of all the elements of the vector shown in the equation (1) in each element as defined on the right side, and the vector on the left side of the equation (3c) Is a T-dimensional vector in which all elements are 1.

The biological signal similarity calculating unit 107 uses the biological signal similarity matrix M ^c _x calculated by the above formulas (1), (2), (3a), (3b), and (3c) as the similarity level. Output to the mapping calculation means 109. Here, the correlation coefficient is used as the biological signal similarity between subjects, but instead, other similarity indices such as inner product between vectors and cosine distance may be used.

<Subjective evaluation grasping means 104>
The subjective evaluation grasping means 104 is the subjective response to which each of the M subjects corresponding to the biological signal data for M persons (for example, 10 persons) stored in the biological signal storage means 105 responds for each evaluation axis after viewing the video content. Data of the evaluation result 102 is input, a subjective evaluation vector is created from the data of the subjective evaluation result 102, and the M subjective evaluation vectors are associated with the identification information of the subject and stored in the subjective evaluation accumulation means 106. As will be described later, in order to obtain mapping information (correspondence), data for N (N <M) people (for example, 9 people) is used.

  The subjective evaluation grasping means 104 inputs the results (input numerical values) answered by a plurality of subjects in response to the question about the video content via input means (not shown) such as a keyboard and a touch panel.

For example, the subjective evaluation grasping means 104 gives 1 if the evaluation value for the evaluation item in the subjective evaluation result 102 exceeds a certain threshold value, and gives 0 if the evaluation value is less than the threshold value. Here, an average value of evaluation values obtained from a plurality of subjects may be used as the threshold value. Therefore, if each evaluation item is assigned a numerical value of 1 and 0 and the number of evaluation items is N _q , the subjective evaluation grasping means 104 outputs an N _q -dimensional vector. For example, subjective evaluation vector y _i of the subject i is expressed as shown in the following equation (4). At this time, each element takes a value of 0 or 1. The k-th element y _{i, k} represents an answer to the evaluation item k.

  This subjective evaluation grasping means 104 writes the subjective evaluation vector shown in Expression (4) into the subjective evaluation accumulating means 106. Note that a continuous amount can be used instead of 0 and 1, and the quantification method can be changed according to the subjective evaluation method to be set.

<Subjective evaluation accumulating means 106>
The subjective evaluation accumulating unit 106 stores the evaluation result data (subjective evaluation vector) input by the subjective evaluation grasping unit 104. The subjective evaluation accumulation means 106 can be configured by a general storage device such as a semiconductor memory or a hard disk.
The subjective evaluation accumulation means 106 stores the subjective evaluation vector shown in the equation (4) associated with the subject i as evaluation result data (subjective evaluation vector). At this time, it is assumed that the subjective evaluation vector is associated with the corresponding preprocessed biological signal (output of the biological signal measuring unit 103). The subjective evaluation vector stored in the subjective evaluation accumulation unit 106 is referred to by the subjective evaluation similarity calculation unit 108, the mapping selection unit 120, and the subjective evaluation similarity calculation unit 113.

<Subjective evaluation similarity calculation means 108>
The subjective evaluation similarity calculating means 108 calculates the subjective evaluation similarity between subjects for the evaluation result data (subjective evaluation vectors) of a plurality of subjects who viewed the same video content stored in the subjective evaluation accumulating means 106. It is. In addition, in the subjective evaluation similarity between subjects, in addition to a pair of a subject a and another subject b, a pair of the subject a and the subject a (diagonal component of the similarity matrix) is also considered.
This subjective evaluation similarity calculation means 108 combines N (N <M) subjects using evaluation signal data (subjective evaluation vector y _i ) for all items of a plurality of evaluation axes, and N A subjective evaluation similarity matrix having the subjective evaluation similarity among N subjects as each element is calculated.

Specifically, the subjective evaluation similarity calculation means 108 calculates a subjective evaluation similarity matrix as follows. The subjective evaluation similarity matrix (M _y ) between subjects calculated by the subjective evaluation similarity calculation means 108 is defined as an N × N-dimensional matrix as shown on the right side of the following equation (5). . At this time, the i-row and j-column components of the matrix on the right side of Equation (5) (the ij element of the matrix = M _{y, i, j} ) are similar to the subjective evaluation vector of the subject i and the subjective evaluation vector of the subject j Degree. In addition, there is a relationship such as My _{, 1, N} = My _{, N, 1,} etc. between the elements of this matrix. Thus, subjective evaluation similarity matrix M _y becomes N × N dimensional symmetric matrix.

Furthermore, it is assumed that the ij element My _{, i, j} (similarity) of the subjectivity evaluation similarity matrix between subjects is an inner product between the subject evaluation vector of subject i and the subject evaluation vector of subject j. , Ij element My _{, i, j} can be expressed by the right side of the following equation (6).

This subjective assessment similarity calculation means 108 outputs the formula (5) and a subjective assessment similarity matrix M _y calculated in the similarity between the map calculation unit 109 by the equation (6). Here, the subjective evaluation similarity calculation unit 108 uses the inner product between vectors as the subjective evaluation similarity between subjects, but other similarity indices such as a cosine distance may be used.

<Similarity Mapping Calculation Unit 109>
The similarity-to-similarity mapping calculation unit 109 associates the biological signal similarity between subjects calculated by the biological signal similarity calculation unit 107 and the subjective evaluation similarity between subjects calculated by the subjective evaluation similarity calculation unit 108. Is.

The similarity-to-similarity mapping calculation unit 109 includes, as mapping information, a biological signal similarity matrix (M ^c _x : see the above equation (2)), a subjective evaluation similarity matrix (M _y : see the above equation (5)), A mapping matrix set (W ^c , V ^c ) composed of two matrices including N-dimensional vertical vectors as elements is obtained for each measurement channel c and stored in the mapping storage means 110.
Therefore, the similarity between the map calculation unit 109, a biological signal similarity matrix M ^c _x, and a subjective evaluation similarity matrix M _y, first weight vector w ^c and a satisfy the following expression (7) Request 2 weight vector ^{v c.} Each of the weight vectors w ^c and v ^c is an N-dimensional vertical vector corresponding to the measurement channel c.

  Here, when this equation (7) is satisfied, the correlation coefficient is 1, and the following equation (8) may also be satisfied.

  Here, it is assumed that each norm written in the denominator on the left side of the equation (8) is a constant. If this constant is set to 1, for example, Expression (8) can be replaced with Expression (9) below.

Here, the biological signal similarity matrix M ^c _x, with the measurement of the biological signal 101 includes a measurement error, also, the subjective evaluation similarity matrix M _y, with the understanding of the subjective evaluation results 102 Therefore, it is difficult to obtain the first and second weight vectors w ^c and v ^c so that the calculation result on the left side of the equation (9) is equal to 1. Therefore, the similarity-to-similarity mapping calculation unit 109 obtains first and second weight vectors w ^c and v ^c that maximize the calculation result on the left side of Equation (9).

Here, the similarity-to-similarity mapping calculation unit 109 calculates a first vector (which is a product of the biological signal similarity matrix M ^c _x for each measurement channel c and an unknown first weight vector w ^c which is an N-dimensional vertical vector. and M ^c _x w ^c), and the second vector is the product of the unknown second weight vector ^{v c} is the column vector of subjective evaluation similarity matrix _{M y} and N-dimensional _(M y ^{v c),} the inner product ( M ^c _x w ^c ) ^t (M _y v ^c ) is under the constraint that the norm of the first vector (M ^c _x w ^c ) and the second vector (M _y v ^c ) is a predetermined constant. The maximum first weight vector w ^c and second weight vector v ^c are calculated as elements of columns of the mapping matrices W ^c and V ^c . A mapping matrix set (W ^c , V ^c ) is obtained for each measurement channel c.

That is, the similarity-to-similarity mapping calculation unit 109 calculates first and second weight vectors w ^c and v ^c that satisfy the following expressions (10a) and (10b), respectively.

The maximization problem of equations (10a) and (10b) can be solved using Lagrange's undetermined constant method. Here, the error function (Lagrangian) L is expressed by the following equation (11). Note that λ _w and λ _v are undetermined multipliers.

Incidentally, when the noise or fluctuation in the data of the biological signal 101 and subjective evaluation results 102 are abundant, tend to first and second weight vector w ^c, v norm of ^c increases. Therefore, here, a new error function is defined that takes into consideration that the norm does not become excessive. Therefore, two penalty terms depending on the norms of the first and second weight vectors w ^c and v ^c are added to the error function L to define an error function L ′ shown in the following equation (12). Note that L on the right side of Expression (12) is an error function represented by Expression (11).

Here, the parameters ρ _w and ρ _v are parameters for controlling the vulnerability to the noise and fluctuation of the data of the biological signal 101 and the subjective evaluation result 102, and are set in advance depending on each data. (Ρ _w > 0, ρ _v > 0).

Based on the equation (11) and (12), 'and expressions results were differentiated in the first weight vector w ^c is 0, the error function L' error function L obtained by differentiating the second weight vector v ^c Obtaining an equation with a result of 0 results in the eigenvalue problem shown in the following equations (13a) and (13b).

In Expressions (13a) and (13b), I represents an N × N-dimensional unit matrix. In addition, the eigenvalue λ and the parameters r _w and r _v satisfy the relationship r _w = ρ _w / λ _w , r _v = ρ _v / λ _v , 4λ _w λ _v = λ ² . The derivation of the equations (13a) and (13b) will be described later.

Here, set in the error function L 'shown in Equation (12), the parameter [rho _w, was previously set [rho _v, instead of setting the parameters [rho _w, [rho _v, the parameter _r w, the _{r v} advance (E.g., r _w = 1, r _v = 1).

By solving the eigenvalue problem of the equations (13a) and (13b), the first and second weight vectors w ^c and v ^c and the eigenvalue λ can be determined.
However, the first and second weight vectors w ^c and v ^c are also eigenvectors, and usually a plurality of weight vectors are determined by the number of eigenvalues λ. Therefore, the similarity-to-similarity mapping calculation unit 109 selects a predetermined number (here, P) of eigenvalues λ in descending order, and sets the first weight vector w ^c and the second weight vector v ^c as corresponding eigenvectors. As many pairs as the number of selected eigenvalues λ, that is, P pairs are determined, and P pair vectors are used as mapping information to obtain two different mapping matrices as shown in the following equations (14a) and (14b). A set (W ^c , V ^c ) is generated.

As a result, the biological signal similarity matrix M ^c _x (see the equation (2)) and the subjective evaluation similarity matrix M _y (see the equation (5)) for the measurement channel c are expressed by the following equation (15): As shown in FIG. 5, the mapping matrixes W ^c and V ^{c in the} equations (14a) and (14b) can be associated through the P-dimensional space.

This similarity-to-similarity mapping calculation unit 109 calculates mapping information (a set of mapping matrices (W ^c , V ^c )) for all measurement channels c and writes the mapping information to the mapping storage unit 110.

<Mapping storage means 110>
The mapping storage means 110 stores the mapping information (mapping matrix set (W ^c , V ^c )) calculated for each measurement channel c by the similarity mapping calculation means 109. The mapping storage unit 110 can be configured by a general storage device such as a semiconductor memory or a hard disk. The mapping information stored in the mapping storage unit 110 is referred to by the mapping selection unit 120. Note that the mapping storage unit 110 may be provided outside the video evaluation apparatus 100 as long as the video evaluation apparatus 100 can write and reference information.

<Mapping selection means 120>
The mapping selection unit 120 selects at least one measurement channel C (εc) from the mapping storage unit 110 and selects mapping information (a set of mapping matrices (W ^C , V ^C )) for the measurement channel. It is.

The mapping selection unit 120 selects a measurement channel that is closely related to an answer to the subjective evaluation question (evaluation result data) from the measurement channels in which the biological signal is measured. For this reason, the mapping selection unit 120 is not used when obtaining a plurality of mapping matrix sets (W ^c , V ^c ) obtained by the similarity mapping calculation unit 109 and a plurality of mapping matrix sets. Using at least one measurement channel C (Cεc) whose prediction accuracy satisfies a predetermined standard, using biological signal data x ^c _{i of} other subjects and evaluation result data y _i , the determined measurement A set of mapping matrices (W ^C , V ^C ) for channel C is selected.
For example, if the number of measurement channels c is 10,000, for example, 100 of them may be selected as the measurement channels C.

  Here, the mapping selecting unit 120 includes a biological signal similarity calculating unit 121, a subjective evaluation similarity calculating unit 122, an error calculating unit 123, and a channel specifying unit 124.

<< Biological signal similarity calculation means 121 >>
The biological signal similarity calculation unit 121 does not use the subject i (i = 1, 2,..., N) belonging to the N groups Ga used for estimating the mapping matrix and the mapping matrix (MN). ) The biological signal similarity between subjects is calculated with subjects j (k = N + 1, N + 2,..., M) belonging to the group Gb of people.
Here, the biological signal similarity calculating unit 121 refers to the biological signal data (biological signal vector) stored in the biological signal storage unit 105 for each measurement channel c, and the subjects i belonging to the group Ga of N persons. (i = 1,2, ..., N ) and the biological signal vector ^x _{c i} of, (M-N) belonging to the group Gb human subjects j (j = N + 1, N + 2, ..., M) biological signal vector ^{x c} of _As a correlation coefficient with _j , M ^c _{x, i, j} in the equation (3a) is calculated. The similarity may be another similarity index such as an inner product of vectors, a cosine distance, or the like.

Then, the biological signal similarity calculation unit 121 calculates the similarity (correlation coefficient M ^c _{x, i} ) between the subject i belonging to the N group Ga and the subject j belonging to the (MN) group Gb. _{, J} ) as a component, a biological signal similarity matrix M ′ ^c _x (second biological signal similarity matrix) represented by the following expression (16) is calculated for each measurement channel c, and this biological signal similarity matrix M ′ is calculated. ^c _x is output to the error calculation means 123. The biological signal similarity matrix M ′ ^c _x is not a square matrix but an (MN) × N matrix.

<< Subjective evaluation similarity calculation means 122 >>
The subjective evaluation similarity calculation means 122 does not use the subjects i (i = 1, 2,..., N) belonging to the N groups Ga used for the mapping matrix estimation and the mapping matrix estimation (M -N) Subjective evaluation similarity between subjects is calculated among subjects j (k = N + 1, N + 2,..., M) belonging to a group Gb of people.

The subjective evaluation similarity calculating unit 122 refers to the evaluation result data (subjective evaluation vector) stored in the subjective evaluation accumulating unit 106, and subjects i (i = N + 1, N + 2,...) Belonging to N groups Ga. subjective evaluation vector _{y i} of M), a similarity between the subjective evaluation vector _{y j} of (M-N) human group Gb belonging subject j (j = N + 1, N + 2, ..., M), the equation (6 ), The inner product My _{, i, j} between the vectors is calculated. For the similarity, another similarity index such as a cosine distance may be used.

Then, the subjective evaluation similarity calculating means 122 calculates the subjective evaluation similarity My _{, i, j} between the subject i belonging to the N group Ga and the subject j belonging to the (MN) group Gb. A subjective evaluation similarity matrix M ′ _y (second subjective evaluation similarity matrix) represented by the following expression (17) as elements is calculated, and the subjective evaluation similarity matrix M ′ _y is output to the error calculation means 123. Note that the subjective evaluation similarity matrix M ′ _y is not a square matrix but an (MN) × N matrix.

<< Error calculation means 123 >>
The error calculation means 123 includes a biological signal similarity matrix M ′ ^c _{x for} each measurement channel c calculated by the biological signal similarity calculation means 121 and a subjective evaluation similarity matrix M calculated by the subjective evaluation similarity calculation means 122. The mapping error (matrix W ^c , V ^{c of} N × P matrix) stored in the mapping storage means 110 is applied to ′ _y to calculate the prediction error for each measurement channel c. Here, the prediction error E shown in the following equation (18) is defined.

  The error calculation means 123 calculates the prediction error E shown in the equation (18) for each measurement channel c, and outputs the calculated prediction error E to the channel specifying means 124 together with the identification number of the measurement channel c.

<< Channel identification means 124 >>
The channel specifying unit 124 specifies a measurement channel with a small prediction error calculated by the error calculation unit 123 as a measurement channel C (εc) suitable for estimating a scene corresponding to the evaluation axis.
The channel specifying means 124 is closely related to the answer to the subjective evaluation question (evaluation result data) for the measurement channel in which the prediction error E calculated by the error calculating means 123 is lower than a predetermined threshold (for example, 0.1). The measurement channel C is specified.
Note that the channel specifying unit 124 may select one measurement channel with the smallest error, or may select a plurality of measurement channels below a predetermined threshold.

<Subjective evaluation vector creation means 112>
Subjective evaluation vector creating means 112 accepts selection of a specific evaluation item on a predetermined evaluation axis among a plurality of evaluation axes in order to estimate a scene corresponding to the predetermined evaluation axis, and out of all items on the plurality of evaluation axes A subjective evaluation vector y _new is generated that numerically represents that only a specific evaluation item has been selected.
This subjective evaluation vector creating means 112 has the same function as the subjective evaluation grasping means 104. However, subjective evaluation grasping unit 104 is input with the subject's answers (subjective evaluation result 102), the number of evaluation axes (questions number) for all the evaluation axis if N _q, by digitizing the evaluation level N The subjective evaluation vector y _i of the equation (4) is generated as a _q- dimensional vector. On the other hand, the subjective evaluation vector creating means 112 inputs an evaluation axis selection 111 in order to estimate a corresponding scene. Here, the evaluation axis selection 111 is an evaluation axis item selected for identifying a related scene. Then, the subjective evaluation vector creating means 112 creates a subjective evaluation vector y _new as an N _q- dimensional vector shown in the following formula (19), where 1 is a specific evaluation item of a specific evaluation axis and 0 is the others. To do.

On the right side of equation (19), the value of any one element of y _{new, 1} ,..., Y _{new, Nq} is 1, and the values of the other elements are all 0.
The subjective evaluation vector creating unit 112 outputs the created subjective evaluation vector y _new to the subjective evaluation similarity calculating unit 113.

<Subjective evaluation similarity calculation means 113>
The subjective evaluation similarity calculation unit 113 includes the subjective evaluation vector y _new generated by the subjective evaluation vector generation unit 112 and the subjective evaluation vector y of N (N <M) subjects stored in the subjective evaluation storage unit 106. by calculating the similarity between _i and calculates a subjective assessment similarity vector s _y consisting of an N-dimensional vector. The subjective evaluation vector y _i stored in the subjective evaluation accumulating means 106 is evaluation result data regarding all items of a plurality of evaluation axes.

At this time, the similarity between the subjective evaluation vector y _new and the subjective evaluation vectors y _i (i = 1 to N) of N subjects stored in the subjective evaluation accumulating means 106 (see the above equation (4)) is can be represented as a similarity vector s _y N-dimensional as shown in the following equation (20).

<Subjective evaluation corresponding scene estimation means 114>
Subjective evaluation corresponding scene estimation means 114 is an unknown vector (x _new ^c : expression (23)) having a matrix that is data of time-series biosignals of each of N subjects and a predicted time-series biosignal as elements. ) And a biological signal similarity vector (s ^c _x : expression (22)) consisting of an N-dimensional vector and a subjective evaluation vector (y _new : expression (19)) in which only a specific evaluation item is selected The relational expression (Expression (24)) that relates the subjective evaluation similarity vector (s _y : Expression (20)) obtained from the above with the mapping information for the measurement channel C selected by the mapping selection means 120 is established. obtains the estimated value of the time series of the biological signal by performing a calculation to determine the elements of the unknown vector x ^C _{new new} to, as the scene that has contributed to the evaluation in the evaluation axis selected 111 was obtained It is to determine the scene of the image corresponding to the estimated value of the time series of body signals.

For example, in a time-series biological signal (original biological signal vector) x ^C _i corresponding to a subject i and a selected mapping matrix (that is, a selected measurement channel Cεc), M individual vector elements are present. By generating a vector having the value obtained by subtracting the average value of M elements from each value, and normalizing the generated vector with a norm of 1, the normalized biological signal data is obtained. It is defined by the following formula (21). Similarly, normalized biological signal data of N subjects i is generated.

Normalized biosignal data represented by the formula (21) is a column vector Similarly T dimension and original biological signal vector x ^C _i. At this time, in a matrix of N columns, a matrix having normalized biological signal data (T-dimensional vertical vector) of subject i is assumed as an i-th column element, and the transposed matrix related to this matrix will be estimated. An N-dimensional vertical vector generated as a result of multiplying an unknown T-dimensional vector from the right is set as a biological signal similarity vector s ^C _x as shown in the following equation (22).

Here, x ^C _new is an unknown T-dimensional vector and represents a time-series biological signal in the selected measurement channel C (εc). The biological signal x ^C _new corresponds to the brain activity time-series data related to the axis selected in the evaluation axis selection 111 (for example, any one of the time changes α, β, γ, and δ of the characteristic brain activity shown in FIG. 2). . This time-series biological signal x ^C _new can be expressed by a vector of the following equation (23).

The mapping storage means 110 stores a mapping matrix set (W ^c , V ^c ) (see equations (14a) and (14b)) for each measurement channel c. Similarity and subjective evaluation similarity are associated with each other. Therefore, the subjective evaluation N-dimensional similarity vector s _y (see formula (20)) calculated by the subjective evaluation similarity calculation means 113 and the biological signal similarity vector s ^C _{x in} formula (22) are: It has the relationship shown in the following formula (24). C represents the measurement channel selected by the mapping selection means 120. When they are used, the biological signal similarity vector s ^C _x and the subjective evaluation similarity vector s _y can be expressed in association with each other by the following equation (24).

Therefore, the subjective evaluation corresponding scene estimation unit 114 calculates a time-series biological signal x ^C _new in the measurement channel C such that the expression (24) is satisfied as an estimated value of the brain activity time-series data. That is, the subjective evaluation corresponding scene estimation unit 114 satisfies the following expression (25) among the biological signal data (biological signal vector) x ^C _new corresponding to the selected mapping matrix (selected measurement channel C). calculating a biological signal vector as an estimate ^x _{^ C new.}

There are various methods for solving the equation (25). For example, the subjective evaluation corresponding scene estimation unit 114 determines the similarity according to the equation (22) for all combinations of values of the biological signal vector x ^C _new. The vector s ^C _x is generated, and the biological signal data (biological signal vector) x ^C _new that minimizes the square of the norm on the right side of Equation (25) may be used as the estimated value x ^ ^C _new .
It may be difficult to list all combinations of values. Therefore, the subjective evaluation corresponding scene estimation unit 114 may calculate the estimated value x ^ ^C _new using the following equation (26) obtained by solving the equation (25) by the least square method.

In this formula (26), the matrix X ^~ for the selected measurement channels C represents a matrix with normalized biometric signal data (the equation (21) refer) to the i-th column. The matrix Y represents a matrix having the subjective evaluation vector y _i (refer to the equation (4)) in the i-th column. The derivation of this equation (26) will be described later.

As described above, the subjective evaluation corresponding scene estimation unit 114 outputs the estimated value x ^ ^C _new of the biological signal vector satisfying the expression (25) to the outside as the scene 115 corresponding to the evaluation axis.

Each element of the estimated value x ^ ^C _new of the biological signal vector is not observed, but is time-series data similar to the measured biological signal, and can therefore be associated with each scene of the video content.
Therefore, the subjective evaluation corresponding scene estimation means 114 performs a specific evaluation in the evaluation axis selection 111 for a video scene corresponding to a time point when the absolute value of the estimated value x ^ ^C _new of the biological signal vector exceeds a predetermined threshold. The scene is determined as having a large contribution to the evaluation of the item.
In this embodiment, the subjective evaluation corresponding scene estimation means 114 assigns, for example, 1 when the value of each element of the estimated value x ^ ^C _new is equal to or greater than a predetermined threshold value, and 0 when the value is lower than the threshold value. Was assigned. By doing so, the scene of the video content corresponding to the element whose element value is 1 or more becomes the scene corresponding to the evaluation axis selected in the evaluation axis selection 111. Here, the threshold value may be an average value of each element of the estimated value x ^ ^C _new of the biological signal vector.

Further, the number of threshold values is not limited to one. For example, the subjective evaluation corresponding scene estimation unit 114 sets two threshold values in the threshold processing for the estimated value x ^ ^C _new of the biological signal vector, and exceeds the first threshold value. 1 may be assigned, -1 may be assigned if the second threshold value is exceeded, and 0 may be assigned if the second threshold value or more and the first threshold value or less.
In this case, a scene to which 1 is assigned can be regarded as a scene that contributes to a positive evaluation on the evaluation axis, and a scene to which -1 is assigned to a scene that contributes to a negative evaluation on the same evaluation axis. When the first threshold is th1 and the second threshold is th2, for example, the following equations (27a) and (27b) can be set.

Here, the first term on the right side of equation (27a) the first term on the right side and formula (27b) represents the average value of the estimated value ^x _{^ C new new} total vector elements of the biological signal vector. The minimum of the max _{^(x ^ C new)} represents the maximum value of the total vector elements estimate _{^{x ^ C new, min (x}} ^ C new) All vector elements estimate ^x _{^ C new new} Indicates the value. a and b are parameters, each of which is a positive real number. For example, take a = b = 0.2.

A scene corresponding to the evaluation axis can be obtained for each selected measurement channel C. Finally, after obtaining a vector obtained by averaging estimated values x ^ ^C _new of biological signal vectors of the same number as the number of selected measurement channels C, the above threshold processing is performed, and the scene corresponding to the evaluation axis is determined. Identification may be performed.

(Regarding the derivation of Expression (13a) and Expression (13b))
A procedure for deriving the equations (13a) and (13b), which are eigenvalue problems, from the equation (12) in the similarity mapping calculation unit 109 will be described.
First, L ′ in the equation (12) is differentiated by w ^c as shown in the following equation (28) and set to 0.

Here, since M _x ^c is a symmetric matrix, there is a relationship of M _x ^c = M _x ^ct . Therefore, this equation (28) can be transformed into the following equation (29).

Here, I represents an N × N-dimensional unit matrix.
Similarly, the L 'in the formula (12), differentiated by ^{v c} as shown in the following equation (30), put to zero.

Since _{M y} is a symmetric _matrix, a relationship of M y _{= M} ^{y t.} Therefore, this equation (30) can be transformed into the following equation (31).

  Furthermore, this equation (31) can be transformed into the following equation (32).

The ^{v c,} are substituted into the formula (29) by the following expression (33).

Here, if r _w = ρ _w / λ _w , r _v = ρ _v / λ _v , 4λ _w λ _v = λ ² (that is, r _w = 2ρ _w / λ, r _v = 2ρ _v / λ). From the equations (32) and (33), the equations (13a) and (13b) can be derived.

(Derivation of Equation (26))
Derivation of the equation (26) for calculating the estimated value x ^ ^C _new of the biological signal data (biological signal vector) in the subjective evaluation corresponding scene estimation means 114 will be described below.
The equation (25) can be modified as shown in the following equation (34).

Here, the square of the norm on the right side of the equation (34) is differentiated by x ^C _{new as} shown in the following equation (35) and set to 0.

  Then, the equation (26) can be derived by modifying the equation (35) as shown in the following equations (36) to (39).

[Operation of video evaluation system]
Next, the operation of the video evaluation apparatus 100 according to the embodiment of the present invention will be described with reference to FIG.

(Learning stage operation)
A biological signal measuring device (not shown) measures time-series biological signals (biological signals) simultaneously with viewing for each measurement channel from a plurality of subjects viewing video content.
When the biological signal 101 for each measurement channel is input to the biological signal measurement unit 103 via a biological signal measurement device (not shown) connected to the outside, the video evaluation device 100 performs predetermined processing by the biological signal measurement unit 103. The biological signal data is created (measured) by performing the signal processing (step S1), and stored in the biological signal storage means 105 in association with the subject and the measurement channel (step S2).

  Further, when the subjective evaluation result 102 (the result of the subject answering the subjective evaluation for the video content after viewing the video content) is input to the subjective evaluation grasping unit 104, the video evaluation device 100 causes the subjective evaluation grasping unit 104 to A subjective evaluation vector in which the subjective evaluation is digitized is generated (obtained) (step S3), and is stored in the subjective evaluation storage means 106 in association with the subject (step S4).

  Then, the video evaluation apparatus 100 returns to step S1 and repeats the operation if the biological signal and the evaluation result data are not input for the predetermined number of subjects (M) (No in step S5).

On the other hand, when the biological signal and the evaluation result data are input for the predetermined number of subjects (M) (Yes in Step S5), the video evaluation apparatus 100 stores the biological signal similarity calculation unit 107 in Step S2. With respect to the biosignals of N (N <M) subjects, the biosignal similarity between subjects (biological signal similarity matrix M ^c _x [refer to the equation (2)]) is calculated for each measurement channel (step S6).

In addition, the video evaluation apparatus 100 uses the subjective evaluation similarity calculation unit 108 to evaluate the subjective evaluation similarity (subjective evaluation similarity) between the N (N <M) test subject evaluation results data stored in step S4. The matrix M _y [see the above equation (5)])) is calculated (step S7).

  Then, the video evaluation apparatus 100 uses a mapping matrix (similarity between similarities) that associates the biological signal similarity matrix calculated in step S6 with the subjective evaluation similarity matrix calculated in step S7 by the similarity mapping calculation unit 109. Mapping) is calculated (step S8).

Here, the similarity-to-similarity mapping calculation unit 109 solves the maximization problem of the equation (10a) under the constraint condition of the equation (10b), and calculates the first weight vector w ^c and the first weight vector for each measurement channel c. pairs 2 weight vector ^{v c,} a plurality (P-number) is calculated. Then, P first weight vectors w ^c are arranged in the descending order of eigenvalues to form a mapping matrix W ^c, and similarly, the second weight vectors v ^c are arranged in descending order of eigenvalues to form a mapping matrix V ^c . Then, the similarity mapping calculation unit 109 writes the mapping matrix set (W ^c , V ^c ) for each measurement channel c in the mapping storage unit 110.

Then, the video evaluation apparatus 100 selects at least one mapping by the mapping selection unit 120 (step S9). That is, mapping information (a set of mapping matrices (W ^C , V ^C ) for each measurement channel C (εc)) is selected. Thereby, the video evaluation apparatus 100 can select the measurement channel C suitable for estimating the evaluation result data of the target subject.

(Estimation stage operation)
The evaluation axis item selected to identify the related scene, that is, the evaluation axis selection 111 shown in FIG. 1 is input from the outside during the estimation process (step S111). In addition, in the estimation process, instead of inputting each time from the outside, it may be automatically or manually selected from those stored in advance in the video evaluation apparatus 100. Then, the video evaluation apparatus 100 digitizes the evaluation axis selection 111 by the subjective evaluation vector creating means 112 to create the subjective evaluation vector y _new (step S112).

Then, the video evaluation apparatus 100 uses the subjective evaluation similarity calculation unit 113 to perform a plurality of evaluations on the subjective evaluation vector y _new created in step S112 and the subjects (1 to N) stored in the subjective evaluation storage unit 106. Similarity with evaluation result data y _i (i = 1 to N) regarding all items on the axis is calculated (step S113), and subjective evaluation similarity vector s _y (expression (20)) is obtained.

Then, the video evaluation apparatus 100 uses the subjective evaluation-corresponding scene estimation unit 114 based on the mapping matrix set (W ^C , V ^C ) for each measurement channel C and the biological body of the subject stored in the biological signal storage unit 105. a signal vector _{^{x C i (i = 1~N)}} , and a subjective evaluation similarity vector s _y calculated in step S113, to estimate the contribution is large scene for the evaluation of the specific evaluation item selected ( Step S114).

Here, the subjective evaluation corresponding scene estimation unit 114 stores an unknown vector having each element of a time-series biological signal predicted in the selected measurement channel C as x ^C _new and is stored in the biological signal storage unit 105. The similarity vector s ^C _x with the normalized biological signal data represented by the equation (21) obtained by normalizing the biological signal vector x ^C _i (i = 1 to N) is defined as the equation (22).

Then, the subjective evaluation corresponding scene estimation unit 114 selects a biological signal vector x ^C _new satisfying the above equation (24) from the mapping matrix stored in the mapping storage unit 110 corresponding to the measurement channel C in step S111. The biological signal pattern x ^ ^C _new corresponding to the scene having a large contribution to the evaluation of the evaluation axis selection 111 is calculated.
Thereby, the video evaluation apparatus 100 can estimate a scene that greatly contributes to the evaluation of the selected specific evaluation item.

As described above, the video evaluation apparatus 100 selects the specified identification by preliminarily estimating the mapping matrix that associates the biological signal similarity between a plurality of subjects and the subjective evaluation similarity between the plurality of subjects. It is possible to estimate a scene that greatly contributes to the evaluation of the evaluation items.
The video evaluation apparatus 100 can estimate what scene of video content such as a TV program or a movie contributed to subjective evaluation and impression evaluation performed after viewing.
In addition, the video evaluation apparatus 100 does not associate a specific biological reaction with a subjective evaluation, that is, does not provide a specific rule that considers a positive evaluation when the signal intensity exceeds a certain threshold, Since the biological signal time-series data of another person who has watched the same video content is associated with the similarity pattern of subjective evaluation, it can be applied to any video content.

According to the video evaluation apparatus 100, it is possible to evaluate how the video is received from the measured biological signal and the answered subjective evaluation.
In this way, by using the video evaluation apparatus 100, the video content creator can objectively grasp the influence of the video content on the viewer and quantitatively evaluate the effect of the video content.
Furthermore, according to the video evaluation apparatus 100, it is possible to objectively grasp the influence of the video content on the viewer, so that what kind of scene the video system can effectively demonstrate its features is. Can be revealed.

  The video evaluation apparatus 100 can be operated by a program (video evaluation program) for causing a computer to function as each unit having the above-described configuration.

<Variation of biosignal>
In the above-described embodiment, as an example, the biological signal is the measurement data of the cerebral blood oxygen amount by functional nuclear magnetic resonance imaging (fMRI). It may be based on external spectroscopy (NIRS). In this case, by using the NIRS brain measurement device as the biological signal measurement device, it is possible to acquire the amount of oxygen in the brain blood as data indicating the degree of absorption of near infrared light measured by near infrared spectroscopy (NIRS). it can.

Further, the biological signal is not limited to the measurement data of the cerebral blood oxygen amount, but may be, for example, electroencephalogram (EEG) data. In this case, for example, by using an electroencephalograph as the biological signal measuring device, an electroencephalogram can be acquired as potential data measured by an electrode placed on the subject's scalp or the like.
In addition, for example, when data or brain waves measured by near infrared spectroscopy is used as a biological signal, the measurement channel corresponds to a portion of the brain expressed in probe units or electrode units placed on the subject's head surface. To do.
Furthermore, the biological signal may be measurement data of the body surface temperature of the human body. For example, the body surface temperature can be acquired as image data indicating the heat distribution by using thermography as the biological signal measuring device.

<Modification of accuracy verification of mapping information>
In the embodiment, for convenience of explanation of the mathematical formula, as an example, N-th data from 1 to Nth considering the order is used for mapping estimation, and (M + 1) from (N + 1) th to Mth (MN). Although the accuracy verification of mapping information is performed with human data, the present invention is not limited to this, and cross-validation (leave-one-out cross-validation) may be performed.
For example, the biological signal similarity calculation unit 107 may further improve the prediction accuracy by estimating the mapping with each data set in which the combination of selecting N people from M people is sequentially changed. The subjective evaluation similarity calculating unit 108 performs the same processing, and the biological signal similarity calculating unit 107 sequentially calculates the subjective evaluation similarity between subjects for the same combination as selecting N from M people. In addition, when each similarity degree is calculated about a certain N, the data for the remaining (MN) persons corresponding to the N are used when verifying the accuracy of the mapping information. The mapping storage means 110 stores mapping information (mapping matrix set (W ^c , V ^c )) calculated for each different combination of subjects (N persons).

  Similarly, the biological signal similarity calculation unit 121 and the subjective evaluation similarity calculation unit 122 in the mapping selection unit 120 perform cross-validation in the same manner, and the error calculation unit 123 has a sufficient number of combinations (for example, 100 types). ) May be repeated. When there are a plurality of combinations of subjects, the error calculation unit 123 calculates a prediction error E for each combination, and sets the average as the prediction error E again. As described above, the mapping selection unit 120 verifies various mapping matrix sets and selects a measurement channel corresponding to a mapping matrix set with high accuracy, thereby generating a scene corresponding to the evaluation axis of the subjective evaluation by machine learning. A measurement channel (biological signal) suitable for estimation can be accurately estimated.

DESCRIPTION OF SYMBOLS 100 Image | video evaluation apparatus 103 Biosignal measuring means 105 Biosignal storage means 107 Biosignal similarity calculation means 104 Subjective evaluation grasping means 106 Subjective evaluation accumulation means 108 Subjective evaluation similarity calculation means 109 Inter-similarity mapping calculation means 110 Mapping storage means 112 Subjective evaluation vector creation means 113 Subjective evaluation similarity calculation means 114 Subjective evaluation corresponding scene estimation means 120 Mapping selection means 121 Biosignal similarity calculation means 122 Subjective evaluation similarity calculation means 123 Error calculation means 124 Channel identification means

Claims (6)

Time-series biological signal data of each subject measured simultaneously for each of a plurality of measurement channels indicating any part of the body from a plurality of subjects viewing video content presented in the video system, and a plurality of the plurality of subjects The video system is evaluated by estimating the scene of the video content that has contributed to the evaluation on the predetermined evaluation axis based on the evaluation result data of each subject, which is an answer obtained by subjective evaluation of the video content using the evaluation axis. A video evaluation device that
In order to estimate a scene corresponding to a predetermined evaluation axis, a selection of a specific evaluation item in the predetermined evaluation axis among the plurality of evaluation axes is accepted, and only the specific evaluation item among all items of the plurality of evaluation axes A subjective evaluation vector creating means for creating a subjective evaluation vector that numerically represents that is selected,
Subjective evaluation similarity calculation for calculating a similarity evaluation vector of N-dimensional vectors by calculating the similarity between the evaluation result data for all items of the plurality of evaluation axes and the subjective evaluation vector for N subjects. Means,
The biosignal similarity between the subjects calculated for each measurement channel using the biosignal data of each of the N subjects and the evaluation result data for the N subjects using the biosignal data. Mapping for selecting mapping information for at least one measurement channel from mapping storage means for storing, for each measurement channel, mapping information calculated in advance for associating the subjective evaluation similarity between subjects calculated in combination A selection means;
A biological signal similarity vector composed of an N-dimensional vector that is a product of a matrix, which is data of time-series biological signals of each of the N subjects, and an unknown vector whose elements are predicted time-series biological signals; The unknown vector so that a relational expression relating the subjective evaluation similarity vector obtained from the subjective evaluation vector with only the specific evaluation item selected by the mapping information for the selected measurement channel is established. The time series estimated value of the biological signal is obtained by performing an operation to determine each element of the above, and based on the obtained time series estimated value of the biological signal as a scene contributing to the evaluation on the received predetermined evaluation axis A scene estimation means for subjective evaluation that determines the scene of the video,
A video evaluation apparatus comprising: Biological signal similarity calculation for calculating, for each measurement channel, a biological signal similarity matrix having biological signal similarity between N × N subjects as each element using the biological signal data of each of the N subjects. Means,
Subjective evaluation using the evaluation result data for all items of the plurality of evaluation axes for the N subjects using the data of the biological signal as a factor and the subjective evaluation similarity between the N × N subjects. A subjective evaluation similarity calculating means for calculating a similarity matrix;
As the mapping information, a set of mapping matrices consisting of two matrices whose elements include N-dimensional vertical vectors associating the biological signal similarity matrix for each measurement channel and the subjective evaluation similarity matrix is obtained. A similarity-to-similarity mapping calculation means stored in the mapping storage means;
The video evaluation apparatus according to claim 1, further comprising: The similarity mapping calculation means includes:
A first vector that is a product of the biological signal similarity matrix for each measurement channel and an unknown first weight vector that is an N-dimensional vertical vector;
An inner product of the subjective evaluation similarity matrix and a second vector that is a product of an unknown second weight vector, which is an N-dimensional vertical vector,
The first weight vector and the second weight vector that are maximized under a constraint condition in which norms of the first vector and the second vector are predetermined constants are calculated as elements of a column of the mapping matrix. The video evaluation apparatus according to claim 2. The mapping selection means includes
A plurality of mapping matrix sets obtained for each measurement channel by the similarity mapping calculation means, and data and evaluation results of biological signals of other subjects not used when obtaining the plurality of mapping matrix sets 4 or 3, wherein at least one measurement channel satisfying a predetermined criterion is determined using the data, and a set of mapping matrices for the determined measurement channel is selected. The video evaluation apparatus described in 1. The subjective evaluation corresponding scene estimation means includes:
The time when the absolute value of the estimated value of the biological signal obtained by performing the calculation for determining each element of the unknown vector having the biological signal of the predicted time series as each element exceeds a predetermined threshold 5. The scene of the video corresponding to is determined as a scene having a large contribution to the evaluation of a specific evaluation item on the received predetermined evaluation axis. 6. The video evaluation apparatus described.   A video evaluation program for causing a computer to function as the video evaluation apparatus according to claim 1.

Images