JP6824146B2 - Evaluation device, method and program - Google Patents

Description

The present invention relates to an evaluation device, method and program for evaluating the evaluation target based on the reaction of the subject to the evaluation target.

As a method for evaluating the quality of experiments and exhibitions, a questionnaire evaluation method has been conventionally used, and the semantic differential method (SD method) is widely known as one of them (for example, Non-Patent Document 1). See). Evaluation by questionnaire can be systematically evaluated. However, since it is an evaluation of one event as a whole, it is not suitable when small events occur continuously in the event and it is desired to evaluate the events sequentially.

On the other hand, when evaluating the state in which a person or robot is exchanging words and movements, interrupting the exchange in the middle of the evaluation may destroy the atmosphere of the place and make accurate evaluation impossible. ,Not appropriate. Therefore, it is important to obtain continuous evaluations in order to make accurate evaluations and make subsequent improvements. Here, improving the algorithm or improving the system so that the evaluation from the subject or the evaluator is high is called learning.

The continuous data includes information related to the evaluator's movements such as facial expression, voice, line of sight, posture, direction, and movement, and physiological information such as heartbeat and sweating. Conventionally, there is known a technique of using line-of-sight information in real time in order to estimate the next speaker in a conversation between humans (see, for example, Non-Patent Document 2).

However, in the prior art, there is a problem that only an adult is an evaluator, or if the evaluator is different, the estimation system cannot be applied, and the applicable attributes of the technology are biased.

In actual field experiments and exhibitions, attributes such as age and gender vary widely, so even if the attributes change, if the learning method is the same, proper learning may not be possible. Especially in the actual field, many parents and children visit.

As has been said in many experiments with children, children may not be able to properly organize or express their own thoughts, unlike adults, and children make direct evaluation judgments. It's difficult to give. Instead, there is known a method of acquiring information related to movements such as facial expressions and gaze and using them as evaluation values (see, for example, Non-Patent Document 3).

Conventionally, facial expressions are estimated according to the attributes of the subject or the evaluator, but when there are a plurality of inputs, the evaluation has not been made in consideration of the input characteristics.

To give an actual example, when evaluating by providing a push button switch, if the evaluator is a child, he / she finds pleasure in the act of pressing the button switch, and the number of times the button switch is pressed and the actual number of times the button switch is pressed. There are many cases where the evaluation is not tied. On the other hand, facial expressions, laughter, gaze, etc. often directly represent emotions and attention, which increases the reliability of evaluation. On the contrary, when the evaluator is an adult, he / she tends to suppress his / her emotional expression in a public place, so that the evaluation of the act of pressing the button switch itself is more reliable than the data such as facial expressions.

Shigeru Ichihara, "Possibilities of Semantic Differential Law (SD Method) and Future Challenges", Ergonomics, 45.5 (2009): 263-269 Ishii, Ryo, et al., “Analysis of Respiration for Prediction of‘ Who Will Be Next Speaker and When? ’In Multi-Party Meetings.”, Proceedings of the 16th International Conference on Multimodal Interaction. ACM, 2014. Yasuhito Sawahata et al., "Relationship between the distribution of eyes when watching a program and the degree of understanding of the program content", The Journal of The Institute of Image Information and Television Engineers 62.4 (2008): 587-594

When learning or the like is performed by inputting a plurality of types of subject information, in the prior art, even if the attributes of the subjects are different, the learning method is unified and does not reflect the attributes of the subjects. However, if the attributes of the subject are different, the ratio of the motion information expressed by the subject such as facial expression and voice changes, and there are cases where the information expressed by the subject and the evaluation itself are not directly linked. In these cases, the conventional technique does not reflect the attributes of the subject, which is a factor that prevents proper learning.

The present invention has been made by paying attention to the above circumstances, and an object of the present invention is that when the evaluation target is evaluated based on the reaction of the subject to the evaluation target, the evaluation target can be evaluated with higher reliability. It is intended to provide such evaluation devices, methods and programs.

In order to solve the above problem, the first aspect of the present invention is an evaluation device that evaluates the evaluation target based on the reaction of the subject to the evaluation target, and operates to express the reaction of the subject to the evaluation target. Data is acquired, attribute data representing the attributes of the subject is acquired, a reliability value representing the degree of reliability of the operation data is calculated based on the attribute data, and based on the operation data and the reliability value. The evaluation value for the evaluation target is calculated.

A second aspect of the present invention is a learning unit that learns an evaluation value for an evaluation target based on the evaluation value calculated by the evaluation value calculation unit, a learning data to be learned, and a learning evaluation value for the learning data. Further includes a learning database that stores the data in association with the above, and determines whether or not the evaluation target is stored as the learning data in the learning database. When it is determined that the evaluation target is not stored as the learning data in the learning database, the evaluation target is used as the learning data, and the evaluation target calculated by the evaluation value calculation unit is evaluated. The value is stored in the learning database as the learning evaluation value, and when it is determined that the evaluation target is stored in the learning database as the learning data, the learning evaluation associated with the learning data is determined. The value is changed according to the calculated evaluation value for the evaluation target.

A third aspect of the present invention is to collect at least one of the biometric data and motion data representing the subject's reaction to the evaluation target over a predetermined period of time, and at least one of the collected biometric data and motion data. On the other hand, normalization processing is performed, and the normalized data is used as the operation data.

A fourth aspect of the present invention separates the motion data into conscious motion data representing the conscious reaction of the subject and unconscious motion data representing the unconscious reaction of the subject, and the conscious motion. A value obtained by calculating the reliability value of the data and the reliability value of the unconscious operation data and multiplying the conscious operation data by the reliability value of the conscious operation data calculated by the calculation processing unit. The evaluation value is calculated by synthesizing the value obtained by multiplying the unconscious operation data by the reliability value of the unconscious operation data calculated by the calculation processing unit.

In the fifth aspect of the present invention, when the value indicating the attribute of the subject is x and the predetermined values for each type of the attribute of the subject are α and β, the confidence value of the conscious movement data is f. (X) = 1 / (1 + exp (-αx + β))
The reliability value of the unconscious operation data is calculated by 1-f (x).

According to the first aspect of the present invention, the evaluation value for the evaluation target is calculated based on the motion data representing the reaction of the subject to the evaluation target and the reliability value calculated based on the attribute data of the subject. .. Therefore, as compared with the case where the evaluation target is evaluated only by the motion data representing the reaction of the subject, it is possible to perform the evaluation with higher reliability reflecting the attributes of the subject.

According to the second aspect of the present invention, learning is performed using the evaluation value for the evaluation target calculated based on the motion data representing the reaction of the subject to the evaluation target and the reliability value calculated based on the attribute data. Is done. That is, learning that reflects the attributes of the subject is performed. Therefore, even when the evaluation is performed using the learning result, the evaluation can be performed with high reliability.

According to the third aspect of the present invention, at least one of the biometric data and the motion data collected over a certain period of time is normalized, so that the evaluation value calculation process based on the biometric data and the motion data is optimal. It becomes possible to change.

According to the fourth aspect of the present invention, the motion data of the subject is separated into the conscious motion data representing the conscious reaction and the unconscious motion data representing the unconscious reaction, and the reliability value for each of these data. Is calculated. Therefore, a more appropriate reliability value can be calculated for each of the conscious reaction and the unconscious reaction of the subject, and thus the reliability of the evaluation value can be further enhanced.

According to the fifth aspect of the present invention, when the values indicating the attributes of the subject are x and the values predetermined for each type of the attribute of the subject are α and β, the confidence value of the conscious movement and the unconscious movement The confidence values of can be calculated as f (x) = 1 / (1 + exp (-αx + β)) and 1-f (x), respectively, which enables more evaluation considering the type of subject's attributes. it can.

That is, according to each aspect of the present invention, when the evaluation target is evaluated based on the reaction of the subject to the evaluation target, an evaluation device, a method and a program capable of performing a more reliable evaluation of the evaluation target are provided. can do.

The block diagram which shows the functional structure of the evaluation apparatus which concerns on one Embodiment of this invention. The flowchart which shows an example of the processing procedure by the evaluation apparatus shown in FIG. FIG. 5 is a flowchart showing an example of a reliability value calculation process among the processes shown in FIG. The flowchart which shows an example of the evaluation value calculation process among the process shown in FIG. The figure which shows the example of the sigmoid function used for the confidence value calculation. The figure which shows the example of the data sent from the subject information detection device to the information processing apparatus.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

[One Embodiment]
(Constitution)
FIG. 1 is a block diagram showing a functional configuration of a system including an evaluation device according to an embodiment of the present invention. In the figure, 1 is an information processing device as an evaluation device, 2 is a subject information detection device, 3 is a subject, 4 is a display device, and 5 is an operator terminal.

The operator terminal 5 is, for example, a personal computer used by an operator of the system, and is used by the operator to set an evaluation target to be presented to a subject. The operator terminal 5 is also used as an output destination of the evaluation result or the learning result obtained by the information processing device 1 described later. However, the operator terminal 5 is not an indispensable component, and may be built in the information processing device 1 or omitted.

The display device 4 is, for example, a display used for presenting a sentence, an image, a video, or the like to be evaluated to the subject 3. It is also possible to replace the evaluation target with a speaker or the like that outputs voice. Further, the display device 4 can be built in the information processing device 1, or can be omitted in the case where the actual product such as an exhibit or a demonstration is to be evaluated.

The subject information detection device 2 includes an operation information detection unit 21, an attribute information detection unit 22, and an output unit 23.

The motion information detection unit 21 includes a motion information detection unit 211 and a biological information detection unit 212. The motion information detection unit 211 detects the reaction of the subject 3 to the evaluation target presented to the subject by the display device 4 as information representing the motion of the subject. Information representing the movement of the subject includes, for example, the unconscious movement of the subject such as the facial expression, voice, line of sight, posture, and orientation of the subject. This unconscious movement can be detected from, for example, an image of the subject's face or figure captured by a camera, detected from the sound picked up by a microphone, an acceleration sensor or a distance measuring sensor attached to the subject's body, or the like. Can be detected using. Examples of data representing facial expressions include numerical data representing each ratio of joy, surprise, anger, sadness, and expressionlessness, and smile degree data. The data representing this facial expression is obtained by, for example, a known facial image analysis technique from the facial image of the subject captured by the camera, and a specific example will be described in detail later.

Further, the motion information detection unit 211 includes a push button switch that can be operated by the subject. Pushbutton switches are used to detect the subject's conscious movements. For example, two pushbutton switches are prepared. One pushbutton switch is used to enter the intention of the subject when he / she finds it interesting to the object to be evaluated. On the other hand, the other push button switch is used to input the intention when the subject feels that the evaluation object is not interesting. Further, the level of fun and the level when it is not interesting can be input by, for example, the number of times the push button switch is pressed or the pressing time.

The biological information detection unit 212 detects the reaction of the subject 3 to the evaluation target presented to the subject by the display device 4 as the biological information or the physiological information of the subject. Examples of biological information or physiological information include the heartbeat and sweating of the subject. For the detection of these biological information, for example, a Holter electrocardiograph worn on the subject can be used.

The motion information detection unit 21 does not necessarily have to include both the motion information detection unit 211 and the biometric information detection unit 212, and the motion information to be detected can be arbitrarily set by the system administrator or the like.

The attribute information detection unit 22 detects or acquires the attribute information of the subject 3. The attribute information detected or acquired by the attribute information detection unit 22 includes, for example, age, gender, occupation, and the like. For example, the attribute information detection unit 22 acquires the attribute information by reading the personal information related to the above attributes from a database on the Web in which the subject's attribute information is recorded, a smartphone owned by the subject, or the like. Alternatively, the operator can input it arbitrarily. Further, the attribute information detection unit 22 may detect the attribute information by using software or the like that automatically estimates the age or the like from the face image of the subject.

The output unit 23 informs the motion data including the motion information and the biological information detected as the motion information by the motion information detection unit 21 and the attribute data detected as the attribute information by the attribute information detection unit 22 at a predetermined cycle. Output to processing device 1. Hereinafter, the motion data and the attribute data are collectively referred to as subject data.

The information processing device 1 is composed of, for example, a personal computer or a server device, and includes an input / output interface unit 11, a processing unit 12, and a storage unit 13.

The input / output interface unit 11 has, for example, a wired or wireless interface, and has a function of receiving various information output from an external device, for example, a subject information detection device 2 or an operator terminal 5, and an evaluation target, an evaluation result, or a learning result. It has a function of transmitting information representing the above. As the wired interface, for example, a LAN (Local Area Network) is used, and as the wireless interface, an interface adopting a low power wireless data communication standard such as Bluetooth (registered trademark) is used. The subject information detection device 2 may be built in the information processing device 1.

The storage unit 13 uses a non-volatile memory such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that can be written and read at any time as a storage medium, and in order to realize this embodiment. In addition to the program storage unit, a data storage unit 131, a reliability evaluation database (DB) 132, a learning database (DB) 133, and an output data storage unit 134 are provided as necessary storage areas.

The processing unit 12 has a CPU (Central Processing Unit), and as processing functions necessary for implementing this embodiment, the evaluation target acquisition unit 121, the output processing unit 122, the data acquisition unit 123, and the normalization A unit 124, a reliability value calculation unit 125, an evaluation value calculation unit 126, and a learning unit 127 are provided. All of these processing functions are realized by causing the CPU to execute an application program stored in a program storage unit (not shown).

Next, the configuration included in the processing unit 12 of the information processing device 1 will be further described. In the following, in particular, the facial expression data of the subject and the information input by the push button switch indicating whether the evaluation target is interesting or not are input as the motion data, and the information indicating the age is input as the attribute data. The case will be described as an example, but it can be replaced with other data, and other data can be additionally used.

The evaluation target acquisition unit 121 acquires information representing the evaluation target to be presented to the subject. For example, the information of the evaluation target set by the operator is acquired from the operator terminal 5. In addition, information on the evaluation target registered in advance can be acquired from a database or the like on the Web (not shown) via a network (not shown), or the learning result stored in the learning database 133 can be acquired as information to be evaluated. it can. The evaluation target acquisition unit 121 sends information representing the acquired evaluation target to the output processing unit 122.

The output processing unit 122 stores the information representing the evaluation target sent from the evaluation target acquisition unit 121 in the output data storage unit 134, and among the stored information representing the evaluation target, the evaluation target to be presented to the subject is stored. The information to be represented is output from the input / output interface unit 11 to the display device 4. Further, the output processing unit 122 also outputs the information representing the evaluation target output to the display device 4 to the learning unit 127 together with the output time.

Further, the output processing unit 122 selectively reads out the data representing the evaluation result calculated by the evaluation value calculation unit 126 and the data representing the learning result stored in the learning database 133 according to the request of the operator, and is an input / output interface unit. It is transmitted from 11 to the display device 4 or the operator terminal 5. The output processing unit 122 can also transmit the evaluation result or the data representing the learning result to another external device or the like (not shown).

For example, the output processing unit 122 reads out the data stored in the output data storage unit 134 at a predetermined time and outputs the data. The data set to be output is, for example, a set of the output time and the output target data. As an example of the output target data, data representing an evaluation target such as a sentence to be presented to the subject can be considered.

The data acquisition unit 123 acquires subject data output from the subject information detection device 2, that is, motion data including motion information and biological information, and attribute data at a predetermined cycle via the input / output interface unit 11. , Stored in the data storage unit 131.

The normalization unit 124 reads out the motion data, the biometric data, and the attribute data stored in the data storage unit 131 for a fixed period of time, and performs resampling processing on the motion data, the biometric data, and the attribute data for the fixed time. Further, the normalization unit 124 performs normalization processing on the read motion data and biometric data.

For example, when the resampling cycle is 1 minute, the normalization unit 124 calculates the average value of each of the motion data and the biometric data every 1 minute. The resampling cycle can be changed arbitrarily. Further, the normalization unit 124 further normalizes the resampled motion data and biometric data. An example of the normalization process will be described in detail later.

The reliability value calculation unit 125 uses the reliability evaluation DB 132 to calculate a reliability value indicating the degree of reliability of the motion data of the subject based on the attributes of the subject. The reliability evaluation DB 132 stores a defined model for each attribute of the subject.

For example, in one embodiment, the reliability value calculation unit 125 separates the motion data and the biological data of the subject normalized by the normalization unit 124 into conscious motion data and unconscious motion data. Then, the reliability value calculation unit 125 follows the specified model for each attribute of the subject stored in the reliability evaluation DB 132, and based on the attribute data resampled by the normalization unit 124, the reliability value and the unconsciousness of the conscious motion data. Calculate the reliability value of the target operation data. An example of the reliability value calculation process will be described in detail later. The reliability value calculation unit 125 outputs the calculated reliability value to the evaluation value calculation unit 126 together with the conscious operation data and the unconscious operation data.

The evaluation value calculation unit 126 calculates an evaluation value for each of the conscious operation data and the unconscious operation data output from the reliability value calculation unit 125, taking into account the reliability value output from the reliability value calculation unit 125 at the same time. For example, in one embodiment, the evaluation value calculation unit 126 multiplies the conscious motion data by the confidence value of the conscious motion data, and multiplies the unconscious motion data by the confidence value of the unconscious motion data. Then, the evaluation value calculation unit 126 calculates the evaluation value of the operation data by summing the conscious operation data multiplied by the reliability value and the unconscious operation data multiplied by the reliability value.

That is, the evaluation value calculation unit 126 weights the subject's motion information and biological information detected by the subject information detection device 2 for each of the conscious motion data and the unconscious motion data based on the attribute information of the subject. Make an evaluation. The evaluation value calculation unit 126 outputs the calculated evaluation value together with, for example, time data to the output processing unit 122 and the learning unit 127.

Each time the evaluation value is calculated by the evaluation value calculation unit 126, the learning unit 127 performs a learning process on the evaluation value based on the evaluation value and the learning result stored in the learning DB 133, and this learning process The later evaluation value is stored in the learning DB 133. An example of the above learning process will also be described in detail later.

(motion)
Next, the information processing operation by the information processing device 1 as the evaluation device configured as described above will be described. FIG. 2 is a flowchart showing the processing procedure and the processing content.

(1) Presentation of information representing an evaluation target Here, a case where text data is presented as an evaluation target will be described as an example. When the operator inputs text data to be evaluated on the operator terminal 5, the text data is transmitted from the operator terminal 5 to the information processing device 1.

Under the control of the evaluation target acquisition unit 121, the information processing apparatus 1 monitors in step S10 whether or not the data to be evaluated is received. When the text data is sent from the operator terminal 5 in this state, the text data transmitted from the operator terminal 5 in step S11 is taken in via the input / output interface unit 11 and passed to the output processing unit 122. The output processing unit 122 stores the text data in the output data storage unit 134 and transmits the text data from the input / output interface unit 11 to the display device 4. As a result, the text data is displayed on the display device 4, and the text data is presented to the subject 3 as an evaluation target. It is also possible to acquire the text data to be evaluated from the database on the WEB or the learning DB 133 and display it on the display device 4.

The output processing unit 122 also outputs the sentence data output to the display device 4 to the learning unit 127 together with the data indicating the output time for the learning process described later.

(2) Acquisition of subject data In the subject information detection device 2, the subject inputs an evaluation of the text data displayed on the display device 4, for example, whether it is interesting or not by operating a push button switch. Then, the operation information of the push button switch is detected by the motion information detection unit 211.

Further, in the motion information detection unit 211, the subject's unconscious movements such as the facial expression, voice, line of sight, posture, and orientation of the subject who read the above sentence data are detected in a constant detection cycle of the face image, voice, position information, and body. Detected by acceleration data representing motion. Of these, facial expressions are classified into, for example, joy, surprise, anger, sadness, and expressionless, and their levels are quantified. The input information by the detected push button switch and the detection information such as the facial expression of the subject are transmitted from the output unit 23 to the information processing device 1 together with the identification information (subject ID) of the subject as motion detection information.

In parallel with the motion detection operation by the motion information detection unit 211, the biometric information detection unit 212 displays the biometric information of the subject who has read the text data, such as heartbeat and sweating, in the electrocardiograph and the electrocardiograph in the constant detection cycle. It is detected by a polygraph meter or the like, and the detected data is transmitted from the output unit 23 to the information processing device 1 together with the subject ID.

Further, in the subject information detection device 2, the attribute information detection unit 22 detects the attribute information of the subject, for example, age, gender, and occupation. The detection of this attribute information is performed, for example, by acquiring information on the attributes of the subject from a database on the Web, a smartphone owned by the subject, or the like, or is input by the operator. The age and the like may be estimated from the facial image of the subject. The attribute information is transmitted from the output unit 23 to the information processing device 1 together with the subject ID.

Here, FIG. 6 is a diagram showing an example of subject data output from the output unit 23 of the subject information detection device 2. In the example of FIG. 6, each ratio of joy (Joy), surprise (Surprise), anger (Angry), sadness (Sorrow), and expressionless (Expressionless) is acquired as the expression data, and the subject is interesting as the button data. The number of times the push button switch (“interesting button”) was pressed to operate when it was felt (Funny), and the push button switch (“uninteresting button”) to operate when the subject felt uninteresting. The number of times (Nofunny) is pressed is acquired and sent to the information processing apparatus 1 together with the time and the subject ID at a predetermined cycle. For example, in the example of FIG. 6, only the joy (Joy) data can be used for evaluation, or the ratio of joy (Joy) and anger (Angry) can be used for evaluation, and similarly, an interesting button is pressed. Only the number of times (Funny) can be used for evaluation, or the ratio of the number of times an interesting button is pressed (Funny) to the number of times an uninteresting button is pressed (Nofunny) can be used for evaluation, such as the nature of the evaluation target. The system administrator or the like can freely set according to the above.

The data acquired by the operation information detection unit 21 and the attribute information detection unit 22 may be acquired by a plurality of devices or by one device. FIG. 6 shows an example of data when facial expression data and age data are acquired by one device and button data is acquired by another device. As shown in FIG. 6, when the sampling number of each device is different, the data that has not been acquired at a certain time is transmitted as a blank.

The data in FIG. 6 is an example of data acquired by using commercially available software, and various methods can be used for facial expression analysis.

Under the control of the data acquisition unit 123, the information processing device 1 monitors the presence / absence of reception of subject data in step S12. In this state, when subject data, that is, motion data, biological data, and attribute data are sent from the subject information detection device 2, the subject data is received via the input / output interface unit 11 in step S13, and the data storage unit 131. To memorize. After that, each time the subject information detection device 2 sends the subject data, the subject data is received and stored in the data storage unit 131.

(3) Normalization processing Under the control of the normalization unit 124, the information processing device 1 reads the subject data for the fixed time from the data storage unit 131 in step S14 every time a fixed time elapses, and reads the subject data. Of the subject data, the motion data, the biometric data, and the attribute data are resampled, and the motion data and the biometric data are normalized.

For example, the resampling cycle is set to 1 minute, and the average value for each minute is calculated for each of the motion data and the biometric data. In the normalization process, the total value of the motion data and the biometric data is converted to 1 for each subject ID in a predetermined normalization cycle. More specifically, when the total value of the detected values of the facial expression data is 1200 and the total number of presses by the push button switch is 80 in the normalization cycle of 15 minutes, the facial expression data for each minute is divided by 1200. The number of presses is divided by 80. Note that the conversion of the normalization process does not have to be a conversion in which the total value is 1, a conversion in which the distribution has a Gaussian distribution, or another conversion process may be used. The resampling-processed attribute data and the normalized motion data and biometric data are sent to the reliability value calculation unit 125.

(4) Confidence value calculation process In step S15, the information processing apparatus 1 calculates the reliability value of the operation data under the control of the reliability value calculation unit 125 as follows.

FIG. 3 is a flowchart showing an example of the reliability value calculation process.

In step S151, the reliability value calculation unit 125 acquires the resampling-processed attribute data and the resampling-processed and normalized-processed motion data and biometric data from the normalization unit 124.

Next, in step S152, the reliability value calculation unit 125 separates the motion data, that is, the motion data and the biometric data into the conscious motion data and the unconscious motion data. Examples of conscious motion data include questionnaires, applause, and the number of times a pushbutton switch is pressed. Examples of unconscious motion data include physiological information such as facial expressions, heartbeats, and sweating. Some data, such as voice, line of sight, posture, orientation, etc., can be classified into either conscious movement data or unconscious movement data, and the system administrator etc. decides in advance which attribute should be classified into which. Can be kept.

Next, in step S153, the reliability value calculation unit 125 calculates the reliability value of the conscious operation data according to the attribute data based on the reliability evaluation DB 132. In the reliability evaluation DB 132, a defined model for each attribute of the subject is stored in advance. The default model is defined, for example, by the sigmoid function. An example is shown in FIG. The reliability value calculation unit 125 sets the reliability value of the conscious movement data as f (x) = 1 when the value indicating the attribute of the subject as a numerical value is x and the parameters predetermined for each type of attribute are α and β. / (1 + exp (-αx + β))
Calculated by

Next, in step S154, the reliability value calculation unit 125 calculates the reliability value of the unconscious operation data. As an example, the confidence value of the unconscious motion data is calculated by subtracting the confidence value of the conscious motion data from 1, that is, 1-f (x).

For example, when the type of attribute is age, the initial values of α = 1/5 and β = -40 are taken. The values of α and β can be updated by fitting the sigmoid function at a predetermined update timing. As an example of the fitting method, when the attribute of the subject is x, the ratio of the value of the conscious movement data to the total value of the unconscious movement data and the conscious movement data can be f (x). .. It is also possible to set α and β to arbitrary fixed values.

In step S155, the reliability value calculation unit 125 outputs the conscious operation data, the unconscious operation data, the reliability value of the conscious operation data, and the reliability value of the unconscious operation data to the evaluation value calculation unit 126.

(5) Evaluation value calculation process In step S16, the information processing apparatus 1 evaluates the operation data based on the operation data output from the reliability value calculation unit 125 and the calculated reliability value by the evaluation value calculation unit 126. Calculate the value as follows.

FIG. 4 is a flowchart showing an example of the evaluation value calculation process.

In step S161, the evaluation value calculation unit 126 obtains the operation data from the reliability value calculation unit 125, that is, the above-mentioned conscious operation data and unconscious operation data, and the reliability value of the conscious operation data and the reliability value of the unconscious operation data. get.

In step S162, the evaluation value calculation unit 126 applies the reliability value as a weight to the operation data to calculate the evaluation value. As an example, the value obtained by multiplying the conscious motion data by the confidence value of the conscious motion data is calculated as the evaluation value of the conscious motion data, and the value obtained by multiplying the unconscious motion data by the confidence value of the unconscious motion data is unconscious. It is calculated as an evaluation value of the target operation data. Next, the evaluation value of the calculated conscious movement data and the evaluation value of the unconscious movement data are combined. For the synthesis, for example, an addition process is used. Thus, the evaluation value of the operation data is calculated as the operation data evaluation value.

As a result, when a plurality of types of information are input, each information can be weighted based on the attributes of the subject and used for evaluation. When a plurality of types of attributes are used, a value obtained by further multiplying the reliability value for each attribute can be calculated as an evaluation value. As an example, if the confidence value obtained for attribute 1 (for example, age) is f ₁ (x) and the confidence value obtained for attribute 2 (for example, gender) is f ₂ (x), the subconscious action. The data multiplied by f ₁ (x) × f ₂ (x) can be used as the evaluation value of the conscious operation data, and the unconscious operation data is (1-f ₁ (x)) × (1-f). ₂ (x)) can be multiplied as the evaluation value of the unconscious motion data.

In step S163, the evaluation value calculation unit 126 outputs the calculated operation data evaluation value. As an example, the evaluation value calculation unit 126 outputs the calculated operation data evaluation value together with data indicating the association with the evaluation target, for example, data indicating the time when the operation data is acquired by the subject information detection device 2. It can be sent to 122 and the learning unit 127.

(6) Learning process Under the control of the learning unit 127, the learning processing device 1 sets the operation data evaluation value newly calculated by the evaluation value calculation unit 126 and the learning result stored in the learning DB 133 in step S17. Based on this, the learning process is performed on the operation data evaluation value as follows.

That is, the learning DB 133 stores the learning data learned so far and the learning evaluation value for each learning data, and the learning unit 127 stores the time data and the operation data evaluation value sent from the evaluation value calculation unit 126. Then, learning processing is performed on the operation data evaluation value based on the output time and evaluation target data sent from the output processing unit 122 and the past learning result stored in the learning DB 133.

Hereinafter, an example in which an interesting sentence is used as an example of learning data will be described. Examples of sentences that you find interesting include puns, short stories, and questions and answers (for example, the subject "I don't like this XX" and a set of answers to put in XX). However, the learning data is not limited to text data, and can be arbitrarily set by the system administrator or the like.

The learning DB 133 stores the sentence data learned so far and the fun evaluation value for each sentence data. As an example of the initial setting, the learning DB 133 stores the sentence data and the fun evaluation value of each sentence data classified by, for example, the random forest method or the neural network as the initial value. The classification can be a binary value of 0 or 1, or a rate can be assigned. The learning method can also be set arbitrarily by the system administrator.

The learning unit 127 associates the evaluation target data sent from the output processing unit 122 with the operation data evaluation value sent from the evaluation value calculation unit 126. More specifically, the learning unit 127 identifies the data indicating the output time of the evaluation target sent from the output processing unit 122 together with the evaluation target data, and includes time information corresponding to the output time to calculate the evaluation value. The operation data evaluation value sent from the unit 126 is identified, and the evaluation target data and the operation data evaluation value are associated with each other.

Next, the learning unit 127 determines whether or not the evaluation target data, that is, the sentence data to be evaluated is stored as learning data in the learning DB 133. When it is determined that the learning data is not stored in the learning DB 133, the learning unit 127 uses the evaluation target data as the learning data and the operation data evaluation value associated with the evaluation target data as the learning evaluation value. Update. When it is determined that the evaluation target data has already been stored as learning data in the learning DB 133, the learning unit 127 updates only the learning evaluation value of the learning DB 133 with the operation data evaluation value associated with the evaluation target data. be able to. The learning result stored in the learning DB 133 can be output through the output processing unit 122 at any time in response to a request from the operator or the like.

(7) Output processing The information processing apparatus 1 reads the evaluation value data updated by the learning process from the learning DB 133 and transmits it to the operator terminal 5 in step S18 under the control of the output processing unit 122. In the output processing unit 122, the evaluation value calculated in step S16, the reliability value obtained in step S15, the normalized motion detection information and the biological detection information obtained in step S14, and the attribute information, or It is also possible to output the subject data obtained in step S13. The output destination may be the display device 4, or may be an external terminal designated in advance by the operator.

As an example, when the learning result by the learning unit 127 is output to an external device such as the operator terminal 5, for example, the learning result can be output in order from the one having the highest learning evaluation value in the learning DB 133.

(effect)
As described in detail above, in one embodiment of the present invention, the text data set by the operator of the system or the like is acquired as an evaluation target, the text data is output and presented to the subject, and the subject with respect to the presented text data. Detect the reaction of. In particular, among the reactions of the subject, the facial image of the subject is detected as the number of times the push button switch is pressed as a conscious action representing the conscious reaction of the subject, and as an unconscious action representing the unconscious reaction of the subject. Detects numerical values that represent the percentages of joy, surprise, anger, sadness, and expressionlessness obtained from. In addition, the age of the subject is acquired or estimated as the attribute data representing the attribute of the subject. Using a prescribed model corresponding to the attribute as age, a reliability value representing the degree of trust of each of the conscious movement data and the unconscious movement data is calculated from the age of the subject. Then, the evaluation value of the entire motion data is calculated by multiplying each of the conscious motion data and the unconscious motion data by their respective trust values and then synthesizing them. Further, based on the time when the evaluation target is output for presentation to the subject and the time when the motion data is acquired, the evaluation target and the evaluation value for the evaluation target are associated with each other and used for learning.

Therefore, according to one embodiment, the evaluation target can be flexibly set and the evaluation can be performed by inputting by the operator or using a predetermined algorithm. In addition, since the data collected over time can be used, it is possible to perform not only the overall evaluation but also the sequential evaluation for each event included in the evaluation target group including a plurality of events. In particular, even for a series of events that are continuous in time, a more detailed and accurate evaluation can be performed for a desired timing and a desired period without interrupting the event. Furthermore, since the evaluation can be performed in consideration of the attributes, it is possible to have the subject or the evaluator regardless of the age and gender of the visitors, and the evaluation based on a larger number of samples can be performed. In addition, by detecting a type of reaction that is difficult for adults to express from a child subject, it is possible to perform various evaluations that were difficult to obtain in the past.

Even when a plurality of types of motion information are used for input, each of them can be weighted based on the attributes of the subject and used for the evaluation, so that a more reliable evaluation can be performed. In addition, by setting different models according to the attributes, the system administrator or the like can freely set the evaluation environment, such as strengthening the factor of age difference rather than gender difference. The model according to the attribute can be optimized by fitting a sigmoid function or the like. Further, it is possible to calculate a more detailed evaluation value in consideration of a plurality of attributes. Further, since the evaluation value calculation method can be applied even if the types of operation data and attribute data to be acquired increase, the system administrator can freely set the data used for evaluation.

Further, according to one embodiment, since the ratios of each of the five types of facial expressions are detected, normalized and used for the evaluation, more complicated and diverse evaluations can be performed as compared with the case where only a smile is detected. it can. In addition, two pushbutton switches are prepared, and not only the number of times the subject pressed the button switch when he / she felt it was interesting, but also the number of times he / she pressed it when he / she felt it was not interesting was used for evaluation, so it was more accurate. Evaluation can be performed.

[Other Embodiments]
The present invention is not limited to the above embodiment. For example, although a push button switch that can be operated by the subject is shown as an example of the detection device included in the motion information detection unit 211, it is also possible to use a keyboard or display on which the subject can input an evaluation score, an impression, or the like. Further, a volume sensor, a vibration sensor, or the like that detects the applause of the subject can also be adopted. It is also possible to acquire position information with a distance measuring sensor or the like. Moreover, it is also possible to use not only an acceleration sensor but also an angular velocity sensor, a geomagnetic sensor and the like.

Further, although the embodiment in which the evaluation target is presented via the output processing unit 122 of the information processing apparatus 1 has been described, it is not always necessary to pass through the output processing unit 122. Further, as described above, the method of presenting the evaluation target is not limited to the presentation on the display device 4. The evaluation target can be output on the display device 4 using another external device, can be output as audio by a speaker (not shown), or can be presented by a demonstration. It is possible.

The sigmoid function is shown as an example of the specified model used for reliability evaluation, but various other functions can be used. Further, although the method of obtaining the reliability value by separating the operation data into the conscious operation data and the unconscious operation data has been illustrated, it is also possible to obtain the reliability value according to the attribute without separating the operation data.

Further, although the learning example of determining whether or not the learning data is in the learning DB and updating the learning data has been described, the learning method can be appropriately set by the system administrator.

In addition, the types of operation data and attribute data, their acquisition means, the method of calculating the reliability value, a series of processing procedures, processing contents, and the like can be variously modified and implemented without departing from the gist of the present invention.

In short, the present invention is not limited to the above-described embodiment as it is, and at the implementation stage, the components can be modified and embodied without departing from the gist thereof. In addition, various inventions can be formed by an appropriate combination of a plurality of components disclosed in the above-described embodiment. For example, some components may be removed from all the components shown in the embodiments. In addition, components from different embodiments may be combined as appropriate.

1 ... Information processing device, 2 ... Subject information detection device, 3 ... Subject, 4 ... Display device, 5 ... Operator terminal, 11 ... Input / output interface unit, 12 ... Processing unit, 13 ... Storage unit, 21 ... Operation information detection unit , 22 ... Attribute information detection unit, 23 ... Output unit, 121 ... Evaluation target acquisition unit, 122 ... Output processing unit, 123 ... Data acquisition unit, 124 ... Normalization unit, 125 ... Reliability value calculation unit, 126 ... Evaluation value calculation Units 127 ... Learning unit, 131 ... Data storage unit, 132 ... Reliability evaluation database, 133 ... Learning database, 134 ... Output data storage unit, 211 ... Motion information detection unit, 212 ... Biological information detection unit.

Claims (6)

An evaluation device that evaluates the evaluation target based on the reaction of the subject to the evaluation target.
A first acquisition unit that acquires motion data representing the reaction of the subject to the evaluation target, and
A separation unit that separates the motion data into conscious motion data representing the conscious reaction of the subject and unconscious motion data representing the unconscious reaction of the subject.
A reliability value calculation processing unit that calculates a reliability value of the conscious operation data and a reliability value of the unconscious operation data,
It is provided with an evaluation value calculation unit that calculates an evaluation value for the evaluation target based on the conscious movement data, the unconscious movement data, the reliability value of the conscious movement data, and the reliability value of the unconscious movement data. Evaluation device. The evaluation value calculation unit was obtained by multiplying the conscious motion data by the reliability value of the conscious motion data and the unconscious motion data by the confidence value of the unconscious motion data. The evaluation device according to claim 1, wherein the evaluation value is calculated by synthesizing the value with the value. A second acquisition unit for acquiring attribute data representing the attributes of the subject is further provided.
The evaluation device according to claim 1 or 2, wherein the reliability value calculation processing unit calculates a reliability value of the conscious operation data and a reliability value of the unconscious operation data based on the attribute data. When the value indicating the attribute of the subject is x and the predetermined values for each type of the attribute of the subject are α and β, the reliability value calculation processing unit sets the reliability value of the conscious movement data to f ( x) = 1 / (1 + exp (-αx + β))
The reliability value of the unconscious operation data is calculated by 1-f (x).
The evaluation device according to claim 3 . An evaluation device including a processor and a storage unit is an evaluation method for evaluating the evaluation target based on the reaction of the subject to the evaluation target.
Acquiring motion data representing the reaction of the subject to the evaluation target,
Separating the motion data into conscious motion data representing the conscious reaction of the subject and unconscious motion data representing the unconscious reaction of the subject.
To calculate the reliability value of the conscious operation data and the reliability value of the unconscious operation data,
An evaluation method comprising calculating an evaluation value for the evaluation target based on the conscious movement data, the unconscious movement data, the reliability value of the conscious movement data, and the reliability value of the unconscious movement data. .. A program that causes a processor to execute each means of the evaluation device according to any one of claims 1 to 4 .