JP2019159586A - Impression derivation system, impression derivation method and impression derivation program - Google Patents

Abstract

To provide an impression derivation system, an impression derivation method and an impression derivation program capable of objectively evaluating an impression of a facing object person of communication to be given to a third person.SOLUTION: An impression derivation system includes: an acquisition unit for acquiring any one or both of a feeling value being a value obtained by digitizing a parameter showing a feeling of an object person derived on the basis of information acquired by a sensor and an action value being a value obtained by digitizing a parameter showing an action; a storage unit for storing impression derivation information being information obtained by associating information showing an impression with information for deriving an evaluation value of the impression created by using a plurality of any one or both of the parameter showing the feeling and the parameter showing the action; and a derivation unit for deriving the evaluation value of the impression of the object person to be given to a third person on the basis of any one or both of the feeling value and the action value acquired by the acquisition unit, and the impression derivation information stored in the storage unit.SELECTED DRAWING: Figure 2

Description

  Embodiments described herein relate generally to an impression derivation system, an impression derivation method, and an impression derivation program.

There are many scenes where face-to-face communication such as interviews and training is evaluated. In most of the situations where face-to-face communication is evaluated, people evaluate qualitatively based on their own subjectivity. For example, the following (1)-(4) are mentioned as an example of the scene which evaluates face-to-face communication.
(1) The interviewer evaluates the suitability of the candidate during the company recruitment interview.
(2) In training to strengthen sales capabilities, the instructor evaluates the students and teaches exemplary behavior.
(3) In negotiations with the customer, the sales representative advances the talk while grasping the degree of understanding of the customer.
(4) At corporate seminars and school classes, the instructor advances the conversation while watching the audience's reaction.

  A technique for efficiently and accurately evaluating the response at the time of customer service by a store clerk is known (see, for example, Patent Document 1). In this technology, the management device acquires individual sales data from the POS register, acquires the face image data of the salesclerk corresponding to customer service from the camera, and is in service from the face image data corresponding to the customer service time zone included in the individual sales data. The customer clerk's facial expression is identified, and a customer service point indicating an evaluation value for the customer service is calculated according to the facial expression. In addition, the management device manages individual sales data and customer service points in association with each other, and sums up these data to quantify and output customer service responses for each store clerk. The evaluation value is related to and output.

JP 2017-102846 A

The evaluation of face-to-face communication based on the subjectivity of a specific person has poor objectivity and lacks the satisfaction of the person being evaluated, and may not lead to an accurate improvement in communication. Specifically, because the candidate's evaluation differs depending on the interviewer, there are cases where appropriate matching cannot be performed. Participants may not be able to correctly recognize differences in behavior between themselves and good salespeople. There may be cases where accurate explanations according to customer understanding cannot be made. The audience may not be able to practice the lecture structure and how to communicate.
In view of the above circumstances, an object of the present invention is to provide an impression derivation system, an impression derivation method, and an impression derivation program that can objectively evaluate an impression given to a third party of a person to be face-to-face communication.

  According to one aspect of the present invention, an emotion value that is a value obtained by quantifying a parameter indicating a subject's emotion derived based on information acquired by a sensor, and an action value that is a value obtained by quantifying a parameter indicating behavior In order to derive an evaluation value of an impression created using a plurality of one or both of an acquisition unit that acquires one or both, information indicating an impression, a parameter indicating an emotion, and a parameter indicating an action A storage unit that stores impression derivation information, which is information associated with the information, and one or both of the emotion value and the action value acquired by the acquisition unit, and the impression stored in the storage unit An impression derivation system comprising: a derivation unit that derives an evaluation value of an impression given to a third party of the subject based on derivation information.

  One aspect of the present invention is the above-described impression derivation system, wherein the storage unit stores impression derivation information for each of the scenes for deriving an evaluation value of an impression given to a third party of the subject person, The derivation unit derives an evaluation value of an impression given to the third person of the target person based on impression derivation information corresponding to a scene from which an evaluation value of an impression is derived among the plurality of scenes.

  One aspect of the present invention is the impression derivation system described above, wherein either one of a parameter indicating an emotion and a parameter indicating an action are included in the information for deriving the evaluation value of the impression included in the impression derivation information Or both depend on the attribute of the third party.

  One aspect of the present invention is the impression derivation system described above, in which time series data of the evaluation value of the impression derived by the derivation unit or a comparison result between the evaluation value of the impression and the evaluation value of another person's impression is obtained. An analysis unit is provided.

  One aspect of the present invention is the impression derivation system described above, wherein the analysis unit is one of a parameter indicating the emotion and a parameter indicating the action that affects the evaluation value of the impression based on the comparison result. One or both are extracted.

  One aspect of the present invention is the impression derivation system described above, in which the storage unit stores an explanation procedure and an explanation reference time, and the acquisition unit includes information acquired by a terminal device used by the subject. The behavior value of the subject derived based on the explanation procedure and the explanation reference time stored in the storage unit is acquired.

  One aspect of the present invention is the impression derivation system described above, and the information acquired by the sensor includes one or both of information related to the subject and information related to the third party.

  According to one aspect of the present invention, an emotion value that is a value obtained by quantifying a parameter indicating a subject's emotion derived based on information acquired by a sensor, and an action value that is a value obtained by quantifying a parameter indicating behavior A method for deriving an evaluation value of an impression created using a plurality of one or both of a step of acquiring one or both, information indicating an impression, a parameter indicating an emotion, and a parameter indicating an action The evaluation value of the impression given to the third party of the target person is derived based on the information associated with the information and one or both of the emotion value and the behavior value acquired in the acquiring step. And an impression derivation method executed by an impression derivation system.

  According to one embodiment of the present invention, an action value that is a value obtained by quantifying an emotion value that is a value obtained by quantifying a parameter that represents an emotion of a target person derived based on information acquired by a sensor and a parameter that indicates an action Deriving an evaluation value of an impression created by using one or both of a step of acquiring one or both of a value, information indicating an impression, and a parameter indicating an emotion and a parameter indicating an action Evaluation value to be given to the third person of the target person based on the information associated with the information to be performed and one or both of the emotion value and the action value acquired in the acquiring step Is a program that executes the step of deriving.

  According to the embodiment of the present invention, it is possible to provide an impression derivation system, an impression derivation method, and an impression derivation program that can objectively evaluate an impression given to a third party of a person to be face-to-face communication.

It is a figure which shows an example of the impression derivation system of embodiment. It is a block diagram showing an example of an impression derivation device, a cloud server, and a client terminal included in the impression derivation system of the embodiment. It is a figure which shows an example of emotion value action value information. It is a figure which shows an example of impression derivation information. It is a figure which shows an example of impression information DB. It is a figure which shows an example of the screen which performs evaluation of an impression. It is a figure which shows an example of the display screen of an impression evaluation result. It is a sequence chart which shows an example of operation | movement of the impression derivation system of embodiment.

Next, an impression derivation system, an impression derivation method, and an impression derivation program according to this embodiment will be described with reference to the drawings. Embodiment described below is only an example and embodiment to which this invention is applied is not restricted to the following embodiment.
Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiments, and the repetitive description will be omitted.
Further, “based on XX” in the present application means “based on at least XX”, and includes a case based on another element in addition to XX. Further, “based on XX” is not limited to the case where XX is directly used, but also includes the case where it is based on an operation or processing performed on XX. “XX” is an arbitrary element (for example, arbitrary information).

(Embodiment)
(Impression derivation system)
FIG. 1 is a diagram illustrating an example of an impression derivation system according to an embodiment.
The impression derivation system 10 of the embodiment includes an imaging device 20, a microphone 30, a terminal device 40, a client terminal 400, an impression derivation device 100, and a cloud server 200.
The imaging device 20, the microphone 30, the terminal device 40, the client terminal 400, the impression derivation device 100, and the cloud server 200 are connected via a network 50 such as the Internet in a wired or wireless manner.
An example of the installation location of the imaging device 20, the microphone 30, and the terminal device 40 is a facility 60 such as a training facility. In the facility 60 of the present embodiment, face-to-face communication training is performed.

Have the person who receives the impression evaluation (hereinafter referred to as “subject”) communicate face-to-face with a third party. During the communication, information on one or both of the target person and the third party is acquired. Specifically, the imaging device 20 images the subject, the microphone 30 records the contents of communication, and the terminal device 40 acquires document data such as materials used for communication and an operation log during communication. .
The imaging device 20 creates moving image data including the target person ID and moving image data obtained by imaging the target person, and transmits the created moving image data to the cloud server 200. The microphone 30 creates voice data including the target person ID and voice data obtained by recording the contents of communication, and transmits the created voice data to the cloud server 200. The terminal device 40 transmits the target person ID, document data such as materials used for communication, and information indicating the operation log of the terminal device 40 to the cloud server 200.
The cloud server 200 acquires the moving image data transmitted by the imaging device 20, creates a file in which the acquired moving image data is associated with the target person ID, and stores the created file. The cloud server 200 acquires the audio data transmitted by the microphone 30, creates a file in which the acquired audio data is associated with the target person ID, and stores the created file. The cloud server 200 acquires the document data transmitted by the terminal device 40, creates a file in which the acquired document file is associated with the target person ID, and stores the created file. The cloud server 200 acquires information indicating the operation log transmitted by the terminal device 40, creates a file in which the information indicating the acquired operation log is associated with the target person ID, and stores the created file.

A user who evaluates the impression of the target person (hereinafter referred to as “user”) performs an operation for requesting evaluation of the impression to the client terminal 400 when evaluating the impression of the target person. The client terminal 400 creates an impression evaluation request, which is information for requesting impression evaluation, in response to a user operation, and transmits the created impression evaluation request to the impression derivation device 100. The impression evaluation request includes a target person ID of a target person to be evaluated, information indicating a scene to be evaluated and information indicating an impression to be evaluated. Further, when comparing with data of another subject, the impression evaluation request includes information (for example, a file name) indicating the data of the other subject. When comparing with the past data of the subject, the impression evaluation request includes information (for example, a file name) indicating the past data of the subject. The scene will be described later.
When receiving an impression evaluation request transmitted from the client terminal 400, the impression derivation device 100 accesses the cloud server 200 and requests sensor acquisition information such as video data, audio data, document data, and operation logs of the target person. A sensor acquisition information request that is information is transmitted to the cloud server 200. The sensor acquisition information request includes the target person ID.
The cloud server 200 acquires sensor acquisition information such as video data, audio data, document data, and operation logs stored in association with the target ID in accordance with the sensor acquisition information request transmitted by the impression derivation device 100. To do. The cloud server 200 creates a sensor acquisition information response including the acquired sensor acquisition information, and transmits the created sensor acquisition information response to the impression derivation device 100.

The impression derivation device 100 receives the sensor acquisition information response transmitted by the cloud server 200, and receives the moving image data, the audio data, the data indicating the document, and the information indicating the operation log included in the received sensor acquisition information response. get. The impression deriving device 100 recognizes the facial expression, gesture, etc. of the target person based on the acquired moving image data, audio data, data indicating the document, and information indicating the operation log. The impression deriving device 100 derives one or both of one or more parameters indicating emotion and one or more parameters indicating behavior based on the result of recognizing the facial expression, gesture, etc. of the subject. .
The impression deriving device 100 derives an emotion value for each of one or more parameters indicating the derived emotion. The impression deriving device 100 derives an action value for each of one or a plurality of parameters indicating the derived action. Here, the emotion value is obtained by quantifying the degree of each of one or a plurality of parameters indicating the emotion derived based on the result of recognizing the facial expression, gesture, etc. of the subject person. The behavior value is obtained by quantifying the degree of each of one or a plurality of parameters representing the behavior derived based on the result of recognizing the subject's facial expression, gesture, and the like. The impression deriving device 100 stores an emotion value in association with each of one or more parameters indicating emotion. The impression derivation device 100 stores an action value in association with each of one or more parameters indicating the action.
The impression derivation device 100 stores impression derivation information in which information indicating an impression and a derivation formula for deriving an impression evaluation value are associated with each scene. The derivation formula is created by using one or both of a parameter indicating emotion and a parameter indicating action. The derivation formula may be created by using one parameter indicating emotion and one parameter indicating action.

The client terminal 400 transmits to the impression derivation device 100 an impression evaluation request that includes the target person ID, information indicating the scene for evaluating the impression, and information indicating the impression to be evaluated.
The impression derivation device 100 receives the impression evaluation request transmitted by the client terminal 400, and based on the scene that evaluates the impression included in the received impression evaluation request and the information indicating the derived impression, the impression evaluation value is calculated. Get the derivation formula for derivation. The impression deriving device 100 acquires parameters used in the acquired derivation formula from one or both of one or more parameters indicating the stored emotion and one or more parameters indicating the action. Then, the evaluation value of the impression is derived by substituting the acquired emotion value or action value of the parameter into the derivation formula. The impression deriving device 100 analyzes the impression based on the derived evaluation value of the impression. The impression derivation device 100 creates an impression evaluation response including the impression evaluation value and the impression analysis result, and transmits the created impression evaluation response to the client terminal 400.
The client terminal 400 receives the impression evaluation result transmitted by the impression derivation device 100, and displays the impression evaluation value and the impression analysis result included in the received impression evaluation result.

Hereinafter, the impression derivation device 100, the cloud server 200, and the client terminal 400 that constitute the impression derivation system 10 will be described in detail.
(Impression derivation device 100)
FIG. 2 is a block diagram illustrating an example of an impression derivation device, a cloud server, and a client terminal included in the impression derivation system of the embodiment. The impression derivation device 100 will be described with reference to FIG.
The impression deriving device 100 is realized by a device such as a personal computer, a server, a smartphone, a tablet computer, or an industrial computer.
The impression derivation device 100 includes, for example, a communication unit 110, an information processing unit 130, and a storage unit 140.
The communication unit 110 is realized by a communication module. The communication unit 110 communicates with an external communication device via the network 50. The communication unit 110 may perform communication using a communication method such as a wireless local area network (LAN), a wired LAN, Bluetooth (registered trademark), or LTE (Long Term Evolution) (registered trademark).
The communication unit 110 receives the impression evaluation request transmitted from the client terminal 400 and outputs the received impression evaluation request to the information processing unit 130. The communication unit 110 acquires the sensor acquisition information request output from the information processing unit 130 and transmits the acquired sensor acquisition information request to the cloud server 200. The communication unit 110 receives the sensor acquisition information response transmitted from the cloud server 200 and outputs the received sensor acquisition information response to the information processing unit 130. The communication unit 110 acquires the impression evaluation response output by the information processing unit 130 and transmits the acquired impression evaluation response to the client terminal 400.

  The storage unit 140 is realized by, for example, a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), a flash memory, or a hybrid storage device in which a plurality of these are combined. Instead of being provided as a part of the impression derivation device 100, a part or all of the storage unit 140 is replaced by a processor of the impression derivation device 100 such as a NAS (Network Attached Storage) or an external storage server via the network 50. It may be realized by an accessible external device. The storage unit 140 stores a program 141 executed by the information processing unit 130 and an application 142. Further, emotion value action value information 143, impression derivation information 144, and impression information DB 145 are stored in the storage unit 140.

The application 142 causes the impression derivation device 100 to acquire an impression evaluation request, creates a sensor acquisition information request based on the acquired impression evaluation request, and transmits the created sensor acquisition information request to the cloud server 200. . In addition, the application 142 causes the impression derivation device 100 to acquire a sensor acquisition information response that is a response to the transmitted sensor acquisition information request, and includes video data, audio data, and document data included in the acquired sensor acquisition information response. Acquire information indicating the operation log. Further, the application 142 causes the impression derivation device 100 to derive one or both of the emotion value and the action value based on the acquired video data, audio data, document data, and information indicating the operation log. .
In addition, the app 142 stores the impression deriving device 100 in association with the information indicating the impression based on the information indicating the scene and the information indicating the impression included in the impression evaluation request from the impression deriving information 144 of the storage unit 140. Get the derived derivation. Further, the app 142, based on the derivation formula acquired by the impression derivation device 100, the emotion value of one or more parameters indicating the emotion included in the derivation formula and the behavior of one or more parameters indicating the behavior. Either or both of the values are acquired from the emotion value action value information 143 in the storage unit 140, and either or both of the acquired emotion value and action value are substituted into the derivation formula, The evaluation value is derived. In addition, the application 142 causes the impression derivation device 100 to acquire the target person ID and the information indicating the evaluation value of the impression, and to analyze the information indicating the acquired evaluation value of the impression.

(Emotion value action value information 143)
The emotion value action value information 143 is an emotion value for one or both of time and one or more parameters indicating the emotion derived from the sensor acquisition information by the information processing unit 130 and one or more parameters indicating the action. And / or action value is information associated with each subject ID. For example, emotion values are derived mainly from facial expressions. For example, the action value is derived mainly from gestures.
FIG. 3 is a diagram illustrating an example of emotion value action value information. In the example shown in FIG. 3, with respect to the target person ID “aaa”, the time “00:00:00”, the emotion value “7” of joy, which is a parameter indicating the emotion derived from the facial expression, and the emotion value “ 2 ”is associated with a whispering action value“ 0 ”, which is a parameter indicating the action derived from gestures, and a gesture action value“ 12 ”. Returning to FIG. 2, the description will be continued.

(Impression derivation information 144)
The impression derivation information 144 is information in which impression information (impression information), an impression evaluation value, and a derivation formula for deriving an impression evaluation value are associated with each scene. The impression derivation information 144 is prepared for each scene such as grasping the audience's reaction / understanding degree in the case of hiring interview / employee interview, training for strengthening sales ability, and support for face-to-face sales in real time.
Here, in the recruitment interview / employee interview, the interviewee is the target person, and the target person is photographed and the voice is recorded. Based on the moving image data obtained by shooting and the sound data obtained by recording, one or both of the emotion value of the parameter indicating emotion and the behavior value of the parameter indicating behavior are acquired. . Based on one or both of the emotion value of the parameter indicating the acquired emotion and the behavior value of the parameter indicating the behavior, the evaluation value is obtained by digitizing the impression that the speaker is speaking or is appropriate Is derived. Based on the impression that you are speaking digitally and that you have the appropriateness, you can try to find the best match, and find the correlation with the percentage of people who are excellent and the turnover rate.
In training to strengthen sales capabilities, sales representatives are targeted and photographed. Based on the moving image data obtained by shooting, either or both of the emotion value of the parameter indicating emotion and the behavior value of the parameter indicating behavior are acquired. Based on one or both of the acquired emotion value of the parameter indicating the emotion and the behavior value of the parameter indicating the behavior, the evaluation value is derived by digitizing the impression such as the quality of the behavior. You can use the impressions of the digitalized behavior in training for sales representatives.

When supporting face-to-face sales in real time, a customer who is negotiating is targeted, and the subject is photographed and voice is recorded. Based on the moving image data obtained by shooting and the sound data obtained by recording, one or both of the emotion value of the parameter indicating emotion and the behavior value of the parameter indicating behavior are acquired. . Based on one or both of the acquired emotion value of the parameter indicating emotion and the behavior value of the parameter indicating behavior, an evaluation value is derived by digitizing an impression such as a customer reaction. Notify sales representatives of digital customer impressions and other responses, and encourage appropriate responses.
When grasping the audience's reaction and level of understanding, the audience, such as seminars and classes, is targeted, and the audience is photographed and voices are recorded. Based on the moving image data obtained by shooting and the sound data obtained by recording, one or both of the emotion value of the parameter indicating emotion and the behavior value of the parameter indicating behavior are acquired. . Based on one or both of the acquired emotion value of the parameter indicating emotion and the behavior value of the parameter indicating behavior, an evaluation value is derived by digitizing impressions such as emotion and degree of understanding. Encourage speakers to take appropriate actions based on their digital feelings and impressions of understanding. In this case, there are a plurality of subjects.

FIG. 4 is a diagram illustrating an example of impression derivation information. FIG. 4 shows information indicating an impression, an evaluation value of the impression, and a derivation formula for deriving the evaluation value of the impression for each of the scenes 1 to n (n is an integer of n> 0). Associated. In the example shown in FIG. 4, for the scene 1, the information “liking” indicating the impression, the impression evaluation value “A”, and the derivation expression “joy-anger + 10 × whisper + hand gesture” are associated. This expression is obtained by subtracting the emotion value of “anger”, which is a parameter indicating emotion, from the emotion value of “joy”, which is a parameter indicating emotion, and subtracting the evaluation value “A” of likability. It is obtained by subtracting the product of 10 and the action value of “whit” which is a parameter indicating action from the value, and adding the action value of “hand gesture” which is a parameter indicating action from the value obtained as a result of the subtraction. It is shown that. Here, “10” is a weight, and “whisker” is weighted. As a result, it is shown that the influence of “whit” is large in the favorable sensitivity.
For scene 1, the impression information “favorability”, the impression evaluation value “B”, and the derivation formula “happy> 80 & anger × 2 + sad <10” are associated. This expression shows that the evaluation value “B” of the impression has a feeling value of “joyful” that is a parameter indicating emotion greater than 80, and the emotion value of “anger” that is a parameter indicating emotion is doubled. It is shown that the value obtained by adding the emotion value of “sad”, which is a parameter indicating “”, is given by a derivation formula that is less than 10. Here, “2” is weighting, and “anger” is weighted. As a result, it is shown that the influence of “anger” is large in the favorable feeling.
For scene 1, the impression information “favorability”, the impression evaluation value “C”, and the derivation expression “happy> 50 & anger × 2 + sad <30” are associated. This expression indicates that the impression evaluation value “C” has an emotion value of “joyful” that is a parameter indicating emotion greater than 50, and the emotion value of “anger” that is a parameter indicating emotion is doubled. It is shown that the value obtained by adding the emotion value of “sad”, which is a parameter indicating “”, is given by a derivation formula that is less than 30. Here, “2” is weighting, and “anger” is weighted. As a result, it is shown that the influence of “anger” is large in the favorable feeling.
Further, for the scene 1, the information “persuasion” indicating the impression, the evaluation value “D” of the impression, and the derivation expression “100−fear−disgust + 10 × hand gesture” are associated. In this formula, the persuasive evaluation value “D” subtracts the emotion value of “fear”, which is a parameter indicating emotion, from 100, and “hate” is a parameter indicating emotion from the value obtained by subtraction. Is obtained by subtracting the product of 10 and the action value of “hand gesture” which is a parameter indicating action from the subtracted value. Here, “10” is a weight, and “hand gesture” is weighted. This shows that the influence of “hand gesture” is great on persuasive power.

  Further, as an example other than that shown in FIG. 4, the information “Familiarity” indicating the impression includes the emotion value of the parameter “joy” indicating the emotion, the emotion value of “anger”, and the behavior of the parameter “swing” indicating the behavior. It is derived from the value and the action value of “hand gesture”. In addition, the information “confidence” indicating impression is derived from the emotion value of the parameter “sadness” indicating emotion, the emotion value of “fear”, the emotion value of “disgust”, and the behavior value of the parameter “swing” indicating behavior Is done. In addition, the impression information “enthusiasm” and “sympathy” are derived from the emotion value of the parameter “attention” indicating emotion, the behavior value of the parameter “buzz” indicating behavior, and the behavior value of “shake” . In addition, the information “domination” and “responsibility” indicating impression are derived from the emotion value of the parameter “surprise” indicating emotion, the behavior value of the parameter “dominance rate” indicating behavior, and the behavior value of “speed” Is done. Returning to FIG. 2, the description will be continued.

(Impression information DB 145)
The impression information DB 145 includes each emotion value of one or more parameters indicating the subject ID, impression information, and emotion used to derive the evaluation value of the impression, and each of one or more parameters indicating the action. It is the information which linked | related one or both of action value.
The impression information DB 145 stores a file output by an impression information deriving unit 124 described later. The file includes the target person ID, information indicating the impression from which the impression evaluation value is derived, one or more parameters indicating the emotion used to derive the impression evaluation value, their emotion values, The information which linked | related the action value of the 1 or several parameter which shows this, and those action values is contained.
FIG. 5 is a diagram illustrating an example of the impression information DB. In the example shown in FIG. 5, the target person ID “aaa”, the impression information “favorability”, the emotion value “X” of joy that is a parameter indicating the emotion used to derive the evaluation value of the impression, An anger emotion value “Y” which is a parameter indicating emotion, a whispering behavior value “Z” which is a parameter indicating behavior, and an action value “W” of a hand gesture which is a parameter indicating behavior are associated with each other. Information stored in the impression information DB 145 is used by the analysis unit 138 described later. Returning to FIG.

The information processing unit 130 is a functional unit (hereinafter referred to as a software functional unit) realized by a processor such as a CPU (Central Processing Unit) executing the program 141 and the application 142 stored in the storage unit 140, for example. is there. Note that all or part of the information processing unit 130 may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array). It may be realized by a combination of a unit and hardware.
The information processing unit 130 includes, for example, an acquiring unit 132, an emotion value / behavior value deriving unit 133, an impression information deriving unit 134, a creating unit 136, and an analyzing unit 138.

  The acquisition unit 132 acquires the sensor acquisition information response output by the communication unit 110, and outputs the acquired sensor acquisition information response to the emotion value behavior value deriving unit 133. The acquisition unit 132 acquires the impression evaluation request output by the communication unit 110, and outputs the acquired impression evaluation request to the impression information deriving unit 134 and the creation unit 136.

The emotion value behavior value deriving unit 133 acquires the sensor acquisition information response output by the acquisition unit 132, and acquires video data, audio data, document data, and an operation log included in the acquired sensor acquisition information response. The emotion value action value deriving unit 133 is based on the acquired video data, audio data, document data, and operation log, and one or more parameters indicating emotion and one or more parameters indicating action Or both are derived. The emotion value action value deriving unit 133 derives an emotion value for each of the derived parameters indicating one or more emotions. The emotion value action value deriving unit 133 derives an action value for each of the derived parameters indicating one or more actions.
Specifically, the emotion value behavior value deriving unit 133 identifies the face part of the target person from the video obtained by reproducing the video data, tracks the identified part, and performs slight movement of the muscle. The expression is identified by converting it into data. The emotion value / behavior value deriving unit 133 derives one or more parameters indicating emotions such as joy, anger, sadness, surprise, fear, disgust, and contempt by analyzing the identified facial expressions. The emotion value behavior value deriving unit 133 derives an emotion value by quantifying the degree of each of one or a plurality of parameters indicating the derived emotion.

Further, the emotion value behavior value deriving unit 133 recognizes the joint point of the target person from the video obtained by reproducing the video data, and gestures, stands, sleeps, walks, falls, etc. based on the recognized joint point One or a plurality of parameters indicating the behavior of The emotion value action value deriving unit 133 derives an action value by quantifying the degree of each of one or a plurality of parameters indicating the derived action. Further, the emotion value behavior value deriving unit 133 may identify not only a large motion but also a fine motion specifically for the upper body, and derive one or a plurality of parameters indicating the identified motion. Then, the emotion value behavior value deriving unit 133 may derive the behavior value by quantifying the degree of each of one or a plurality of parameters indicating the derived motion.
Here, as an example, the emotion value action value deriving unit 133 selects one or more parameters indicating emotion and one or more parameters indicating action from the video obtained by reproducing the video data. Or the case where both were derived was explained. The emotion value / behavior value deriving unit 133 derives one or both of one or more parameters indicating emotion and one or more parameters indicating behavior from the voice data, document data, and operation log. Then, the emotion value behavior value deriving unit 133 derives an action value by quantifying the degree of each of one or a plurality of parameters indicating the derived motion.
For example, the emotion value / behavior value deriving unit 133 derives one or a plurality of parameters indicating emotions such as joy, anger, sadness, excitement, and normality from the sound obtained by reproducing the sound data. Further, for example, the explanation procedure and the explanation reference time are stored in the storage unit 140. Whether the emotion value action value deriving unit 133 follows the explanation order and the explanation reference time by comparing the information indicating the document data and the operation log with the explanation order stored in the storage unit 140 and the explanation reference time. Determine whether or not. The emotion value behavior value deriving unit 133 derives one or a plurality of parameters indicating the behavior such as the explanation order and the explanation time based on the determination result of whether or not the explanation order and the explanation reference time are followed.
The emotion value / behavior value deriving unit 133 is one or both of a combination of one or more parameters indicating emotion and an emotion value, and a combination of each of one or more parameters indicating action and an action value. Is stored in the emotion value action value information 143 of the storage unit 140 in association with the subject ID.

The impression information deriving unit 134 acquires the impression evaluation request output by the acquiring unit 132, and acquires impression derivation information corresponding to information indicating the scene included in the acquired impression evaluation request from the impression derivation information 144 of the storage unit 140. To do. Furthermore, the impression information deriving unit 134 obtains a derivation formula associated with the information indicating the impression information from the acquired impression derivation information based on the information indicating the impression included in the impression evaluation request.
The impression information deriving unit 134, based on the acquired derivation formula, each emotion value of one or more parameters indicating emotions included in the derivation formula, and each action value of one or more parameters indicating behavior Either or both of the emotion values of one or more parameters indicating the emotion associated with the subject ID of the emotion value behavior value information 143 in the storage unit 140 and one or more parameters indicating the behavior Obtained from each action value.
The impression information deriving unit 134 substitutes one or both of the emotion value of each of the acquired one or more parameters and the action value of each of the acquired one or more parameters for the corresponding parameter of the derivation formula. Thus, the evaluation value of the impression is derived. With this configuration, an impression evaluation value can be derived in real time.

Here, the impression information deriving unit 134 determines the emotion value with respect to time for each of one or more parameters indicating the emotion associated with the subject ID of the emotion value action value information 143 in the storage unit 140. The emotion value of the parameter indicating each emotion may be derived by performing statistical processing such as averaging and summing. In addition, the impression information deriving unit 134 averages the behavior value with respect to time for each of one or a plurality of parameters indicating the behavior associated with the target person ID of the emotion value behavior value information 143 in the storage unit 140. The behavior value of the parameter indicating each behavior may be derived by performing statistical processing such as summing and summing.
The impression information deriving unit 134 may derive the evaluation value of the impression by substituting one or both of the derived emotion value and the derived action value into the corresponding parameter of the derivation formula.
The impression information deriving unit 134 outputs the target person ID and information indicating the derived evaluation value of the impression to the analyzing unit 138. Furthermore, the impression information deriving unit 134 includes the target person ID, the information indicating the impression from which the evaluation value of the impression is derived, one or a plurality of parameters indicating the emotion used to derive the evaluation value of the impression, and those parameters. A file in which the emotion value, the action value of one or a plurality of parameters indicating the action, and the action value are associated is created, and the created file is stored in the impression information DB 145.

The analysis unit 138 acquires the target person ID output by the impression information deriving unit 134 and information indicating the evaluation value of the impression, and analyzes the information indicating the acquired evaluation value of the impression. Specifically, the analysis unit 138 creates time-series data of impression evaluation values. Further, the analysis unit 138 acquires the evaluation value of the past impression of the target person from the impression information DB 145, and based on the acquired evaluation value of the past impression, the evaluation value and the past impression of the same target person. Create a comparison result with the evaluation value.
Further, the analysis unit 138 acquires the evaluation value of the impression of the other person from the impression information DB 145, and based on the acquired evaluation value of the impression of the other person, the evaluation value of the impression of the target person and the evaluation value of the impression of the other person Create comparison results. Further, the analysis unit 138 determines either or both of a parameter indicating an emotion and a parameter indicating an action that affect the evaluation value of the impression based on a comparison result between the evaluation value of the impression and the evaluation value of another person's impression. Extract.
For example, an exemplary insurance recruiter and a general recruiter are photographed, and an evaluation value of an impression is derived based on video data obtained by photographing. The analysis unit 138 compares the evaluation result of the impression of the exemplary insurance recruiter with the evaluation result of the impression of the general recruiter, and verifies whether or not a parameter with a significant difference can be extracted. As a result of the verification, when a parameter with a significant difference can be extracted, it can be determined that the extracted parameter affects the impression of the exemplary insurance recruiter and the general recruiter. Furthermore, the analysis unit 138 may create a derivation formula that includes the extracted parameters. By configuring in this way, it is possible to create a derivation formula for deriving an impression evaluation value focusing on parameters that affect the impression of an exemplary insurance recruiter and a general recruiter.
The analysis unit 138 outputs information indicating the target person ID and impression evaluation value and information indicating the analysis result of the impression evaluation value to the creation unit 136.
The creation unit 136 acquires the impression evaluation request output by the acquisition unit 132, creates a sensor acquisition information request including the target person ID based on the acquired impression evaluation request, and transmits the created sensor acquisition information request to the communication unit 110. Output to. The creation unit 136 creates an impression evaluation response including the target person ID output by the analysis unit 138, information indicating the impression evaluation value, and information indicating the analysis result of the impression evaluation value, and the created impression evaluation response is generated. And output to the communication unit 110.

(Cloud server 200)
The cloud server 200 will be described with reference to FIG.
The cloud server 200 is realized by a device such as a personal computer, a server, or an industrial computer.
The cloud server 200 includes, for example, a communication unit 210, an information processing unit 230, and a storage unit 240.

The communication unit 210 is realized by a communication module. The communication unit 210 communicates with an external communication device via the network 50. The communication unit 210 may communicate using a communication method such as a wireless LAN, a wired LAN, Bluetooth (registered trademark), or LTE (registered trademark).
The communication unit 210 receives the video data transmitted by the imaging device 20 and outputs the received video data to the information processing unit 230. The communication unit 210 receives the audio data transmitted by the microphone 30 and outputs the received audio data to the information processing unit 230. The communication unit 210 receives the document data transmitted by the terminal device 40 and information indicating the operation log, and outputs the received document data and information indicating the operation log to the information processing unit 230. The communication unit 210 receives the sensor acquisition information request transmitted by the impression derivation device 100, and outputs the received sensor acquisition information request to the information processing unit 230. The communication unit 210 transmits the sensor acquisition information response output from the information processing unit 230 to the impression derivation device 100.

The storage unit 240 is realized by, for example, a RAM, a ROM, an HDD, a flash memory, or a hybrid storage device in which a plurality of these are combined. The storage unit 240 stores a program 242 executed by the information processing unit 230 and an application 244. Furthermore, detection information 246 is stored in the storage unit 240.
The application 244 causes the cloud server 200 to acquire video data, audio data, document data, and information indicating an operation log, and displays the acquired video data, audio data, document data, and an operation log. The information is stored in the storage unit 240 in association with the subject ID. Further, the application 244 causes the cloud server 200 to acquire a sensor acquisition information request, acquires sensor acquisition information stored in the storage unit 240 based on the acquired sensor acquisition information request, and acquires the acquired sensor. Information is transmitted to the impression derivation device 100.
The detection information 246 stores information output by the information processing unit 230 among the video data, audio data, document data, and operation log for each target person ID.

The information processing unit 230 is a software function unit that is realized, for example, when a processor such as a CPU executes a program 242 stored in the storage unit 240. Note that all or part of the information processing unit 230 may be realized by hardware such as LSI, ASIC, or FPGA, or may be realized by a combination of a software function unit and hardware.
The information processing unit 230 includes, for example, an acquisition unit 232 and a creation unit 236.

The acquisition unit 232 acquires the video data output by the communication unit 210, creates a file in which the acquired video data is associated with the target person ID, and stores the created file in the detection information 246 of the storage unit 240. The acquisition unit 232 acquires the audio data output from the communication unit 210, creates a file in which the acquired audio data is associated with the target person ID, and stores the created file in the detection information 246 of the storage unit 240. The acquisition unit 232 acquires the document data output by the communication unit 210, creates a file in which the acquired document data is associated with the target person ID, and stores the created file in the detection information 246 of the storage unit 240. The acquisition unit 232 acquires information indicating the operation log output from the communication unit 210, creates a file in which the information indicating the acquired operation log is associated with the target person ID, and detects the created file in the storage unit 240 Store in information 246.
The acquisition unit 232 acquires the sensor acquisition information request output from the communication unit 210, and the video data file, the audio data file, and the document data associated with the target person ID included in the acquired sensor acquisition information request. And the operation log file are acquired from the detection information 246 in the storage unit 240. The acquisition unit 232 outputs the target person ID, the acquired video data file, audio data file, document data file, and operation log file to the creation unit 236.
The creation unit 236 acquires the target person ID, the video data file, the audio data file, the document data file, and the operation log file output from the acquisition unit 232, and the acquired target ID and video data file A sensor acquisition information response including the audio data file, the document data file, and the operation log file is generated with the impression derivation device 100 as the destination. The creation unit 236 outputs the created sensor acquisition information response to the communication unit 210.

(Client terminal 400)
The client terminal 400 will be described with reference to FIG.
The client terminal 400 is realized by a device such as a personal computer, a smartphone, or a tablet computer.
The client terminal 400 includes, for example, a communication unit 410, an information processing unit 430, a storage unit 440, an operation unit 450, and a display unit 460.

The communication unit 410 is realized by a communication module. The communication unit 410 communicates with an external communication device via the network 50. The communication unit 410 may communicate with a communication method such as a wireless LAN, a wired LAN, Bluetooth (registered trademark), or LTE (registered trademark).
The communication unit 410 transmits the impression evaluation request output from the information processing unit 430 to the impression derivation device 100. The communication unit 410 receives the impression evaluation response transmitted by the impression derivation device 100, and outputs the received impression evaluation response to the information processing unit 430.

The storage unit 440 is realized by, for example, a RAM, a ROM, an HDD, a flash memory, or a hybrid storage device in which a plurality of these are combined. The storage unit 440 stores a program 442 executed by the information processing unit 430 and an application 444.
The application 444 causes the client terminal 400 to create an impression evaluation request and transmits the created impression evaluation request to the impression derivation device 100. The application 444 receives the impression evaluation response transmitted by the impression derivation device 100 and processes the received impression evaluation response.
The operation unit 450 includes an input device and accepts user operations. The input device includes a device for inputting character information such as a keyboard, a pointing device such as a mouse and a touch panel, a button, a dial, a joystick, a touch sensor, and a touch pad.
The display unit 460 is controlled by a processor such as a CPU, and displays an image, a GUI (Graphical User Interface), and the like. In this example, the operation unit 450 is a touch panel.

For example, the information processing unit 430 is a software function unit realized by a processor such as a CPU executing a program 442 and an application 444 stored in the storage unit 440. Note that all or part of the information processing unit 430 may be realized by hardware such as LSI, ASIC, or FPGA, or may be realized by a combination of a software function unit and hardware.
The information processing unit 430 includes, for example, an acquisition unit 432, a processing unit 434, and a creation unit 436.

The acquisition unit 432 acquires the impression evaluation response output by the communication unit 410 and outputs the acquired impression evaluation response to the processing unit 434.
When the user performs an operation for requesting impression evaluation from the operation unit 450, the processing unit 434 displays a screen for performing impression evaluation on the display unit 460.
FIG. 6 is a diagram illustrating an example of a screen for evaluating an impression. In the example shown in FIG. 6, an impression evaluation screen includes “1. Analysis target data specification” column, “2. Evaluation pattern specification” column, “3. Communication analysis target specification” column, “ 4. “Comprehensive evaluation confirmation” column is displayed.
In the “1. Analysis target data designation” column, information (for example, a file name) indicating moving image data of a target person whose impression is to be evaluated, information (for example, a file name) indicating audio data, and information (for example, a file name) For example, at least one piece of information is input between the file name) and information indicating the operation log (for example, file name).
In the “2. Evaluation pattern designation” column, information indicating a scene for evaluating an impression is input as an evaluation pattern, and information indicating an impression is input as an evaluation item.
In the “3. Communication analysis target designation” field, information (for example, a file name) indicating past data of the target person is input when compared with the past impression of the target person. In addition, in the “3. Communication analysis target designation” field, information (for example, a file name) indicating data of other target persons is input when compared with impressions of other target persons.
In the “4. Comprehensive evaluation confirmation” column, a comprehensive evaluation result including a parameter indicating emotion, a detail of a parameter indicating action, and an evaluation value of impression is displayed.

After inputting necessary items on the screen for evaluating impressions, when “analysis start” is pressed, the processing unit 434 causes the creation unit 436 to send information indicating the target person ID and scene, information indicating the impression, Is output. Furthermore, when information indicating the target person's past data or information indicating data of another target person is input in the “3. Communication analysis target designation” field, the processing unit 434 displays the target person ID and In addition to the information indicating the impression, information indicating past data of the target person and information indicating data of other target persons are output. Here, the case where the processing unit 434 outputs the target person ID, information indicating a scene, information indicating an impression, and information indicating data of another target person to the creation unit 436 will be described.
The processing unit 434 acquires the impression evaluation response output by the acquisition unit 432 and displays the impression evaluation result on the display unit 460 by processing the acquired impression evaluation response.

FIG. 7 is a diagram illustrating an example of an impression evaluation result display screen. In the example shown in FIG. 7, the impression evaluation result display screen is divided into an “A” field, a “B” field, and a “C” field. In the “A” column, information indicating an impression, an evaluation value of the impression, and a rank of the evaluation value of the impression are displayed in association with each other. Here, the rank of the evaluation value is obtained by classifying the evaluation value of the impression as A, B, C,... From the higher evaluation value. For example, the evaluation value of impression is 80 or more as A rank, 70 or more and less than 80 as B rank, 60 or more and less than 70 as C rank, and 40 or more and less than 60 as D rank.
In the “B” column, a comparison result of evaluation values of impressions of the subject and other subjects is displayed. In the example shown in FIG. 8, the comparison result of the evaluation values of the impressions of the subject and other subjects is represented by a table and a radar chart.
In the “C” column, the time series data and the other values for one or both of the emotion value of each of one or more parameters indicating emotion and the action value of each of one or more parameters indicating behavior The comparison with the subject and the evaluation value of the impression are shown. In the example shown in FIG. 8, the time series data is represented by a stacked graph. Comparison with other subjects is represented by a radar chart. The evaluation value of the impression is represented by a table.
The creation unit 436 acquires the target person ID, the information indicating the scene, the information indicating the impression, the information indicating the data of the other target person, and the information indicating the data of the other target person, and the information indicating the acquired target person ID and the scene. An impression evaluation request including information indicating an impression and information indicating data of another target person and having the impression derivation device 100 as a destination is created. The creation unit 436 outputs the created impression evaluation request to the communication unit 410.

(Operation of impression derivation system 10)
FIG. 8 is a sequence chart illustrating an example of the operation of the impression derivation system according to the embodiment. In the example shown in FIG. 8, the training target person is based on video data, audio data, document data, and operation logs acquired during a face-to-face communication with a third party. A case will be described in which the evaluation value of the impression of the target person is derived and the evaluation value of the derived impression is compared with the evaluation value of the impression of another person.
(Step S11)
The imaging device 20 images a target person who is communicating face-to-face with a third party.
(Step S12)
The imaging device 20 creates video data including the target person ID and video data obtained by capturing the target person.
(Step S13)
The imaging device 20 transmits the created video data to the cloud server 200.
(Step S14)
The microphone 30 records the content of the subject communicating face-to-face with a third party.
(Step S15)
The microphone 30 creates voice data including the target person ID and voice data obtained by recording the contents of communication.

(Step S16)
The microphone 30 transmits the created audio data to the cloud server 200.
(Step S17)
The terminal device 40 acquires document data and an operation log of the terminal device 40 that are used when the target person performs face-to-face communication with a third party.
(Step S18)
The terminal device 40 creates information indicating the target person ID, document data, and operation log, document data, and an operation log.
(Step S19)
The terminal device 40 transmits the created document data and operation log to the cloud server 200.
(Step S20)
The communication unit 210 of the cloud server 200 receives the video data transmitted by the imaging device 20 and outputs the received video data to the information processing unit 230. The communication unit 210 receives the audio data transmitted by the microphone 30 and outputs the received audio data to the information processing unit 230. The communication unit 210 receives the document data transmitted by the terminal device 40 and information indicating the operation log, and outputs the received document data and information indicating the operation log to the information processing unit 230.
The acquisition unit 232 of the information processing unit 230 acquires the video data output from the communication unit 210, creates a file in which the acquired video data is associated with the target person ID, and uses the created file as detection information in the storage unit 240. Store to H.246. The acquisition unit 232 acquires the audio data output from the communication unit 210, creates a file in which the acquired audio data is associated with the target person ID, and stores the created file in the detection information 246 of the storage unit 240. The acquisition unit 232 acquires the document data output by the communication unit 210, creates a file in which the acquired document data is associated with the target person ID, and stores the created file in the detection information 246 of the storage unit 240. The acquisition unit 232 acquires information indicating the operation log output from the communication unit 210, creates a file in which the information indicating the acquired operation log is associated with the target person ID, and detects the created file in the storage unit 240 Store in information 246.

(Step S21)
The user performs an operation for requesting impression evaluation to the client terminal 400. The processing unit 434 of the client terminal 400 outputs a target person ID, information indicating a scene, information indicating an impression, and information indicating data of another target person to the creation unit 436. The creation unit 436 acquires the target person ID, the information indicating the scene, the information indicating the impression, the information indicating the data of the other target person, and the information indicating the data of the other target person, and the information indicating the acquired target person ID and the scene. An impression evaluation request including information indicating an impression and information indicating data of another target person and having the impression derivation device 100 as a destination is created. The creation unit 436 outputs the created impression evaluation request to the communication unit 410. The communication unit 410 acquires the impression evaluation request output by the creation unit 436 and transmits the acquired impression evaluation request to the impression derivation device 100.
(Step S22)
The communication unit 110 of the impression derivation device 100 receives the impression evaluation request transmitted by the client terminal 400 and outputs the received impression evaluation request to the information processing unit 130. The acquisition unit 132 acquires the impression evaluation request output by the communication unit 110 and outputs the acquired impression evaluation request to the creation unit 136. The creation unit 136 acquires the impression evaluation request output by the acquisition unit 132, creates a sensor acquisition information request including the target person ID based on the acquired impression evaluation request, and transmits the created sensor acquisition information request to the communication unit 110. Output to. The communication unit 110 acquires the sensor acquisition request output by the creation unit 136 and transmits the acquired sensor acquisition information request to the cloud server 200.

(Step S23)
The communication unit 210 of the cloud server 200 receives the sensor acquisition information request transmitted by the impression derivation device 100 and outputs the received sensor acquisition information request to the information processing unit 230. The acquisition unit 232 acquires the sensor acquisition information request output from the communication unit 210, and the video data file, the audio data file, and the document data file associated with the target person ID included in the acquired sensor acquisition information request And the operation log file are acquired from the detection information 246 in the storage unit 240. The acquisition unit 232 outputs the target person ID, the acquired video data file, audio data file, document data file, and operation log file to the creation unit 236.
The creation unit 236 acquires the target person ID output by the acquisition unit 232, the acquired video data file, audio data file, document data file, and operation log file, and acquires the acquired target person ID and A sensor acquisition information response including the video data file, the audio data file, the document data file, and the operation log file is created and the destination is the impression derivation device 100. The creation unit 236 outputs the created sensor acquisition information response to the communication unit 210. The communication unit 210 acquires the sensor acquisition information response output by the creation unit 236, and transmits the acquired sensor acquisition information response to the impression derivation device 100.
(Step S24)
The communication unit 110 of the impression derivation device 100 receives the sensor acquisition information response transmitted by the cloud server 200 and outputs the received sensor acquisition information response to the information processing unit 130. The acquisition unit 132 acquires the sensor acquisition information response output by the communication unit 110, and outputs the acquired sensor acquisition information response to the emotion value behavior value deriving unit 133.

(Step S25)
The emotion value behavior value deriving unit 133 of the impression deriving device 100 acquires the sensor acquisition information response output by the acquisition unit 132, and the video data, audio data, document data, and operation log included in the acquired sensor acquisition information response. get. The emotion value action value deriving unit 133 is based on the acquired video data, audio data, document data, and operation log, and one or more parameters indicating emotion and one or more parameters indicating action Or both are derived. The emotion value action value deriving unit 133 derives an emotion value for each of the derived parameters indicating one or more emotions. The emotion value action value deriving unit 133 derives an action value for each of the derived parameters indicating one or more actions. The emotion value action value deriving unit 133 is one of a combination of one or a plurality of parameters indicating the derived emotion and an emotion value, and a combination of each of one or more parameters indicating the action and an action value. One or both are associated with the subject ID and stored in the emotion value / action value information 143 of the storage unit 140.
(Step S26)
The impression information deriving unit 134 acquires the impression evaluation request output by the acquiring unit 132, and acquires impression derivation information corresponding to information indicating the scene included in the acquired impression evaluation request from the impression derivation information 144 of the storage unit 140. To do. Further, the impression information deriving unit 134 acquires a derivation formula stored in association with information indicating impression information from the acquired impression derivation information based on information indicating the impression included in the impression evaluation request.
The impression information deriving unit 134, based on the acquired derivation formula, each emotion value of one or more parameters indicating emotions included in the derivation formula, and each action value of one or more parameters indicating behavior Either or both of the emotion values of one or more parameters indicating the emotion associated with the subject ID of the emotion value behavior value information 143 in the storage unit 140 and one or more parameters indicating the behavior Obtained from each action value. The impression information deriving unit 134 derives an impression evaluation value by substituting one or both of the emotion value and the action value into the corresponding parameter of the derivation formula. The impression information deriving unit 134 outputs the target person ID and information indicating the derived evaluation value of the impression to the analyzing unit 138.

The analysis unit 138 acquires the target person ID output by the impression information deriving unit 134 and information indicating the impression evaluation value, and analyzes the acquired information indicating the evaluation value of the impression. The analysis unit 138 outputs information indicating the target person ID and impression evaluation value and information indicating the analysis result of the impression evaluation value to the creation unit 136. The creation unit 136 creates an impression evaluation response including the target person ID output by the analysis unit 138, information indicating the evaluation value of the impression, and information indicating the analysis result of the impression evaluation value, and the created impression evaluation response is communicated. Output to the unit 110.
(Step S27)
The communication unit 110 transmits the impression evaluation response output from the information processing unit 130 to the client terminal 400.
Thereafter, the communication unit 410 of the client terminal 400 receives the impression evaluation response transmitted by the impression derivation device 100, and outputs the received impression evaluation response to the information processing unit 430. The processing unit 434 of the information processing unit 430 acquires the impression evaluation response output by the communication unit 410 and displays the impression evaluation result on the display unit 460 by processing the acquired impression evaluation response.
In the sequence chart shown in FIG. 9, step S11 to step S13, step S14 to step S16, and step S17 to step S19 may be switched in order or performed in parallel.

In the above-described embodiment, the case where the facility 60 such as a training facility is provided as an example of the installation location of the imaging device 20, the microphone 30, and the terminal device 40 is not limited to this example. For example, the imaging device 20, the microphone 30, and the terminal device 40 are installed at a store counter, and video data, audio data, document data, and information indicating an operation log of the person in charge at the counter are sent to the impression derivation device 100. May be sent. Then, based on the impression evaluation result derived by the impression deriving device 100, an appropriate response may be urged to the person in charge at the window.
In the above-described embodiment, during the communication, the imaging device 20 images the subject, the microphone 30 records the content of the communication, and the terminal device 40 records the document data such as materials used for the communication or the communication. Although the case where the operation log in the middle is acquired has been described, the present invention is not limited to this example. For example, during communication, an image ahead of the subject's line of sight may be acquired, or an image ahead of the third person's line of sight may be acquired. In this case, the evaluation value of the impression of the subject toward the third party may be derived based on one or both of the image ahead of the subject's line of sight and the image ahead of the eye of the third party.
In the embodiment described above, the case where the impression derivation device 100 stores the impression derivation information 144 in which the information indicating the impression and the derivation formula for deriving the evaluation value of the impression are associated with each scene is described. This is not the case. For example, the impression derivation device 100 prepares information indicating an impression and a derivation formula for deriving an impression evaluation value for each scene, for each attribute such as the age and sex of a third party. The impression derivation information 144 may be stored. Here, the information in which the impression is associated with the derivation formula for deriving the evaluation value of the impression for each scene may differ depending on the attribute of the third party. By configuring in this way, it is possible to derive an evaluation value of an impression corresponding to a third party attribute.

In the embodiment described above, the cloud server 200 creates and creates a sensor acquisition information response including the video data, audio data, document data, and information indicating the operation log of the target person stored in association with the target person ID. The obtained sensor acquisition information response is transmitted to the impression deriving device 100. The impression deriving device 100 has been described with respect to the case of acquiring the subject's moving image data, audio data, data indicating the document, and information indicating the operation log included in the received sensor acquisition information response. I can't. For example, the cloud server 200 creates and creates a sensor acquisition information response including at least one of video data, audio data, document data, and information indicating an operation log of the target person stored in association with the target person ID. The acquired sensor information response may be transmitted to the impression derivation device 100. In this case, the impression derivation device 100 acquires at least one of the moving image data, audio data, data indicating the document, and information indicating the operation log included in the received sensor acquisition information response.
In the embodiment described above, a case has been described in which the client terminal 400 transmits an impression evaluation request to the impression derivation device 100 and receives an impression evaluation response transmitted by the impression derivation device 100 in response to the transmitted impression evaluation request. However, the present invention is not limited to this example. For example, the impression derivation device 100 may include an operation unit and a display unit, and the impression evaluation result may be displayed on the display unit by operating the operation unit.
In the above-described embodiment, video data obtained by imaging by the imaging device 20, audio data obtained by recording by the microphone 30, document data acquired by the terminal device 40, and information indicating an operation log However, this is not the case. For example, an impression of video data obtained by imaging by the imaging device 20, audio data obtained by recording by the microphone 30, document data acquired by the terminal device 40, and information indicating an operation log is given. You may make it memorize | store in the derivation | leading-out apparatus 100. FIG.

According to the impression deriving system of the embodiment, the impression deriving device indicates the emotion of the subject derived based on information acquired by the sensors (in the embodiment, the imaging device 20, the microphone 30, and the terminal device 40). An acquisition unit that acquires one or both of an emotion value that is a value obtained by quantifying a parameter and an action value that is a value obtained by quantifying a parameter indicating an action (in the embodiment, the acquisition unit 132 and an emotion value action value) The deriving unit 133) is associated with information indicating an impression and information for deriving an evaluation value of an impression created by using one or both of a parameter indicating emotion and a parameter indicating action One or both of a storage unit (impression derivation information 144 of the storage unit 140 in the embodiment) that stores impression derivation information that is information, and an emotion value and an action value acquired by the acquisition unit , Based on the impression stored in the storage unit derived information (in the embodiment, the impression information deriving unit 134) deriving unit that derives the evaluation value of the impression which gives the subject to third parties and a.
By configuring in this way, the impression derivation device 100 is one of one or more parameters indicating emotion and one or more parameters indicating action based on communication elements such as facial expressions, gestures, and voices. One or both can be derived. The impression deriving device 100 can derive an emotion value for each of one or more parameters indicating the derived emotion, and can derive an action value for each of the one or more parameters indicating the derived action. Since the impression deriving device 100 can derive the evaluation value of the impression given to the third party of the target person based on the impression deriving information and one or both of the emotion value and the action value, it is not determined by human subjectivity. No quantitative and objective index values (impression evaluation values) can be derived.

Further, the impression deriving device 100 stores the impression deriving information for each of the scenes for deriving the evaluation value of the impression given to the third person of the subject in the storage unit 140, and the deriving unit (in the embodiment, the impression The information deriving unit 134) derives the evaluation value of the impression given to the third party of the target person based on the impression derivation information corresponding to the scene from which the evaluation value of the impression is derived among the plurality of scenes. By configuring in this way, it is possible to derive an evaluation value of the impression given to the third person of the subject for each scene for which the impression is evaluated.
In addition, one or both of a parameter indicating emotion and a parameter indicating action included in the information for deriving the evaluation value of the impression included in the impression derivation information vary depending on the attribute of the third party. By configuring in this way, it is possible to derive an evaluation value of an impression given to the third party of the subject person according to the attribute of the third party.
Also, an analysis unit that creates time-series data of the evaluation value of the impression derived by the deriving unit (in the embodiment, the impression information deriving unit 134) or a comparison result between the evaluation value of the impression and the evaluation value of another person's impression (In the embodiment, an analysis unit 138) is provided. By configuring in this way, the impression derivation device 100 can create time-series data of the evaluation value of the subject's impression or a comparison result between the evaluation value of the subject's impression and the evaluation value of the impression of the other person. it can.
Further, the analysis unit (in the embodiment, the analysis unit 138) extracts one or both of the parameter indicating the emotion and the parameter indicating the action that affect the evaluation value of the impression based on the comparison result. . By configuring in this way, the impression deriving device 100 can extract one or both of an emotion value and an action value that affect the evaluation value of the impression. For this reason, the impression derivation device 100 newly uses a derivation formula for deriving an impression evaluation value by using either one or both of an emotion value and an action value that affect the extracted impression evaluation value. Can be defined.

As mentioned above, although embodiment was described, these embodiment was shown as an example and is not intending limiting the range of invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the scope of the invention. These embodiments are included in the scope and gist of the invention, and at the same time, are included in the invention described in the claims and the equivalents thereof.
The impression derivation device 100, the cloud server 200, and the client terminal 400 described above may be realized by a computer. In that case, a program for realizing the function of each functional block is recorded on a computer-readable recording medium. The program recorded on the recording medium may be read by a computer system and executed by the CPU. The “computer system” here includes hardware such as an OS (Operating System) and peripheral devices.
The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM. The “computer-readable recording medium” includes a storage device such as a hard disk built in the computer system.

Furthermore, the “computer-readable recording medium” may include a medium that dynamically holds a program for a short time. What holds the program dynamically for a short time is, for example, a communication line when the program is transmitted via a network such as the Internet or a communication line such as a telephone line.
In addition, the “computer-readable recording medium” may include a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client. The program may be for realizing a part of the functions described above. Further, the program may be a program that can realize the above-described functions in combination with a program already recorded in the computer system. The program may be realized using a programmable logic device. The programmable logic device is, for example, an FPGA (Field Programmable Gate Array).

The impression derivation device 100, the cloud server 200, and the client terminal 400 described above have a computer inside. Each process of the impression derivation device 100, the cloud server 200, and the client terminal 400 described above is stored in a computer-readable recording medium in the form of a program, and this program is read and executed by the computer. The above processing is performed.
Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 10 ... Impression derivation system, 20 ... Imaging device, 30 ... Microphone, 40 ... Terminal device, 50 ... Network, 60 ... Facility, 100 ... Impression derivation device, 110 ... Communication part, 130 ... Information processing part, 132 ... Acquisition part, 133 ... emotion value action value deriving unit, 134 ... impression information deriving unit, 136 ... creating unit, 138 ... analyzing unit, 140 ... storage unit, 141 ... program, 142 ... app, 143 ... emotion value action value information, 144 ... impression Derived information, 145 ... Impression information DB, 200 ... Cloud server, 210 ... Communication unit, 230 ... Information processing unit, 232 ... Acquisition unit, 236 ... Creation unit, 240 ... Storage unit, 242 ... Program, 244 ... Application, 246 ... Detection information, 400 ... client terminal, 410 ... communication unit, 430 ... information processing unit, 432 ... acquisition unit, 434 ... processing unit, 436 ... creation unit, 24 ... storage unit, 442 ... program, 444 ... application, 450 ... operation unit, 460 ... display unit

Claims (9)

Acquire one or both of an emotion value, which is a value obtained by quantifying a parameter indicating the emotion of the subject derived based on information acquired by the sensor, and an action value, which is a value obtained by quantifying the parameter indicating the behavior. An acquisition unit to
Impression derivation information that is information that associates information indicating an impression with information for deriving an evaluation value of an impression created by using one or both of a parameter indicating emotion and a parameter indicating action A storage unit for storing
Evaluation of impression given to a third party of the target person based on one or both of the emotion value and the action value acquired by the acquisition unit and the impression derivation information stored in the storage unit An impression derivation system comprising: a derivation unit for deriving a value. The storage unit stores impression derivation information for each of the scenes for deriving an evaluation value of an impression given to a third party of the target person,
The derivation unit derives an evaluation value of an impression given to the third party of the target person based on impression derivation information corresponding to a scene from which an evaluation value of an impression is derived among the plurality of scenes. The impression derivation system according to 1.   The parameter for indicating emotion and / or the parameter for indicating behavior included in the information for deriving the evaluation value of the impression included in the impression derivation information are different depending on the attribute of the third party. Or the impression derivation system according to claim 2. The analysis unit for creating time-series data of the evaluation value of the impression derived by the deriving unit or a comparison result between the evaluation value of the impression and the evaluation value of another person's impression is provided. The impression derivation system according to one item.   5. The impression derivation system according to claim 4, wherein the analysis unit extracts one or both of a parameter indicating the emotion and a parameter indicating the action that affect the evaluation value of the impression based on the comparison result. . The storage unit stores an explanation procedure and an explanation reference time,
The acquisition unit acquires a behavior value of the target derived based on information acquired by a terminal device used by the target, the description procedure stored in the storage unit, and the description reference time. The impression derivation system according to any one of claims 1 to 5.   The impression derivation system according to any one of claims 1 to 6, wherein the information acquired by the sensor includes one or both of information on the subject and information on the third party. . Acquire one or both of an emotion value, which is a value obtained by quantifying a parameter indicating the emotion of the subject derived based on information acquired by the sensor, and an action value, which is a value obtained by quantifying the parameter indicating the behavior. And steps to
The information obtained by associating information indicating an impression with information for deriving an evaluation value of an impression created using one or both of a parameter indicating emotion and a parameter indicating behavior Deriving an evaluation value of an impression given to a third party of the target person based on either one or both of the emotion value and the action value acquired in the step, and executed by an impression derivation system Impression derivation method. On the computer,
Acquire one or both of an emotion value, which is a value obtained by quantifying a parameter indicating the emotion of the subject derived based on information acquired by the sensor, and an action value, which is a value obtained by quantifying the parameter indicating the behavior. And steps to
The information obtained by associating information indicating an impression with information for deriving an evaluation value of an impression created using one or both of a parameter indicating emotion and a parameter indicating behavior A program that executes a step of deriving an evaluation value of an impression given to a third party of the target person based on one or both of the emotion value and the action value acquired in the step.

Images