JP6933076B2 - Control devices, control methods, programs and control systems - Google Patents

Description

The present invention relates to control devices, control methods, programs and control systems.

From the results of the evaluation experiment conducted by the inventor of the present application, it is known that at the beginning of a conversation, the more positive the emotional state of the person with whom the person wants to talk is, the easier it is to talk. In addition, in the survey conducted by the inventor of the present application regarding the communication support system, experience comments such as "when consulting with a boss about important matters, talk in a good mood" have been obtained. In this way, being able to speak and start a conversation when the other person's emotional state is positive is important for realizing smooth communication. The same can be said for remote communication using a telephone or the like in a distributed environment.

Patent Document 1 proposes an information communication system capable of acquiring information including or indicating an emotional state of another user before the start of communication (hereinafter, emotional information) via a communication terminal.

Japanese Unexamined Patent Publication No. 2003-110703

However, in the information communication system of Patent Document 1, a user can "know" the emotions of another user from the acquired information, but "transform" the emotions before talking to (calling) the other user. It is not possible. For example, even if it is possible to know that another user has a negative emotion by using the information communication system of Patent Document 1, it cannot be transformed, so that the other user knows that the other user has a negative emotion. There was the problem of having to put up with it and talk to it, or wait for the other person to naturally turn into positive emotions without knowing when.

Further, in Non-Patent Document 1, as a result of having the experimental participants whose emotions are positively or negatively induced to have a conversation and evaluating the conversation satisfaction and the interpersonal impression, the condition that the emotions are similar to each other (small difference). ) Was smaller in the evaluation values of interpersonal impression and conversation satisfaction than the dissimilar conditions (large difference). In the evaluation as a place of conversation, it is desirable that the emotional state of the speakers and the difference between the evaluations are small (one is highly evaluated and the other is low, which is not desirable as a whole.) Therefore, it is better that the emotional states of the speakers are similar. Considering the above-mentioned problem of ease of speaking, the difference in emotional state between the talking side and the talking side is small, and both are as positive emotions as possible (that is, both are equally positive emotions). ), But it can be said that it is the most desirable situation for starting a conversation. (Non-Patent Document 1: Ken Fujiwara, Ikuo Daibo, Effect of Emotion on Conversational Satisfaction and Interpersonal Impression-Focusing on Emotional Composition between Speakers, Psychological Research, 84 (5), pp.522-528, 2013).

Therefore, an object of the present invention is to provide a technique for realizing smoother communication. As an example, the present invention acquires emotional information of a calling user and a calling user, and presents an emotion-inducing stimulus to at least one speaker when there is a difference in emotional states between the two. By doing so, the purpose is to change the emotional state of the speaker to which the emotion-inducing stimulus is presented, and then to start communication between the two. Further, as an example, the present invention presents an emotion-inducing stimulus to a speaker with a lower positive degree to increase the positive degree, thereby reducing the difference between the two by aligning the positive degree between the two to the higher one. The purpose is to provide a remote communication system with a new call control function that initiates a call.

In order to solve the above problem, according to a certain viewpoint of the present invention, a difference between the emotion estimation information of the calling side user and the emotion estimation information of the calling side user is calculated, and the difference is set to a predetermined amount. For at least one of the calling side user and the calling side user when it is determined that the difference exceeds the predetermined amount, the judgment unit for determining whether or not to exceed the value. A control device is provided that includes a presentation control unit that controls the presentation of a predetermined stimulus.

The emotion estimation information of the calling side user may include the positive degree of the calling side user, and the emotion estimation information of the calling side user may include the positive degree of the calling side user.

The presentation control unit may control so that the predetermined stimulus is presented to at least the user having a lower positive degree among the calling side user and the calling side user.

The control device may include an opportunity control unit that controls at least the difference to be visually presented to the calling user when it is determined that the difference exceeds the predetermined amount. ..

The control device may include an opportunity control unit that starts communication between the calling side user and the calling side user when a predetermined condition is satisfied.

When the determination unit determines that the difference exceeds the predetermined amount, the determination unit recalculates the difference based on at least the emotion estimation information of the user presented with the predetermined stimulus, and the difference is the predetermined amount. It may be determined whether or not the amount is less than the amount.

When it is determined that the difference is equal to or less than the predetermined amount, the trigger control unit may start communication between the calling side user and the calling side user.

When a predetermined communication start instruction is input, the trigger control unit may start communication between the calling side user and the calling side user.

The determination unit may change the predetermined amount so that the predetermined amount increases when the predetermined time elapses for the time when the difference exceeds the predetermined amount.

When the control device determines that the difference exceeds the predetermined amount, the estimated time until the difference becomes equal to or less than the predetermined amount is determined by the calling side user and the calling side user. A prediction unit may be provided for controlling so as to be presented to at least one of them.

The predetermined stimulus changes the emotion based on an estimation model generated by a learning process based on the data of the known emotion-induced stimulus and the data of the correct answer information of the emotional transformation of the user who received the emotion-induced stimulus. It may be a stimulus presumed to be easy.

The predetermined stimulus may be presented by at least one of image display and sound output.

Further, according to another aspect of the present invention, the computer calculates the difference between the emotion estimation information of the calling side user and the emotion estimation information of the calling side user, and whether the difference exceeds a predetermined amount. Determining whether or not, and when the computer determines that the difference exceeds the predetermined amount, for at least one of the calling side user and the calling side user. A control method is provided, including controlling the presentation unit so that a predetermined stimulus is presented.

Further, according to another aspect of the present invention, the computer calculates the difference between the emotion estimation information of the calling side user and the emotion estimation information of the calling side user, and whether the difference exceeds a predetermined amount. A determination unit for determining whether or not, and when it is determined that the difference exceeds the predetermined amount, a predetermined value is given to at least one of the calling side user and the calling side user. A program for functioning as a control device including a presentation control unit that controls the presentation of a stimulus is provided.

Further, according to another viewpoint of the present invention, the difference between the emotion estimation information of the calling side user and the emotion estimation information of the calling side user is calculated, and whether or not the difference exceeds a predetermined amount is determined. When it is determined that the difference exceeds the predetermined amount, the predetermined stimulus is presented to at least one of the calling side user and the calling side user. Provided is a control system comprising a control device comprising a presentation control unit for controlling the stimulus to be performed, and a presentation device for presenting the predetermined stimulus.

As described above, according to the present invention, it is possible to provide a technique for realizing smoother communication.

It is explanatory drawing which shows an example of the schematic structure of the communication system which concerns on this embodiment. It is a block diagram which shows an example of the hardware composition of the emotion control server, the sensor terminal, and a communication terminal which concerns on this embodiment. It is a block diagram which shows an example of the functional structure of the sensor terminal which concerns on this embodiment. It is a block diagram which shows an example of the functional structure of the communication terminal which concerns on this embodiment. It is a block diagram which shows an example of the functional structure of the emotion control server which concerns on 1st Embodiment. It is explanatory drawing for demonstrating an example of the display screen which shows that the emotional balance adjustment processing is in progress. It is explanatory drawing for demonstrating an example of a display screen including an emotion-inducing stimulus. It is explanatory drawing for demonstrating an example of the display screen during a call process. It is explanatory drawing for demonstrating an example of the data table stored in the storage part by an emotion estimation part. It is explanatory drawing which shows an example of the operation sequence of the communication system which concerns on this embodiment. It is explanatory drawing which shows an example of the operation sequence of the communication system which concerns on this embodiment. It is a block diagram which shows an example of the functional structure of the emotion control server which concerns on 2nd Embodiment. It is explanatory drawing for demonstrating another example of the display screen which shows that the emotion balance adjustment processing is in progress.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same or similar functional configurations are distinguished by adding different alphabets after the same reference numerals. However, if it is not necessary to distinguish each of the plurality of components having substantially the same or similar functional configurations, only the same reference numerals are given.

First, a schematic configuration of a communication system (control system) according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is an explanatory diagram showing an example of a schematic configuration of a communication system according to the present embodiment. Referring to FIG. 1, the communication system includes an emotion control server 100 (control device), a plurality of users 900, a plurality of sensor terminals 200, a plurality of communication terminals 300 (presentation device), and a LAN 50.

Since this communication system has an information communication function between remote locations, a plurality of users 900 may exist spatially apart from each other. In FIG. 1, the user 900A, the sensor terminal 200A, and the communication terminal 300A are located at the base bA. An example is shown in which a user 900B, a sensor terminal 200B, and a communication terminal 300B exist at the base bB. The users 900A and 900B are remotely collaborating (teleworking) with each other, and the base bA and the base bB may be in a remote office relationship.

The sensor terminal 200A and the communication terminal 300A may be individually used terminals of the user 900A. In that case, the user 900A, the sensor terminal 200A, and the communication terminal 300A exchange data for communication identification information (hereinafter, "communication ID"; for example, a telephone number or IP (Internet Protocol) address) associated with each other. The setting process for associating with each other is performed in advance, and the data of the setting may be stored in the emotion control server 100 (the same applies to the user 900B).

FIG. 2 shows the “device according to the present embodiment” without distinguishing between the emotion control server 100, the sensor terminal 200, and the communication terminal 300 (hereinafter, the emotion control server 100, the sensor terminal 200, and the communication terminal 300) according to the present embodiment. It is a block diagram which shows an example of the hardware configuration of). It should be noted that it is not necessary for all of the above devices to be equipped with all of the following hardware configurations (for example, the emotion control server 100 does not need to be directly equipped with a sensor), and for each device described later. A hardware module capable of realizing a functional configuration may be provided with an appropriate limitation.

Referring to FIG. 2, the apparatus according to the present embodiment includes a bus 801 and a CPU (Central Processing Unit) 803, and a ROM (Read Only).
It includes a Memory) 805, a RAM (Random Access Memory) 807, a storage device 809, a communication interface 811 and a sensor 813, an input device 815, a display device 817, and a speaker 819.

The CPU 803 executes various processes in the apparatus according to the present embodiment. Further, the ROM 805 stores a program and data for causing the CPU 803 to execute the processing in the apparatus according to the present embodiment. Further, the RAM 807 temporarily stores the program and the data when the processing of the CPU 803 is executed.

The bus 801 connects the CPU 803, the ROM 805, and the RAM 807 to each other. A storage device 809, a communication interface 811, a sensor 813, an input device 815, a display device 817, and a speaker 819 are further connected to the bus 801. Bus 801 includes, for example, a plurality of types of buses. As an example, bus 801 includes a high-speed bus connecting the CPU 803, ROM 805 and RAM 807, and one or more other buses slower than the high-speed bus.
The storage device 809 stores data to be temporarily or permanently stored in the device according to the present embodiment. The storage device 809 may be, for example, a magnetic storage device such as a hard disk, or an EEPROM (Electrically).
Erasable and Programmable Read Only Memory, flash memory
memory), MRAM (Magnetoresistive Random Access Memory), FeRAM (Ferroelectric Random Access Memory) and PRAM (Phase)
Nonvolatile memory (nonvolatile) such as change Random Access Memory)
memory) may be used.

The communication interface 811 is a communication means included in the device according to the present embodiment, and communicates with an external device via a network (or directly). The communication interface 811 may be an interface for wireless communication, and in this case, for example, a communication antenna, an RF circuit, and other circuits for communication processing may be included. Further, the communication interface 811 may be an interface for wired communication, and in this case, for example, a LAN terminal, a transmission circuit, and other circuits for communication processing may be included.

The sensor 813 is, for example, a camera, a microphone, a biosensor, another sensor, or a combination thereof. The camera captures a subject and includes, for example, an optical system, an image sensor, and an image processing circuit. A microphone picks up ambient sound, converts the sound into an electric signal, and converts the electric signal into digital data. The input device 815 is a touch panel, a mouse, a line-of-sight detection device, and the like.

The display device 817 displays an output image (that is, a display screen) from the device according to the present embodiment, and is, for example, a liquid crystal display, an organic EL (Organic Light-Emitting Diode), or a CRT (Cathode).
It can be realized by using Ray Tube) or the like. The speaker 819 outputs sound, converts digital data into an electric signal, and converts the electric signal into sound.

Next, an example of the functional configuration of the “sensor terminal 200” according to the present embodiment will be described with reference to FIG. The sensor terminal 200 externally measures the behavior and physiological reaction of the user 900 and transmits the acquired data to the emotion control server 100 described later.

FIG. 3 is a block diagram showing an example of the functional configuration of the sensor terminal 200 according to the present embodiment. Referring to FIG. 3, the sensor terminal 200 includes a communication unit 210, a measurement unit 220, and a control unit 230. Although not shown in FIG. 3, the sensor terminal 200 may further include a storage unit for storing measurement data, a display unit for showing the internal operation status to the user, and the like.

The communication unit 210 communicates with another device. For example, the communication unit 210 is directly connected to the LAN 50 and communicates with the emotion control server 100. Further, the communication unit 210 may communicate with another sensor terminal 200. The communication unit 210 may be implemented by the communication interface 811.

The measurement unit 220 measures the outside world to generate multimedia data related to communication, and externally measures the behavior and physiological reaction of the user 900 to acquire the data. The behavior and biological reaction data include, for example, image data including facial expressions and body posture state contents measured by a camera, voice data measured by a microphone, acceleration data of body movements measured by an acceleration sensor, and a keyboard.・ Device operation data measured by input devices such as mouse and touch panel, data of autonomic nervous system activity (heartbeat activity, skin electrical activity, blood pressure, sweating, breathing, skin / core body temperature, etc.) measured by various biological sensors, Includes data on central nervous system activity (brain waves, cerebral blood flow, etc.), data on gaze movement, pupil diameter, number of blinks, etc. measured by a gaze measuring device, data on immune components in saliva or blood, and the like.

The measuring unit 220 may be mounted by the sensor 813. The sensor 813 of the sensor terminal 200 may be a non-contact type or a contact type to the user. When the sensor of the sensor terminal 200 is a contact type, for example, the sensor terminal 200 may be a wearable terminal in the shape of a wristwatch or the like. Alternatively, the sensor terminal 200 may be an HMD (Head Mounted Display). The type of HMD may be a goggle type or a glass type. Further, the sensor terminal 200 may be a smartphone or tablet terminal provided with a camera, a microphone, an acceleration sensor, a communication unit, or the like.

As shown in 200A and B in FIG. 1, one sensor terminal 200 may exist corresponding to one user 900, or one sensor terminal 200 is assumed to be an environment-embedded sensor in a collaborative space. The sensor terminal 200 may acquire sensor data of a plurality of users 900. In that case, the sensor terminal 200 performs a one-to-one correspondence process of the sensor data and the identification data of the measurement target person for each of the plurality of users 900, and obtains each combination of the sensor data and the identification data of the measurement target person. Remember or send. The identification data of the person to be measured may be obtained in any way, and as an example, it may be identification data of a person identified by face recognition from an image captured by a camera.

The control unit 230 provides various functions of the sensor terminal 200. The sensor terminal 200 may perform not only measurement but also data analysis processing including preprocessing, feature extraction processing, and estimation, and the control unit 230 may perform various arithmetic processing in that case. The control unit 230 may be mounted by the CPU 803, the ROM 805, and the RAM 807.

Next, an example of the functional configuration of the "communication terminal 300" according to the present embodiment will be described with reference to FIG. The communication terminal 300 processes a communication request to the communication terminal 300 of another user 900 in response to an input from the user 900, acquires information that estimates the emotion of the other user 900, and presents the information to the user 900. Can be done.

As an example, the communication terminal 300 may be a general-purpose smartphone or tablet terminal. Further, although the communication terminal 300 is shown in FIG. 1 so as to correspond to one user 900, it may be a shared terminal shared by a plurality of users 900. As yet another example, the communication terminal 300 is transmitted from an automated teller machine (VTM) (VTM) with a video communication function, an automatic teller machine at a station, a housing device with a display unit such as a visual call center system, or the like. It may be a terminal that presents information that estimates the customer's emotions based on the measurement data to the support staff.

FIG. 4 is a block diagram showing an example of the functional configuration of the communication terminal 300 according to the present embodiment. Referring to FIG. 4, the communication terminal 300 includes a communication unit 310, a storage unit 320, a control unit 330, an input unit 340, and a presentation unit 350.

The communication unit 310 communicates with another device. For example, the communication unit 310 is directly connected to the LAN 50 and communicates with the emotion control server 100. The communication unit 310 may be implemented by the communication interface 811.

The storage unit 320 stores programs and data for the operation of the communication terminal 300. The storage unit 320 may be mounted by the storage device 809.

The control unit 330 provides various functions of the communication terminal 300. The control unit 330 may be mounted by the CPU 803, the ROM 805, and the RAM 807.

The input unit 340 accepts input from the user 900. Then, the input unit 340 provides the input result to the control unit 330. The input from the user 900 is, for example, to specify another user 900 as a communication request partner, and is realized by selecting the identification information of the other user 900 or the like. The input unit 340 can be mounted by the input device 815.

The presentation unit 350 presents information that can be perceived by the user under the control of the control unit 330. In the first embodiment of the present invention, it is mainly assumed that the presentation unit 350 displays a display screen visually perceived by the user. In such a case, the presentation unit 350 can be realized by the display device 823. However, when the presenting unit 350 presents information perceived by the user's hearing, the presenting unit 350 may be realized by the speaker. Alternatively, when the presenting unit 350 presents information perceived by the user's sense of touch or smell, the presenting unit 350 may be realized by a tactile or sense of smell presenting device.

Next, with reference to FIG. 5, an example of the functional configuration of the “emotion control server 100” according to the first embodiment will be described. FIG. 5 is a block diagram showing an example of the functional configuration of the emotion control server 100 according to the first embodiment. Referring to FIG. 5, the emotion control server 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

The communication unit 110 communicates with another device. For example, the communication unit 110 is directly connected to the LAN 50 and communicates with the sensor terminal 200 and the communication terminal 300. The communication unit 110 may be implemented by the communication interface 811.

The storage unit 120 stores programs and data for the operation of the emotion control server 100. The storage unit 120 includes an emotion-inducing stimulus DB 121 and an emotion recognition dictionary DB 122. The emotion-induced stimulus DB 121 stores the data of the emotion-induced stimulus described later. Further, the emotion-induced stimulus data may be stored (tagged) in association with information on the emotional transformation effect described later.

In addition, the data of the emotion-inducing stimulus and the information of the emotion-changing effect may be associated with each other in any way. For example, if the knowledge about how the emotions of the user who is presented with the emotion-inducing stimulus are likely to change (for example, the degree of positiveness tends to increase, the degree of positiveness tends to decrease, etc.) is obtained in advance. Information indicating how emotions are likely to be transformed (that is, information on emotional transformation effects) may be manually associated with the data of the emotion-inducing stimulus. Alternatively, if the emotional transformation estimation model is generated by the learning process based on the data of the known emotion-induced stimulus and the data of the correct answer information of the emotional transformation of the user who received the emotion-induced stimulus, the emotional transformation estimation model and the new one. Information on the emotional transformation effect associated with the new emotion-induced stimulus data may be estimated based on the emotion-induced stimulus data.

The emotion recognition dictionary DB 122 stores the data of the behavior and the biological reaction of the user 900, and the data of the behavior and the biological reaction may include the sensor data acquired by the sensor terminal 200. Further, the emotion recognition dictionary DB 122 may store data of an emotion estimation model (emotion recognition dictionary) for estimating and processing the user's emotions. The emotion estimation model is generated by learning processing by associating the sensor data acquired in advance with the data of the correct answer information of the user's emotion at the time of acquiring the sensor data. The emotion correct answer information may be measured by the user in the learning processing phase by a questionnaire method or the like. In addition, the emotion estimation model is generated by classifying and dividing the data for each individual of the user 900, for each predetermined period, for each behavior type of the user 900, etc., and learning processing each, and a plurality of emotion estimation models according to the conditions are generated. It doesn't matter if it exists. Since a method for estimating a user's personal emotion from sensor data is known (for example, Japanese Patent Application Laid-Open No. 2012-59107), further description thereof will be omitted in this paper. The storage unit 120 may be mounted by the storage device 809.

The control unit 130 provides various functions of the emotion control server 100. The control unit 130 includes an emotion estimation unit 131, an emotion balance adjustment control unit 133 (judgment unit), an emotion guidance stimulus presentation control unit 135 (presentation control unit), and a call trigger control unit 137 (trigger control unit). The control unit 130 may be mounted by the CPU 803, the ROM 805, and the RAM 807.

The emotion estimation unit 131 estimates personal emotions for each user 900 based on measurement data (sensor data) of behaviors and biological reactions acquired from the user 900 via the sensor terminal 200 and the communication unit 110, and estimates based on the model data. Generate (classified) emotion estimation data. Further, the emotion estimation unit 131 has a function of storing the generated estimation model data and emotion estimation data in the storage unit 120.

Here, the explanation of personal feelings and their estimation methods is supplemented. Personal emotions are, for example, "evaluative reactions to people, things, events, and the environment among various information processes that people perform in their mental processes" (Ortony et al., 1988; Ohira, 2010). Is defined as. Specific types of emotions include happiness, surprise, fear, anger, disgust, sadness, and psychologist James A. on a basic emotion-based discrete model that corresponds to facial expressions by psychologist Paul Ekman. The quadrants of emotions and sorrows in the emotional dimension-based continuous model of pleasure and alertness by Russel are known. Other continuous models include Watson's positive or negative emotions, Wundt's three-axis model (pleasure, excitement, tension), and Plutchik's four-axis model. Other applied / complex emotions include confusion, interest, mental stress, concentration, fatigue, busyness, creativity, relaxation / tension, motivation, empathy, and reliability. Furthermore, the liveliness that is experienced as a group atmosphere in business activities can be said to be a type of higher-level emotion. The effective range of the definition of emotion in the embodiment of the present invention is wider than the above-mentioned basic emotion, and includes all internal "states" of the user and "situations" in consideration of the influence of the user's surrounding environment and context. As an example, the positive emotion and its degree are the degree of rapidity itself, the combination of the degree of pleasure and the degree of arousal, the magnitude of the intensity of happiness in the basic emotion, or the magnitude of the intensity of fear, anger, disgust, sadness, etc. May be expressed as an index.

The emotions and degree of a person can be determined by, for example, using the questionnaire method, by linguistic reporting of the person's letters, sentences, and symbols. As the questionnaire, "Affect Grid" and the like are well known. However, since the measurement method using the questionnaire requires answering work, the measurement itself may interfere with the original purpose work in daily life where some other work such as work is performed. be.

Therefore, in this communication system, the emotion estimation unit 131 mechanically estimates the emotion (obtained by the questionnaire method or the like) based on the behavior and biological reaction data measured by the sensor terminal 200 described above. In order to perform the estimation process, the data of the emotion estimation model generated in advance by the learning process is required. The emotion estimation model is generated from, for example, a group of training data in which sensor data of the behavior or biological reaction at a certain time point / situation is associated with answer data of the questionnaire. For example, questions about the user's facial expression, voice, heartbeat activity, skin electrical activity, and other behavioral / biological data measured from countless cameras, microphones, and wearable activity meters embedded in the office, and the user's subjective emotions. The correct answer data that was answered on paper is associated with the training data. The sensor data of the behavior or the biological reaction may be feature data for learning processing in which the measured value from the sensor is converted.

The feature amount data may be the position of a representative feature point of the face, the distance of a straight line connecting each of the two points, or the angle formed. Alternatively, the feature data may be the fundamental frequency, power, average speech speed, maximum value and standard deviation of the first-order cepstrum coefficient of the voice. Alternatively, the feature amount data may be the average value of the heart rate or the beat interval, the standard deviation, or the heart rate variability. Alternatively, the feature amount data may be an average value of the skin conductance level, a standard deviation, an increase / decrease rate, or the like. These feature data may be used in any way, as an absolute value at a certain time point, or as a relative rate of change between two time points.

For the generation of the emotion estimation model using the training data, for example, a known SVM (Support Vector Machine) or deep learning method may be used as a learning method, or a regression analysis method is simply used. It may be used. Further, the learning model may be generated for each individual user, or a model common to humans may be generated using training data of a plurality of users. The emotion estimation unit 131 can estimate individual emotions from the behavior / biological data of a certain person by using the obtained emotion estimation model data.

The emotion control server 100 (for example, the emotion estimation unit 131) may have a function of generating training data for the above-mentioned individual emotion estimation process or an emotion estimation model itself. Alternatively, the above-mentioned feature amount data for the training data is generated not by the emotion control server 100 but by the sensor terminal 200, and the sensor terminal 200 transmits the feature amount data to the emotion control server 100. May be good.

FIG. 9 is an explanatory diagram for explaining an example of a data table stored in the storage unit 120 by the emotion estimation unit 131. In the data table of FIG. 9, the ID of the user 900 (may be a communication ID) and the personal emotion estimated at the current time of each user 900 are stored. Further, the data of the past history of the individual emotion may be stored, or the data of the emotion estimation model for each individual may be stored. These data are acquired and used by the emotional balance adjustment control unit 133, which will be described later.

When the user 900 starts a call using this system, the emotional balance adjustment control unit 133 estimates each of the calling side user (speaking side user) and the calling side user (speaking side user). Emotion estimation information is compared (the difference between emotion estimation information is calculated), and if there is a difference exceeding a predetermined amount between the two, at least one of the calling user and the calling user , A processing command for controlling so that a predetermined stimulus (emotion-induced stimulus) is presented is issued to the emotion-induced stimulus presentation control unit 135. Then, the emotion balance adjustment control unit 133 controls the call trigger to start a call (communication) between the two after the emotion estimation information of the user who has received the emotion induction stimulus approaches the emotion estimation information of the other user. Put out to part 137. The process related to the emotion guidance is performed by the emotion guidance stimulus presentation control unit 135 described later, and the process related to the start of the call is performed by the call trigger control unit 137 described later in response to the request command (described later).

In the present embodiment, it is mainly assumed that the absolute value of the emotion value indicating the user's emotion is used as the emotion estimation information. However, as the emotion estimation information, instead of the absolute value of the emotion value indicating the user's emotion, the emotion value indicating the user's emotion is a relative value (for example, the maximum value) normalized (or standardized) for each individual. The reference ratio) may be used.

Further, in the present embodiment, it is mainly assumed that a call between the two is started when the difference between the emotion estimation information becomes a predetermined amount or less. However, in addition to the case where the difference in the emotion estimation information is less than a predetermined amount, the emotion value (absolute value or relative value) of both (or all three or more users who communicate) is larger than the predetermined value. In some cases, communication may be initiated. Then, smoother communication can be realized. For example, in addition to the case where the difference in emotion estimation information is less than or equal to a predetermined amount, the relative value of each of the two (or all three or more users who communicate) is larger than the predetermined value (for example, 80%). If it is large, communication may be started.

Here, the emotion-inducing stimulus may be a stimulus that reduces the difference in emotion estimation information. For example, the emotion balance adjustment control unit 133 presents an emotion-inducing stimulus (for example, a stimulus that tends to increase the positive degree) to at least a user having a lower positive degree among the calling side user and the calling side user. A processing command for controlling the emotions may be issued to the emotion-induced stimulus presentation control unit 135. This is expected to induce emotions in the direction of increasing the positiveness of users with lower positiveness, and to bring the positiveness of both closer.

Alternatively, the emotion balance adjustment control unit 133 is used instead of a processing instruction for controlling the emotion-inducing stimulus (for example, a stimulus that tends to increase the positive degree) to be presented to a user having a lower positive degree, or. Along with the processing instruction, a processing instruction for controlling the emotion-induced stimulus (for example, a stimulus whose positive degree is likely to decrease) is presented to a user having a higher positive degree is issued to the emotion-induced stimulus presentation control unit 135. May be good. This is expected to induce emotions in the direction of lowering the positiveness of users with higher positiveness, and to bring the positiveness of both closer.

In the following, mainly when the emotion estimation information of the calling user includes the positive degree of the calling user and the emotion estimation information of the calling user includes the positive degree of the calling user. Suppose. However, the emotion estimation information of the calling user is not limited to the positive degree, and may be information indicating emotions other than the positive degree. Similarly, the emotion estimation information of the calling user is not limited to the positive degree, and may be information indicating emotions other than the positive degree.

More specifically, the emotional balance adjustment control unit 133 receives, for example, a call request signal from the user 900A to the user 900B using the communication terminal 300A via the communication unit 110, and the storage unit 120 receives the call request signal from the storage unit 120 to the users 900A and 900B. When the data of personal emotion information (emotion estimation information) is acquired and the two are compared and it is determined that the difference in the positive degree between the two is equal to or more than a predetermined amount, the user 900 having the lower positive degree A processing request is sent to the emotion-guided stimulus presentation control unit 135 to transmit stimulus data that changes emotions in the direction of increasing the degree of positivity to the communication terminal 300.

After that, the emotion balance adjustment control unit 133 acquires the data of the personal emotion information (emotion estimation information) of the users 900A and 900B at any time and continues to monitor the positiveness of both, and the difference in the positiveness between the two is a predetermined amount. When it becomes smaller below, the call trigger control unit so as to start the communication (call or media streaming) between the communication terminals 300A and 300B of both based on the data of the communication ID of the user 900A and the communication ID of the user 900B. A processing request is sent to 137.

Here, the predetermined amount may be preset by the administrator of the system or the like (for example, 10% of the positive degree having the higher difference is set as the threshold value of the predetermined amount). Alternatively, as a result of emotional transformation by an emotion-inducing stimulus, which will be described later, a case where the difference between the two disappears (becomes zero) is mechanically detected, and at that time, the emotional balance is adjusted when the difference in positiveness becomes less than a predetermined amount. The control unit 133 may perform the determination process.

In the present embodiment, a predetermined amount to be compared with the difference in positiveness between the two to control the start timing of emotion-induced stimulus presentation, and a positive between the two to control the start timing of communication between the two. It is assumed that the predetermined amount to be compared with the difference in degrees is the same. However, a predetermined amount to be compared with the difference in positiveness between the two to control the start timing of emotion-induced stimulus presentation, and the difference in positiveness between the two to control the start timing of communication between the two. It may not be exactly the same as the predetermined amount to be compared, or it may be different.

Further, in the present embodiment, it is mainly assumed that the predetermined amount is a constant value. However, the predetermined amount may change over time. For example, even if a stimulus is presented to at least one of the users 900A and 900B, it is assumed that the difference in positiveness between the two does not become small. Therefore, the emotional balance adjustment control unit 133 may change a predetermined amount so that the predetermined amount increases when a predetermined time elapses when the difference between the positive degrees of the two exceeds the predetermined amount. good. This facilitates the initiation of communication between the user 900A and the user 900B.

As an example, the emotional balance adjustment control unit 133 increases the predetermined amount when the difference between the positive degrees of the two does not become less than or equal to the predetermined amount even after the lapse of the first time (for example, 1 minute). If the difference between the positivenesses of the two does not become less than or equal to the predetermined amount even after the second time (for example, 3 minutes) longer than the first time elapses, the predetermined amount is further increased. You may let me. The first time and the second time may be set by the administrator or the user of the system, respectively.

The emotion-induced stimulus presentation control unit 135 performs a process of transmitting data of an emotion-induced stimulus that changes the emotion in the direction of increasing the positiveness to the communication terminal 300 of the user 900. More specifically, the emotion-inducing stimulus presentation control unit 135 receives a request command from the emotion balance adjustment control unit 133, and the user 900 designated by the emotion balance adjustment control unit 133 (the user with the lower positive degree as a result of comparison). The communication ID of the above is acquired, and the data of the appropriate emotion-inducing stimulus acquired from the emotion-inducing stimulus DB 121 is transmitted to the communication ID.

For example, the emotion-inducing stimulus may be a photograph (still image), a video (moving image), music, or the like having an effect of changing the emotion of the viewer. Here, the displayed image may be simply a photograph of a landscape or the like, or may be a photograph with a message explaining the characteristics of the landscape. Further, when the image is displayed, a predetermined sound effect may be output together with the image display in order to make the user 900 notice the image display. To have the effect of changing emotions, for example, it is desirable to carry out subjective evaluation, physiological index evaluation, etc. in advance for the stimulus and confirm the effect (effect of increasing the degree of positiveness, etc.). Instead of creating emotion-induced stimuli by yourself, you may use an existing publicly available data set of emotion-induced stimuli. As existing emotion-inducing stimuli, IAPS (International Affective Picture System) and GAPED (Geneva Affective PicturE Database), which are emotion-inducing standard image data sets, are known, and the images included in them are preliminarily used by a large number of subjects. The presentation transformation effect of positiveness and arousal is evaluated and confirmed, and information is tagged.

The emotion-inducing method is not limited to the above-mentioned method, and there are many methods such as the Beltane method and the method of intentionally creating facial expressions as introduced in Non-Patent Document 2, for example, and the emotion-inducing stimulus presentation control unit 135. May perform processing that allows the user to reproduce those methods. (Non-Patent Document 2: Westermann, R. et al., Relative effectiveness and validity of mood
induction procedures: A meta-analysis. European Journal of Social Psychology,
26, pp.557-580, 1996.).

As described above, the emotion-induced stimulus may be a stimulus that is known by experiments that the emotion of the user who receives the presentation is liable to change (for example, the positivity is liable to increase, the positivity is liable to decrease, etc.). .. Alternatively, the emotion-induced stimulus changes the emotion based on an estimation model generated by the learning process based on the data of the known emotion-induced stimulus and the data of the correct answer information of the emotional transformation of the user who received the emotion-induced stimulus. It may be a stimulus that is presumed to be easy (for example, the positive degree tends to increase, the positive degree tends to decrease, etc.).

Further, in the present embodiment, it is mainly assumed that the emotion-inducing stimulus presented to the user is constant. However, the emotion-induced stimuli presented to the user may change over time. For example, if the difference between the two positive degrees does not fall below a predetermined amount even after a predetermined time has passed, the emotion-inducing stimulus presented to the user is changed to an emotion-inducing stimulus having a stronger transforming effect. You may. Emotion-induced stimuli may be modified in any way. The volume output as the emotion-inducing stimulus may be increased, and the type of the image displayed as the emotion-inducing stimulus may be changed.

When the call trigger control unit 137 determines that the difference in positiveness between the calling side user 900A and the calling side user 900B is less than or equal to a predetermined amount, the call trigger control unit 137 of the communication terminal 300A and the user 900B of the user 900A. Performs processing for establishing communication between communication terminals 300B. More specifically, the call trigger control unit 137 receives a request command from the emotion balance adjustment control unit 133, and acquires the communication IDs of the calling side and the calling side user 900 designated by the emotion balance adjustment control unit 133, respectively. Then, when the emotional balance adjustment control unit 133 receives a determination signal that the difference in positiveness between the users 900 is equal to or less than a predetermined amount, communication between the communication IDs is established and the communication ID is linked to the communication ID. It is possible to make a call between the communication terminals 300 (not only voice but also a videophone).

Further, the call trigger control unit 137 receives a call request signal from the user 900A using the communication terminal 300A to the user 900B, and the emotion balance adjustment control unit 133 commands the emotion guidance stimulus presentation control unit 135. Until the difference in the positiveness of the individual emotions of the user 900A and the user 900B is reduced to a predetermined amount or less, the information indicating the status of the emotional balance adjustment processing is transmitted to the user 900 (in this case, especially 900A) via the communication terminal 300. ) May be performed (FIG. 6 described later). Information indicating the status of the emotional balance adjustment process may be presented to the user 900B instead of the user 900A or in addition to the user 900A.

FIG. 6 is an explanatory diagram for explaining an example of a display screen showing that the above-mentioned emotional balance adjustment process presented by the presentation unit 350 of the communication terminal 300 of the calling user 900 is in progress.

On the display screen, the ID 610 of the calling user 900 (ID is “900B” in the example of the figure), and the difference in emotional positiveness between the calling user 900 and the calling user 900 are predetermined amounts. Message 620 that the emotional balance adjustment process is being performed because it is larger than, and the current emotional positiveness of the calling user 900 and the calling user 900 (values are updated in real time). A graph 630 showing (may be good), a button 640 for making a call now, and the like are displayed.

By presenting the graph 630 showing the current emotional positivity of the calling user 900 and the calling user 900, the calling user 900 can visually recognize the difference between the positivity of the two. .. In this way, when it is determined that the difference between the positive degrees of the two exceeds a predetermined amount, the call trigger control unit 137 presents at least the difference between the positive degrees of the two to the calling user 900 so as to be visible. It is good to control so as to. The difference between the positive degrees of both may be presented by a mode other than the graph 630 (for example, the positive degree of both, the difference value of the positive degree of both, etc.).

In the above, when the call trigger control unit 137 determines that the difference in positiveness between the two is less than or equal to a predetermined amount, the call trigger control unit 137 starts communication between the calling user 900 and the calling user 900. The example was mainly explained. However, the timing at which the call trigger control unit 137 starts communication between the calling side user 900 and the calling side user 900 is not limited to such an example. For example, the call trigger control unit 137 may start communication between the calling side user 900 and the calling side user 900 when a predetermined condition is satisfied. For example, the call trigger control unit 137 may start communication between the calling side user 900 and the calling side user 900 when a predetermined communication start instruction is input. The communication start instruction may be an operation of selecting the button 640 to make a call immediately.

The call button 640 is an input interface used not only in the positive state of the user 900 but also in a highly urgent situation where the user wants to make a call immediately, and controls when the input is received by the touch of the user 900 or the like. A signal is sent via the unit 330, the communication unit 310, and the communication unit 110, and the received call trigger control unit 137 immediately establishes communication between the calling side user 900 and the calling side user 900 communication terminal 300. Let me.

The display screen of FIG. 6 is displayed until the emotional balance adjustment process is completed (when the difference in positiveness becomes less than or equal to a predetermined amount, or when communication between the communication terminals 300 is established), and then. May transition to the screen of FIG. 8 described later or a general call status screen, or may continue to display, for example, a graph 630 showing the positiveness of emotions as communication support information during a call even during the call status screen. ..

FIG. 7 is an explanatory diagram for explaining an example of a display screen including the above-mentioned emotion-inducing stimulus presented by the presentation unit 350 of the communication terminal 300 of the calling user 900.

On the display screen, the above-mentioned emotion-inducing stimulus 710 (a photograph of a beautiful landscape in the example of the figure), a button 720 for ending the emotion-inducing process, and the like are displayed. When the button 720 for ending the emotion guidance process is selected by the touch of the user 900 or the like, the control unit 330 ends the display of the display screen of the presentation unit 350, and the control unit 330, the communication unit 310, and the communication unit. A signal is sent via 110, and the received call trigger control unit 137 transmits information to the emotion balance adjustment control unit 133 that the emotion guidance process has been completed.

Upon receiving the information, the emotional balance adjustment control unit 133 ends the emotional balance adjustment process and transmits a message of data including the information to the communication terminal 300 of the calling user 900. The calling user 900 can refer to the message from the communication terminal 300, and can determine the next action (such as skipping the emotional balance adjustment process and making a direct call).

In the above description, the call trigger control unit 137 describes an example in which the communication is established when the emotional balance adjustment process is completed and the call is started immediately, but the present embodiment is limited to this operation. Not done. For example, you may insert an incoming / answering procedure before starting a call. That is, when the emotional balance adjustment process is completed, an INVITE message is transmitted to the communication terminal 300 of the calling user 900, and when the communication terminal 300 receives the INVITE message, the call process (ring tone) of the user 900 is performed. The call may be started after the incoming user 900 receives a call (ringing, etc.).

FIG. 8 is an explanatory diagram for explaining an example of the display screen during the call processing presented by the presentation unit 350 of the communication terminal 300 of the calling side and the calling side user 900. On the display screen, in addition to the general call standby information during the call processing of the user 900, a graph 830 showing the current emotional positivity of the calling user 900 and the calling user 900 is displayed. You may.

Referring to FIG. 8, as an example of general telephone standby information, the display screen displayed by the presentation unit 350A includes a stop button 840 for stopping the call. Further, the display screen displayed by the presentation unit 350B includes a response button 850 for answering an incoming call. Graph 830 has the same information as Graph 630 of FIG. 6 described above, but since the time when the screen of FIG. 8 is displayed is the time when the emotional balance adjustment process is completed, the call is made to the user 900 on the calling side. The difference in positiveness between the 900 users on the side is very small. Furthermore, the magnitude of the current positiveness of both is shown.

The calling side user 900 and the calling side user 900 are in a state where their emotional states are similar to each other and can recognize the degree of positiveness of each other (a state in which they can grasp each other's situation). In other words, it is possible to start a call with shared awareness), so conversation communication can be started more smoothly compared to conventional telephone devices that do not have the emotional balance adjustment processing function of this system. ..

Various variants are also envisioned. In the above, an example of the emotion balance adjustment processing function in the usage situation of one user on the calling side and one user on the calling side has been described. However, this function may be realized in the situation of a multi-person call by the use of three or more users. In that case, the emotional balance adjustment process is performed until the difference between the positive degrees of 3 or more (for example, the difference between the maximum positive degree and the minimum positive degree among the positive degrees of 3 or more people) becomes a predetermined amount or less. You may be disappointed. However, the maximum and minimum positives can change over time.

Further, in the above, a predetermined amount of emotion estimation information is targeted for transformation, but as described above, there is also a theory of a discrete model based on basic emotions in emotions, and in such a case, it is not a quantity but a call. The emotion balance adjustment control unit 133 may operate so as to align the "quality" of the emotion estimation information of the user on the side and the calling side (to the emotion with a stronger positive degree). For example, if the estimated emotion of the calling user is the basic emotion "happiness" and the estimated emotion of the calling user is the basic emotion "sadness", the estimated emotion of the calling user is The presentation control of emotion-inducing stimuli may be performed so as to be discretely transformed into "happiness" or "joy".

In addition, the multi-person call does not need to be started at the same time, and when a new user enters the call later in a situation where the call is already made using this system, the multi-person call becomes a multi-person call. Also, emotional balance adjustment processing may be performed. In that case, the user who enters later is regarded as the calling user, and the emotion control server 100 acquires the data of the emotion estimation information of the user group already in the call, or induces the emotion to the user group already in the call. You may present a stimulus.

Subsequently, an example of the information processing operation according to the present embodiment will be described with reference to FIGS. 10 and 11. 10 and 11 are explanatory views showing an example of an operation sequence of the communication system according to the present embodiment.

In the operation sequence described later, the user 900A of the base bA makes a call to the user 900B at the base bB using the communication terminal 300A, and the difference in positiveness between the users 900A and 900B is predetermined. Since it was more than the amount (for example, 900B is lower), the emotional balance adjustment process for users 900A and 900B was performed by this system, and the difference in positiveness between users 900A and 900B became less than the predetermined amount. This is an example of the flow of processing until a call is established later.

As shown in FIG. 11, in step S1101, the user 900A uses the communication terminal 300A to input an operation to be called to the user 900B (communication terminal 300B). In step S1103, the communication terminal 300A generates a communication request message including the identification information of the users 900A and 900B, and transmits the communication request message to the emotion control server 100.

In step S1105, the emotion control server 100 receives the communication request message from the communication terminal 300A, and the sensor terminals 200A and 200B associated with the identification information of the users 900A and 900B from the setting data stored in advance in the storage unit 120. Performs information communication control processing that specifies the communication ID. In step S1107, the emotion control server 100 transmits a data request message to the sensor terminal 200A, and in step S1109, the emotion control server 100 transmits a data request message to the sensor terminal 200B.

In step S1111, the sensor terminal 200A receives a data request message from the emotion control server 100 in step S1113, and the measurement unit 220 measures the behavior and biological reaction of the outside world, for example, the user 900A or 900B. To get those measurement data. In step S1115, the sensor terminal 200A transmits the measurement data (data of the behavior or biological reaction of the user 900A or 900B) to the emotion control server 100 in step S1117.

In step S1119, the emotion control server 100 is based on the received measurement data.
Estimate the positiveness of personal emotions of users 900A or 900B. Further, the emotion control server 100 compares the positiveness of the individual emotions of the user 900A or 900B, and determines whether or not the difference in the positiveness between the two is equal to or greater than a predetermined amount. If the difference in positiveness is equal to or greater than a predetermined amount, step S1121 is then executed.

In the example of this operation sequence, the case where the emotional positiveness of the user 900B is smaller than the user 900A by a predetermined amount or more will be described. If the difference in positiveness is less than or equal to a predetermined amount, the subsequent steps S1121 to 1143 are skipped, and then step S1145 is executed. In step S1121, the emotion control server 100 decides to execute the emotion balance adjustment process so as to reduce the difference in the positive degree, and prepares for the process execution such as acquisition of the emotion-inducing stimulus.

In step S1123, the emotion control server 100 transmits the data of the emotion-inducing stimulus (which raises the positiveness of the emotion of the user 900B) to the communication terminal 300B. In step S1125, the communication terminal 300B receives the data of the emotion-inducing stimulus from the emotion control server 100, and presents the display screen including the emotion-inducing stimulus to the presentation unit 350B (for example, FIG. 7 described above). In step S1127, the emotion control server 100 transmits the processing status notification data including the fact that the emotion balance adjustment processing is in progress to the communication terminal 300A.

In step S1129, the communication terminal 300A receives the processing status notification data including the emotional balance adjustment processing being performed from the emotion control server 100, and the presentation unit 350A indicates to the display unit 350A that the emotional balance adjustment processing is in progress. (For example, FIG. 6 described above).

Steps S1131 to S1141 are for re-estimating the positiveness of the user 900B who has been emotionally transformed as a result of the processing of steps S1121 to S1129 and the positiveness of the user 900A who may have been emotionally transformed as a result of the passage of time. Will be executed. Since steps S1131 to S1141 are the same processes as steps S1107 to S1117 described above, description thereof will be omitted here.

In step S1143, the emotion control server 100 performs the same process as in step S1119. If the difference in positiveness is equal to or greater than a predetermined amount, step S1121 is executed again. If the difference in positiveness is less than or equal to a predetermined amount, step S1145 is then executed. In step S1145, the emotion control server 100 transmits a message requesting the call start processing to the communication terminal 300A, and in step S1147, the emotion control server 100 transmits the message requesting the communication terminal 300B to the communication terminal 300B.

In step S1149, the communication terminal 300A and the communication terminal 300B can start the communication process related to the call via the emotion control server 100, and can start the call between the communication terminal 300A and the communication terminal 300B.

As described above, according to the present embodiment, the emotional balance adjustment function that aligns the emotions of the calling user and the calling user in the state of higher positiveness is realized in the remote communication system. It is possible to start a conversation with a good emotional state for both the calling side and the calling side user, and it becomes possible to perform smoother remote communication as compared with a conventional telephone system or the like.

Next, a second embodiment of the remote communication system will be described. An example of the functional configuration of the “emotion control server 1000” according to the second embodiment will be described with reference to FIG. FIG. 12 is a block diagram showing an example of the functional configuration of the emotion control server 1000 according to the second embodiment. Referring to FIG. 12, the emotion control server 1000 includes a communication unit 110, a storage unit 120, and a control unit 130. Here, unlike the first embodiment, the control unit 130 has an emotion-inducing action prediction unit 139 (prediction unit).

The emotion-inducing action prediction unit 139 predicts that the difference between the estimated positiveness of the calling user and the calling user will be less than or equal to the predetermined amount when it is determined that the difference exceeds the predetermined amount. The time is controlled to be presented to at least one of the calling user and the called user. In the following, the case where the estimated time is presented to the calling user will be mainly described.

FIG. 13 is an explanatory diagram for explaining another example of the display screen showing that the above-mentioned emotional balance adjustment process is being performed, which is presented by the presentation unit 350 of the communication terminal 300 of the calling user 900. In the second embodiment, as in the predicted time information 650 shown in FIG. 13, the predicted time until the emotional balance adjustment process is completed is further displayed on the display screen indicating that the emotional balance adjustment process is in progress. NS.

The emotion-inducing action prediction unit 139 learns the reaction characteristics to the emotion-inducing action of each user 900, and when a certain user 900 is presented with an emotion-inducing stimulus of a certain property, the user 900 sets a target after, for example, how many seconds. Predict whether it will be in an emotion-induced state. The emotion-inducing action prediction unit 139 acquires data of personal emotion information (emotion estimation information) of the user 900 at any time from the time when the user 900 is presented with the emotion-inducing stimulus by the above-mentioned emotion-inducing stimulus presentation control unit 135. The storage unit 120 stores data including information on the time-dependent transformation characteristics of individual emotions with respect to the emotion-induced stimulus of the user 900.

The data associated in this way is accumulated in the storage unit 120, and is learned and processed by the emotion-inducing action prediction unit 139, and what kind of emotion-inducing stimulus (for example, a stimulus that raises or lowers the positiveness) is used for each user 900. An emotion-induced reaction prediction model that can estimate how individual emotions change over time is generated by presenting (how much is the intensity of the stimulus), and the data of the emotion-induced reaction prediction model is stored in the storage unit 120. ..

When a user 900 calls another user 900 using this system, the emotion-inducing action prediction unit 139 is requested by the emotion balance adjustment control unit 133, and the emotion-inducing stimulus presentation control unit 135 requests the other user 900. Information on the emotion-inducing stimulus presented to the user and its emotion-changing effect is acquired, and the data of the information is used as input data of the emotion-inducing reaction prediction model of the other user 900 to predict the time-dependent change of personal emotion as output data. Data is generated (for example, the difference in positiveness between the calling user 900 and the other user 900 becomes smaller than a predetermined amount "after 6 seconds").

Further, the emotion-inducing action prediction unit 139 transmits the prediction data of the time-dependent change of the individual emotion to the communication terminal 300 of the calling user 900 via the communication unit 110. The communication terminal 300 receives the prediction data of the time-dependent change of the individual emotion, and displays, for example, the prediction time until the emotion balance adjustment process is completed (for example, the prediction time information 650 shown in FIG. 13) on the presentation unit 350. ..

The emotion-induced reaction prediction model may be a prediction model for a plurality of person groups, not for each individual.

Subsequently, the information processing operation according to the second embodiment will be described with reference to FIG. In step S1121 of FIG. 11, when the content of the emotion-inducing stimulus is determined by the emotion balance adjustment control unit 133, the emotion-inducing action prediction unit 139 may generate prediction data of the time-dependent change of the individual emotion. Further, in step S1127 of FIG. 11, the emotion control server 100 may transmit the prediction data of the time-dependent change of the personal emotion to the communication terminal 300A together with the above-mentioned processing status notification data.

Further, in step S1129 of FIG. 11, the communication terminal 300A receives the prediction data of the time-dependent change of the personal emotion from the emotion control server 100, and the presentation unit 350A includes a display screen including the prediction time until the emotion balance adjustment process is completed. May be presented.

As described above, according to the second embodiment, when the emotional balance adjustment function operates in the remote communication system, the estimated time until the calling user finishes the emotional balance adjustment function and can start a conversation. By being able to provide the information of the calling side and grasping the waiting time until the conversation starts, the user can use the emotional balance adjustment function without becoming an uneasy psychological situation where he / she does not know when to start the conversation. Available.

Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It is clear that a person having ordinary knowledge in the field of technology to which the present invention belongs can come up with various modifications or modifications within the scope of the technical idea described in the claims. , These are also naturally understood to belong to the technical scope of the present invention.

For example, in the above, the display screen showing that the control unit 330 of the communication terminal 300 is in the process of laying out various information presented by the presentation unit 350 (for example, the emotional balance adjustment process shown in FIG. 6, FIG. 7). It is mainly assumed that the display screen including the emotion-inducing stimulus shown, the display screen during the call processing shown in FIG. 8, the display screen indicating that the emotion balance adjustment processing shown in FIG. 13 is being performed, and the like) are determined. However, the layout of various information presented by the presentation unit 350 may be determined by another device (for example, the emotion control server 100).

Further, in the above, it is mainly assumed that the emotion control server 100 is provided independently of the communication terminal 300 of the user 900 on the calling side and the calling side. However, some or all of the functions of the emotion control server 100 may be incorporated in the communication terminal 300 of the calling or calling user 900. In such a case, the function realized by the communication between the emotion control server 100 and the communication terminal 300 of the calling side or the calling side user 900 is the communication terminal 300 of the calling side and the calling side user 900. It is realized by P2P communication between.

100 Emotion control server 110 Communication unit 120 Memory unit 121 Emotion-guided stimulus DB
122 Emotion recognition dictionary DB
130 Control unit 131 Emotion estimation unit 133 Emotion balance adjustment control unit 135 Emotion guidance stimulus presentation control unit 137 Call trigger control unit 139 Emotion induction action prediction unit 1000 Emotion control server 200 Sensor terminal 210 Communication unit 220 Measurement unit 230 Control unit 300 Communication terminal 310 Communication unit 320 Storage unit 330 Control unit 340 Input unit 350 Presentation unit

Claims (14)

A determination unit that calculates the difference between the emotion estimation information of the calling side user and the emotion estimation information of the calling side user, and determines whether or not the difference exceeds a predetermined amount.
When it is determined that the difference exceeds the predetermined amount, control is performed so that a predetermined stimulus is presented to at least one of the calling side user and the calling side user. Presentation control unit and
A control device. The emotion estimation information of the calling side user includes the positive degree of the calling side user.
The emotion estimation information of the calling party user includes the positive degree of the calling party user.
The control device according to claim 1. The presentation control unit controls so that the predetermined stimulus is presented to at least the user having a lower positive degree among the calling side user and the calling side user.
The control device according to claim 2. The control device includes an opportunity control unit that controls at least the difference to be visually presented to the calling user when it is determined that the difference exceeds the predetermined amount.
The control device according to claim 1. The control device includes an opportunity control unit that starts communication between the calling side user and the calling side user when a predetermined condition is satisfied.
The control device according to claim 1. When the determination unit determines that the difference exceeds the predetermined amount, the determination unit recalculates the difference based on at least the emotion estimation information of the user presented with the predetermined stimulus, and the difference is the predetermined amount. Judge whether it is less than the amount,
When it is determined that the difference is equal to or less than the predetermined amount, the trigger control unit starts communication between the calling side user and the calling side user.
The control device according to claim 5. When a predetermined communication start instruction is input, the trigger control unit starts communication between the calling side user and the calling side user.
The control device according to claim 5. The determination unit changes the predetermined amount so that the predetermined amount increases when the predetermined time elapses for the time when the difference exceeds the predetermined amount.
The control device according to claim 1. When the control device determines that the difference exceeds the predetermined amount, the estimated time until the difference becomes equal to or less than the predetermined amount is determined by the calling side user and the calling side user. A predictor that controls to be presented to at least one of the
The control device according to claim 1. The predetermined stimulus changes the emotion based on an estimation model generated by a learning process based on the data of the known emotion-induced stimulus and the data of the correct answer information of the emotional transformation of the user who received the emotion-induced stimulus. It is a stimulus that is presumed to be easy,
The control device according to claim 1. The predetermined stimulus is presented by at least one of image display and sound output.
The control device according to claim 1. The computer calculates the difference between the emotion estimation information of the calling user and the emotion estimation information of the calling user, and determines whether or not the difference exceeds a predetermined amount.
When the computer determines that the difference exceeds the predetermined amount, a predetermined stimulus is presented to at least one of the calling side user and the calling side user. To control the presentation part and
Control methods, including. Computer,
A determination unit that calculates the difference between the emotion estimation information of the calling side user and the emotion estimation information of the calling side user, and determines whether or not the difference exceeds a predetermined amount.
When it is determined that the difference exceeds the predetermined amount, control is performed so that a predetermined stimulus is presented to at least one of the calling side user and the calling side user. Presentation control unit and
A program for functioning as a control device. A determination unit that calculates the difference between the emotion estimation information of the calling side user and the emotion estimation information of the calling side user, and determines whether or not the difference exceeds a predetermined amount.
When it is determined that the difference exceeds the predetermined amount, control is performed so that a predetermined stimulus is presented to at least one of the calling side user and the calling side user. Presentation control unit and
With a control device and
A presentation device that presents the predetermined stimulus and
Has a control system.

Images