JPWO2016178329A1 - Information processing system, control method, and storage medium - Google Patents

Abstract

An information processing system, a control method, and a storage medium capable of generating an advice for constructing an ideal communication state based on a signal indicating a user's emotion are provided. An application setting unit that sets an application in which a first user communicates with a second user, a communication unit that receives an emotion signal of at least one of the first user and the second user, and the reception Based on an ideal state according to the use of the sentiment signal, a generation unit that generates advice to the first user, and the generated advice corresponds to the first user via the communication unit And a control unit that transmits the information to the client terminal. [Selection] Figure 1

Description

  The present disclosure relates to an information processing system, a control method, and a storage medium.

  Conventionally, in social communication such as negotiations, meetings, sales, presentations, etc., people talk while listening to the situation of the other party to create an ideal state of communication, but such communication ability depends on individual skills. There were many.

  For example, the following Patent Document 1 has been proposed as an advice system for communicating with a person through dialogue. In the system disclosed in Patent Document 1, the degree of concentration of participants (presenters, students) in presentations, meetings, workshops, etc. is detected, and feedback is given to the participants. For example, when a less concentrated student is in the back seat, a message such as “Please speak louder” is displayed to the presenter.

JP 2012-160173 A

  Here, during communication, people give a signal indicating their true feelings (feelings) by gestures, tone of voice, loudness, whispering, etc. In the method of advancing the story, it is assumed that the other party's emotion is grasped based on such a signal issued by the other party.

  However, it is difficult for some people to detect the other party's signal, or even if you can grasp the signal, you do not know what to do to get closer to your ideal communication state Is also envisaged.

  Therefore, the present disclosure proposes an information processing system, a control method, and a storage medium capable of generating advice for constructing an ideal communication state based on a signal indicating a user's emotion.

  According to the present disclosure, a usage setting unit that sets a usage in which the first user communicates with the second user, a communication unit that receives an emotion signal of at least one of the first user and the second user, Based on an ideal state corresponding to the use of the received emotion signal, a generation unit that generates advice to the first user, and the generated advice is sent to the first user via the communication unit. And an information processing system including a control unit that transmits to a client terminal corresponding to the above.

  According to the present disclosure, the selection input unit that receives a selection of a usage in which the first user communicates with the second user, and the emotion signal detected from at least one of the first user and the second user, Based on the ideal state corresponding to the application selected by the selection input unit, the selected application is transmitted to an external server that generates advice to the first user, and is generated according to the application and the emotion signal. In addition, an information processing system including a communication unit that receives advice to the first user and a notification unit that notifies advice to the first user is proposed.

  According to the present disclosure, the first user sets an application for communicating with the second user, the emotion signal of at least one of the first user and the second user is received by the communication unit, Based on an ideal state according to the use of the received emotion signal, generating an advice to the first user, and sending the generated advice to the first user via the communication unit A control method is proposed, including controlling by a control unit to transmit to a corresponding client terminal.

  According to the present disclosure, the selection input unit that receives the selection of the usage in which the first user communicates with the second user, and the emotion signal detected from at least one of the first user and the second user. Based on an ideal state corresponding to the use selected by the selection input unit, the selected use is transmitted to an external server that generates advice to the first user, and according to the use and emotion signal. A storage medium storing a program for functioning as a communication unit that receives the generated advice to the first user and a notification unit that notifies the advice to the first user is proposed.

  As described above, according to the present disclosure, it is possible to generate advice for constructing an ideal communication state based on a signal indicating a user's emotion.

  Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is a figure explaining an outline of an information processing system by one embodiment of this indication. It is a figure showing the whole information processing system composition by this embodiment. It is a block diagram which shows an example of a structure of the sensor apparatus by this embodiment. It is a block diagram which shows an example of a structure of the emotion signal server by this embodiment. It is a block diagram which shows an example of a structure of the advice production | generation server by this embodiment. It is a block diagram which shows an example of a structure of the user terminal by this embodiment. It is a flowchart which shows the operation processing of the information processing system by this embodiment. It is a figure which shows the example of an initial screen containing the use selection screen by this embodiment. It is a flowchart which shows the advice production | generation process by this embodiment. It is a figure which shows an example of the advice set stored in advice information DB by this embodiment. It is a figure explaining the advice production | generation of the "influence" in the case of the use "negotiation" by this embodiment. It is a figure which shows an example of the display screen of the use "listening" by this embodiment. It is a figure which shows an example of the display screen of the use "negotiation" by this embodiment. It is a flowchart shown about the preliminary | backup process of the advice production | generation process in case only the user A by the modification 1 of this embodiment has a sensor apparatus. It is a figure which shows the example of a display screen by the modification 2 of this embodiment. It is a figure which shows the example of a display screen by the modification 3 of this embodiment. It is a figure which shows the other example of a display screen by the modification 3 of this embodiment. It is a figure which shows the other example of a display screen by the modification 3 of this embodiment.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. Overview of information processing system according to an embodiment of the present disclosure Configuration 2-1. Overall configuration 2-2. Sensor device 2-3. Emotion signal server 2-4. Advice generation server 2-5. 2. User terminal Operation processing 3-1. System operation processing 3-2. 3. Advice generation process 4. Display screen example Modified example 5-1. When only the user A (person) has the sensor device 10 5-2. When only the user B (partner) has the sensor device 10 5-3. Emotional signal scramble processing Summary

<< 1. Overview of Information Processing System According to One Embodiment of Present Disclosure >>
First, an overview of an information processing system according to an embodiment of the present disclosure will be described with reference to FIG. As shown in FIG. 1, when performing social communication such as negotiation or business negotiation, it is assumed that a person expresses an emotion signal indicating his / her emotion. Emotional signals can be quantified by, for example, four social signals: “influence”, “mimicry”, “activity level”, “consistency”.

  The “influence” is calculated from the pattern of changing the speaker during the conversation. For example, when the user A speaks unilaterally or the user A keeps talking while the user B is blocked, it can be said that the influence of the user A is great. Also, if both words are spoken so as to cover the other party's words, both influences are great. Also, if the conversation has long and irregular interruptions, it can be said that its influence is small.

  “Mimikuri” is calculated based on the number of times and time the user imitates the other person's words, gestures and gestures during the conversation. Usually, if you are crazy about conversation and are in the right wavelength or sympathize with each other, you tend to imitate each other's gestures, hand gestures, and words. It can be said that they are in sync with each other.

  The “activity amount” is calculated from the amount of gestures and the intensity of body movement. It can be seen that the greater the “activity” value, the more interested and interested in the other person's story.

  “Consistency” is calculated from the variation of the prosody of the utterance and the gesture gesture, and the variation of the pitch and loudness of the voice while speaking and the rhythm. It can be seen that the greater the “consistency” value, the more concentrated the spirit is, and the smaller the value, the more emotional.

  Such an emotion signal can be calculated from sensor data detected by, for example, a sensor device (a camera sensor, a microphone, a gyro sensor, an acceleration sensor, a geomagnetic sensor, or the like) worn by the user.

  The system recognizes the state of communication based on these emotion signals, and the ideal communication state according to the purpose of communication, the social position of the user, the relationship of the interlocutor, etc., that is, the ideal emotion signal. Advise the user to make the status appear.

  For example, as illustrated in FIG. 1, when the user A is negotiating with the user B, the information processing system according to the present disclosure acquires an emotion signal detected from at least one of the user A and the user B, Recognize the state. Next, the information processing system compares the ideal communication state (that is, the emotion signal state) in the “negotiation” field with the current emotion signal, and gives advice for constructing ideal communication information from the user terminal 5. Present to user A. For example, when the value of “activity level” is lower than the ideal value, advice such as “Please speak with gestures” is presented. In addition, the advice to the user A is presented together with a radar chart indicating the values and ideal values of the detected user A's emotion signals (“influence”, “mimikuri”, “activity level”, and “consistency”). It may be broken.

  Thereby, the user A can adjust the talk tempo, the tone of the voice, the gesture gesture, the amount of speech, etc. according to the advice, and can bring the negotiation with the user B close to an ideal state.

  The overview of the information processing system according to the embodiment of the present disclosure has been described above. The information processing system according to the present embodiment can generate advice for constructing an ideal communication state based on a signal indicating the user's emotion.

<< 2. Basic configuration >>
<2-1. Overall configuration>
Next, the overall configuration of the information processing system according to the present embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating an overall configuration of the information processing system 1 according to the present embodiment. As shown in FIG. 2, the information processing system 1 according to the present embodiment includes a sensor device 10 (1 a, 1 b) that is attached to a user and detects data for calculating an emotion signal, and an emotion signal that calculates an emotion signal. The server 2 includes an advice generation server 3 that generates advice based on emotion signals, and a user terminal 5 that feeds back the generated advice to the user. The sensor device 10, the emotion signal server 2, the advice generation server 3, and the user terminal 5 can be connected via the network 4.

  In addition to the neck-hanging type as shown in FIG. 2, the sensor device 10 may have various forms such as a glasses type, a head-mounted type, a watch type, a bracelet type, and a badge type. The sensor device 10 is not limited to a device worn by the user, and may be a device possessed by the user, such as a smartphone or a mobile phone terminal. The sensor device 10 may be integrated with the user terminal 5. The sensor device 10 detects a user's utterance, loudness, amount of physical activity, whispering, competing, etc. using a camera sensor, a microphone, a gyro sensor, an acceleration sensor, a geomagnetic sensor, and the like, and the detected data (sensor value) Is transmitted to the emotion signal server 2 via the network 4. The transmission to the emotion signal server 2 may be performed via a mobile terminal such as a smartphone, a mobile phone terminal, or a tablet terminal that the user has.

  The emotion signal server 2 analyzes data (sensor value) detected by the sensor device 10 and calculates a user's emotion signal (for example, influence, mimicry, activity level, and consistency). The calculated emotion signal can be stored in the emotion signal DB 22. The emotion signal DB 12 also stores ideal emotion signal values according to the use of this system (for example, types of social communication such as “negotiation”, “listening”, “collaboration”, “guidance”). .

  The advice generation server 3 receives a selection of a use from the user, and performs advice based on the emotion signal value to construct an ideal communication according to the use. Specifically, the advice generation server 3 calls an ideal emotion signal value for the selected application from the emotion signal DB 22 of the emotion signal server 2 and is calculated by the ideal emotion signal value and the emotion signal server 2. Compared with the current emotion signal value, advice is generated from the result, and feedback is made to the user. The advice based on the comparison result may be generated based on, for example, a case (rule) stored in the advice information DB 32, or may be generated using a recommendation technique such as machine learning or collaborative filtering.

  The user terminal 5 is an information processing apparatus that presents a result to the user. The user terminal 5 notifies the user of advice based on the comparison result transmitted from the advice generation server 3 by display output, audio output, vibration, or the like.

  The overall configuration of the information processing system 1 according to the present embodiment has been described above. Next, the basic configuration of each device included in the information processing system 1 of the present embodiment will be specifically described with reference to FIGS.

<2-2. Sensor device>
FIG. 3 is a block diagram illustrating an example of the configuration of the sensor device 10 according to the present embodiment. As shown in FIG. 3, the sensor device 10 according to the present embodiment includes a sensor unit 11, a sensor data extraction unit 12, a control unit 13, a communication unit 14, and a storage unit 15.

  The sensor unit 11 has a function of detecting a user's movement and voice, and can be realized by, for example, a camera sensor, a microphone, a gyro sensor, an acceleration sensor, a geomagnetic sensor, and the like.

  The sensor data extraction unit 12 extracts sensor data (captured image, audio information, etc.) from the sensor unit 11.

  The control unit 13 controls each component of the sensor device 10. The control unit 13 is realized by a microcomputer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and a nonvolatile memory. The control unit 13 according to the present embodiment controls the sensor data extracted by the sensor data extraction unit 12 to be temporarily stored in the storage unit 15 and transmitted from the communication unit 14 to the emotion signal server 2.

  The communication unit 14 functions as an I / F (interface) that transmits and receives data to and from an external device in accordance with instructions from the control unit 13. For example, the communication unit 14 connects to the network 4 and transmits the sensor data to the emotion signal server 2 on the network (cloud) in real time. The communication unit 14 is connected to the user terminal 5 possessed by the user via a short-distance wireless connection via Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark) or the like, and the emotion signal server 2 via the user terminal 5. Alternatively, the sensor data may be transmitted in real time.

  The storage unit 15 stores a program for the control unit 13 to execute each process. The storage unit 15 also has an area for temporarily storing sensor data.

<2-3. Emotional signal server>
Next, the configuration of the emotion signal server 2 will be described with reference to FIG. FIG. 4 is a block diagram illustrating an example of the configuration of the emotion signal server 2 according to the present embodiment. As shown in FIG. 4, the emotion signal server 2 includes a control unit 20, a communication unit 21, an emotion signal DB 22, and a user information DB 23.

  The control unit 20 controls each component of the emotion signal server 2. The control unit 20 is realized by a microcomputer that includes a CPU, a ROM, a RAM, and a nonvolatile memory. As shown in FIG. 4, the control unit 20 according to this embodiment also functions as a data processing unit 20a, a data analysis unit 20b, an emotion signal calculation unit 20c, and a user management unit 20d.

  The data processing unit 20a prepares the sensor data detected by the sensor device 10 received by the communication unit 21, processes the data into a state that can be analyzed more accurately, and outputs the data to the data analysis unit 20b. The data analysis unit 20b analyzes the sensor data arranged by the data processing unit 20a, extracts information such as speech, voice volume, amount of physical activity, gesture gesture, whispering, companion, etc., and an emotion signal calculation unit To 20c.

  The emotion signal calculation unit 20c calculates a user's emotion signal based on information obtained by data analysis. More specifically, the emotion signal calculation unit 20c calculates four values such as influence, mimicry, activity level, and consistency as emotion signals. In the present embodiment, as an example, the values of these emotion signals are calculated as normalized numerical values between 0 and 1. Hereinafter, calculation of four emotion signals will be described in detail.

Influence The emotion signal calculation unit 20c, for example, the frequency of speech, the length of speech, the strength of voice, and the overlap of speech obtained by analyzing voice data of conversation between the user A and the user B collected by the microphone ( "Impact power" is calculated based on the frequency at which speech is interrupted and the like. For example, the emotion signal calculation unit 20c determines that the influence of the user A is small when the frequency of the speech of the user A is less than that of the user B. The emotion signal calculation unit 20c also determines the length of the speech in the same manner (determines that the “influence” is greater as the length of the speech is longer).

  In addition, the emotion signal calculation unit 20c compares the average levels of the voice strengths of the user A and the user B, and determines that the influence of the user A is small when the average level of the voice strength of the user B is larger. . The emotion signal calculation unit 20c determines that the influence of the interrupted user is small when the frequency of interrupting the speech is high when the speech is duplicated.

The mimicry emotion signal calculation unit 20c agrees with the other party obtained by analyzing the movement of the user's head detected by, for example, a camera, an infrared sensor, an acceleration sensor, a gyro sensor, or the utterance collected by the microphone. "Mimikuri" is calculated on the basis of the action (whispering, reconciliation, repetition of the same word, etc.) indicating. For example, the emotion signal calculation unit 20c determines that the mimicry is high when the same word is acquired in the conversation (appearance establishment is high). In addition, the appearance probability of a specific word (such as “I see”, “Is it true”, “Yes”, “Yes”, etc.) indicating a conflict is measured to determine mimicry.

Activity level The emotion signal calculation unit 20c is based on the gesture gesture motion obtained by analyzing the movements of the user's hands and upper body detected by the camera, infrared sensor, acceleration sensor, gyro sensor, etc. Calculate “activity level”. The number of gestures of the user and the movement of the upper body increase when the user is enthusiastic or eager to talk. The emotion signal calculation unit 20c determines the activity level by quantifying the number of gestures and the size of the upper body movement.

-Consistency The emotion signal calculation unit 20c determines, for example, the frequency of speech, the length of speech, the strength of voice, the frequency of speech, the speed of speech obtained by analyzing the speech collected by the microsensor. Based on this, “consistency” is calculated. For example, the emotion signal calculation unit 20c measures changes in voice strength and voice pitch (frequency) during conversation in a time series, and based on the deviation and appearance probability, whether the utterance is consistent or modulation occurs. Measure consistency and calculate consistency (high consistency when consistent, low consistency when modulated).

  The emotion signal calculated as described above is stored in the emotion signal DB 22 and is transmitted to the advice generation server 3 via the communication unit 21.

  The user management unit 20d manages user information stored in the user information DB 23. For example, the user management unit 20d performs processing such as registration, deletion, and update of user information in the user information DB 23. User information includes, for example, the user's name, age, gender, hobbies / preferences, user ID, and speaking tendency (average speech frequency, voice strength, voice frequency, speaking speed, frequency of gestures, etc.) including.

  The communication unit 21 functions as an I / F (interface) that transmits / receives data to / from an external device in accordance with an instruction from the control unit 20. For example, the communication unit 21 is connected to the network 4 and receives sensor data sensed during conversation from the sensor device 10 or transmits a calculated emotion signal to the advice generation server 3. The communication unit 21 returns an ideal emotion signal for a predetermined use extracted from the emotion signal DB 22 in response to a request from the advice generation server 3.

  The emotion signal DB 22 stores the emotion signal calculated by the emotion signal calculation unit 20c. Further, the emotion signal DB 22 may store ideal emotion signals for different uses such as “negotiation”, “listening”, “collaboration”, and “guidance”. The ideal emotion signal is, for example, an ideal value obtained by a machine learning method or a statistical method using many cases in the application as teacher data. The ideal emotion signal may be input to the administrator as a desirable numerical value for each application.

  The user information DB 23 stores information (user information) related to users who are users of this system.

<2-4. Advice generation server>
Next, the configuration of the advice generation server 3 according to the present embodiment will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of the configuration of the advice generation server 3 according to the present embodiment. As illustrated in FIG. 5, the advice generation server 3 includes a control unit 30, a communication unit 31, and an advice information DB 32.

  The control unit 30 controls each configuration of the advice generation server 3. The control unit 30 is realized by a microcomputer that includes a CPU, a ROM, a RAM, and a nonvolatile memory. As shown in FIG. 5, the control unit 30 according to the present embodiment also functions as an application setting unit 30a, a comparison unit 30b, and an advice generation unit 30c.

  The usage setting unit 30a sets the usage of advice generation. For example, the usage setting unit 30 a may set the usage selected by the user on the usage selection screen displayed on the user terminal 5. In this specification, the use of advice generation specifically refers to four categories assumed in social communication between people (or people and robots): “negotiation”, “listening”, “collaboration”, And “instruction” is assumed. “Negotiation” corresponds to situations where, for example, a request for transfer to the boss or improvement of treatment within the company is required, or a situation where a contract is concluded with an outside sales or business partner. Applicable to customers who have complained (complaint response scene). In addition, “collaboration” corresponds to, for example, a case of consulting with a partner who performs together on how to proceed with work or duty, and “instruction” corresponds to, for example, a case of educating a subordinate who repeats lateness.

  The user selects which of the above four categories the communication to be performed on the use selection screen described later. In the middle of a conversation, it is assumed that the type of communication changes dynamically depending on the development of the story. Therefore, the user may select a use each time.

  The comparison unit 30b compares the user's current emotion signal calculated by the emotion signal server 2 with an ideal emotion signal according to the use set by the use setting unit 30a. The ideal emotion signal according to the application may be acquired from the emotion signal DB 22 of the emotion signal server 2, for example, or may be stored in the internal memory of the advice generation server 3. Further, the comparison unit 30b outputs the comparison result to the advice generation unit 30c.

  Based on the comparison result, the advice generation unit 30c refers to the advice information DB 32 and generates an advice to the user so as to approach the ideal state. The advice information DB 32 stores an advice set for each use, which will be described later with reference to FIG. Each advice set depends on the combination of the value of user A (system use: self) and the value of user B (partner) in each emotional signal, that is, “influence”, “mimikuri”, “activity level” and “consistency” Including advice.

  The communication unit 31 transmits / receives data to / from an external device according to an instruction from the control unit 30. For example, the communication unit 31 receives an emotion signal of the user calculated by the emotion signal server 2 or an ideal emotion signal for the set use from the emotion signal server 2, or receives the advice generated by the advice generation unit 30c. Or transmitted to the user terminal 5. Further, the communication unit 31 may transmit the emotion signal of the target user and an ideal emotion signal for the set use together with the advice.

  The advice information DB 32 stores advice information used when the advice generation unit 30c generates advice. The advice information may be an advice set for each purpose-specific emotion signal type, which will be described later with reference to FIG. Such an advice set is an advice rule generated in advance based on a case.

  The configuration of the advice generation server 3 according to the present embodiment has been described above. 2, 4, and 5, the advice generation server 3 and the emotion signal server 2 have been described as separate bodies. However, the present embodiment is not limited to this, and the advice generation server 3 and the emotion signal server 2 are one. May be realized by the same server.

<2-5. User terminal>
Next, the configuration of the user terminal 5 according to the present embodiment will be described with reference to FIG. FIG. 6 is a block diagram illustrating an example of the configuration of the user terminal 5 according to the present embodiment. As illustrated in FIG. 6, the user terminal 5 includes a control unit 50, a communication unit 51, an operation input unit 52, a display unit 53, and a storage unit 54. The user terminal 5 may be realized by a notebook PC (personal computer) as shown in FIG. 1, or may be realized by a tablet terminal, a smartphone, a mobile phone terminal, a desktop PC, or the like. Alternatively, the user terminal 5 may be realized by a wearable device such as a glasses-type device, a watch-type device, or a neck device.

  The control unit 50 controls each configuration of the user terminal 5. Further, the control unit 50 is realized by a microcomputer including a CPU, a ROM, a RAM, and a nonvolatile memory. The control unit 50 according to the present embodiment displays an application selection screen on the display unit 53 or transmits an application selection input from the operation input unit 52 from the communication unit 51 to the advice generation server 3 to give advice according to the application. Control to request generation. In addition, the control unit 50 controls the display unit 53 to display a display screen including the advice information received from the advice generation server 3 and the current emotion signal value or ideal signal value, and notify the user.

  The communication unit 51 transmits / receives data to / from an external device in accordance with an instruction from the control unit 50. For example, the communication unit 51 transmits use selection information to the advice generation server 3 on the network, or receives advice information from the advice generation server 3. The communication unit 51 may have a function of transmitting and receiving data by connecting to an external device present in the vicinity. Specifically, the communication unit 51 can also receive sensor data from the sensor device 10 worn by the user and transmit it to the emotion signal server 2 on the network.

  The operation input unit 52 has a function of detecting a user operation on the user terminal 5 and outputting it to the control unit 50 as operation input information. The operation input unit 52 is realized by, for example, a touch panel, and can be integrated with the display unit 53 to detect a user operation on the display screen. Further, the operation input unit 52 may analyze the sound collected by the microphone to realize voice input by the user, or analyze the captured image captured by the camera to realize gesture input by the user. Also good.

  The display unit 53 has a function of displaying various screens such as a menu screen, an operation screen, a usage selection screen, and an advice notification screen, and is realized by, for example, a liquid crystal display. When the user terminal 5 is realized by a glasses-type device, the display unit 53 corresponds to a lens portion located in front of the user's eyes when worn, and is a lens unit that performs display using, for example, hologram optical technology. Also good. Specifics of the display screen including the usage selection screen and the advice notification screen to be displayed will be described later with reference to FIGS.

  The storage unit 54 stores a program for the control unit 50 to execute each process. Further, the storage unit 54 according to the present embodiment may temporarily store the advice information received from the advice generation server 3.

  The main configuration of the user terminal 5 has been described above. Note that the configuration of the user terminal 5 is not limited to the example illustrated in FIG. 6, and further includes, for example, a sensor unit (camera, microphone, infrared sensor, acceleration sensor, gyro sensor, etc.) and a sensor data extraction unit. You may have. In the configuration shown in FIG. 6, the display unit 53 is used as an example of the output unit. However, the present embodiment is not limited to this. For example, the user terminal 5 includes a speaker or a vibration unit and outputs advice information. You may perform by audio | voice output or vibration output.

<< 3. Action processing >>
<3-1. System operation processing>
Next, operation processing of the information processing system according to the present embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing an operation process of the information processing system according to the present embodiment.

  As shown in FIG. 7, the user terminal 5 displays an opening message and a usage input screen (steps S103 and S106). Specifically, the user terminal 5 displays a usage selection screen on the display unit 53 together with an opening message such as “Please select a usage”. Here, FIG. 8 shows an example of an initial screen including a use selection screen according to the present embodiment. As shown in FIG. 8, for example, a usage selection screen 113 is displayed on the upper right of the initial screen 100. On the usage selection screen 113, four usages such as negotiation, listening, collaboration, and guidance can be selected. In the example illustrated in FIG. 8, “listening” is selected as an example. In addition, an advice display screen 114 is displayed at the lower right of the initial screen 100, and a graph screen 111 on which the value of each emotion signal is displayed is displayed on the left side of the initial screen 100. Here, since the use is still in the selected stage, the advice display screen 114 is blank, and the emotion signal value graph is not displayed. The details of the scramble ON / OFF button 112 will be described in Modification 3 described later, but in this embodiment, it is always “OFF”. The display of the initial screen 100 can be performed based on the display information received from the advice generation server 3.

  Next, when a use is selected (input) on the use selection screen 113 (step S109 / Yes), an emotion signal acquisition process for the user is performed (step S112). Specifically, when the usage is selected, the advice generation server 3 selects the emotion signal (specifically, influence, mimicry) calculated based on the user's sensing data acquired from the sensor device 10 by the emotion signal server 2. , Activity level, consistency value) from the emotion signal server 2 in substantially real time. In this embodiment, as an example, user A and user B communicating with each other wear sensor devices 10a and 10b (see FIG. 2), and user A's emotion signal based on user A's sensing data and user B's The emotion signal of the user B based on the sensing data can be acquired.

  The acquisition of the emotion signal (step S112) is repeated until a significant emotion signal can be acquired (step S115). A significant emotion signal indicates a value of an emotion signal that can be generated by performing comparison processing with an ideal emotion signal.

  Subsequently, when a significant emotion signal can be acquired (step S115 / Yes), the advice generation server 3 generates advice for constructing an ideal state according to the use based on the emotion signal (step S118). ). The specific processing content of the advice generation processing will be described later with reference to FIG. The advice generation server 3 transmits the generated advice information to the user terminal 5.

  Then, the user terminal 5 displays the advice information received from the advice generation server 3 and notifies the user (step S121). Specifically, the user terminal 5 displays advice information for approaching an ideal state corresponding to the selected application on the advice display screen 114 shown in FIG. In addition, the graph screen 111 displays the current user emotion signal values and ideal emotion signal values received from the advice generation server 3 together with the advice information, and presents the current state and ideal state to the user. It can be intuitively grasped. In the case of only the graph screen 111, it is assumed that it is impossible to determine how to respond to the ideal state during communication, and therefore, it is displayed together with specific advice. Specific display examples of advice information and the like will be described later with reference to FIGS.

<3-2. Advice generation processing>
Next, the advice generation processing by the advice generation server 3 shown in S118 will be specifically described with reference to FIG. FIG. 9 is a flowchart showing the advice generation process according to the present embodiment.

  As shown in FIG. 9, first, the advice generation server 3 receives the “negotiation” advice set from the advice information DB 32 when the use set based on the user's selection is “negotiation” (step S133 / Yes). Extraction is performed, and advice on each emotion signal (specifically, influence, mimicry, activity level, and consistency) is acquired using the advice set (step S136). Here, FIG. 10 shows an example of an advice set stored in the advice information DB 32.

  As shown in FIG. 10, the advice information DB 32 stores an advice set 60 for use “negotiation”, an advice set 61 for “listening”, an advice set 62 for “collaboration”, and an advice set 63 for “guidance”. Each advice set includes “influence” advice information 60a, “Mimikuri” advice information 60b, “activity level” advice information 60c, and “consistency” advice information 60d. Each piece of advice information is normalized emotion signal values (influence, mimicry, activity level, consistency) of user A (referred to as “self” in FIG. 10) and user B (referred to as “partner” in FIG. 10). It is linked to the coordinates corresponding to the combination. Here, as an example, a description will be given using a matrix 600 in which advice is stored at each coordinate.

  When the usage is set to “negotiation” by the usage setting unit 30a, the advice generation unit 30c extracts the “negotiation” advice set 60, and first generates “influence” advice. Hereinafter, the “influence” advice generation processing will be described in detail with reference to FIG. In this embodiment, the “influence” of the user in communication is, for example, the frequency of speech, the length of speech, the strength of voice, the overlap of speech, the frequency at which speech is interrupted when the speech overlaps, etc. Based on the above, a normalized numerical value between 0 and 1 is calculated.

  FIG. 11 is a diagram for explaining advice generation of “influence” in the case of the usage “negotiation” according to the present embodiment. When the usage is “negotiation”, the ideal state of “influence” is considered to have the same influence on each other. This is because it is not preferable that one of the persons has a great influence in the negotiation. Therefore, the ideal combination of influences of the user A (self) and the user B (partner) is a portion indicated mainly by a region 601 where the influences of both are balanced. The ideal value is based on data received from the emotion signal server 2 as an “ideal emotion signal”, for example.

  However, as shown in FIG. 11, for example, when the influence (X) of the user A (self) is 0.75 and the influence (Y) of the user B (partner) is 0.25, the combination of both influences is the P coordinate. It will be out of the ideal state. In this case, since the influence of the other party cannot be forcibly changed by the user A, the user A moves closer to an ideal state where both influences are balanced by acting so as to reduce their influence. Can do. From the P-coordinate, for example, advice that instructs an action to reduce its influence is acquired, such as “Please let me speak freely without interrupting what the other party says”. In this way, the advice generation unit 30c refers to the advice information based on the current “influence” values of the user A and the user B and the ideal value, and the user A for approaching the ideal state. Advice can be generated.

  The advice generation unit 30c similarly performs the above-described advice acquisition processing for “Mimikuri”, “Activity Level”, and “Consistency”, and acquires respective advices.

  Next, returning to FIG. 9, if the usage set based on the user's selection is “listening” (step S139 / Yes), the advice generation server 3 extracts the “listening” advice set from the advice information DB 32. Similarly, advice on each emotion signal is acquired using the advice set (step S142).

  Next, when the usage set based on the user's selection is “collaboration” (step S145 / Yes), the advice generation server 3 extracts the advice set of “collaboration” from the advice information DB 32, and the advice set Similarly, advice on each emotion signal is acquired (step S148).

  Subsequently, when the usage set based on the user's selection is “guidance” (step S151 / Yes), the advice generation server 3 extracts the “guidance” advice set from the advice information DB 32, and the advice set Similarly, advice on each emotion signal is acquired using (Step S154).

  And if the advice production | generation server 3 can acquire the advice according to the selected use, in addition to advice, for example, user A's current emotion signal value (current value) and the ideal value according to the selected use A display is generated (step S157).

  The advice generation process according to the present embodiment has been described in detail above.

<< 4. Display screen example >>
Next, an example of a display screen including advice and the current value / ideal value of each emotion signal according to the present embodiment will be described with reference to FIGS.

(Display example 1)
FIG. 12 is a diagram showing an example of the display screen 101 of the application “listening” according to the present embodiment. The display screen 101 is an example of a display screen displayed on the display unit 53 of the user terminal 5 in the display process shown in step S121 shown in FIG. As shown in FIG. 12, the display screen 101 includes a graph screen 111a, a usage selection screen 113a, and an advice notification screen 114a. In the example shown in FIG. 12, “listening” is selected as shown in the application selection screen 113a.

  On the graph screen 111a, the current value of each emotion signal of the user A and the ideal value of each emotion signal in the usage “listening” are shown in a radar chart. By viewing the graph screen 111a, the user A can intuitively grasp that his / her influence is greater than the ideal state, and his / her consistency, activity level, and mimicry are smaller than the ideal state. it can. In the case of “listening”, for example, the state where the influence on the user A side is reduced to talk to the other party (user B) and the person is sympathizing with the other party is ideal. The ideal value of Mimikuri is small and large.

  The advice notification screen 114a displays advice information indicating how the user A should take action in order to approach the ideal state from the current state. In the advice information, advice corresponding to each emotion signal value, that is, advice regarding “influence”, advice regarding “Mimikuri”, advice regarding “activity level”, and advice regarding “consistency” are specified. Here, the advice generation server 3 can also obtain advice using an advice rule as shown in Table 1 below, for example. Such advice rules are provided for each use and are stored in the advice information DB 32.

  As a result, the advice generation server 3 determines whether the current value of the emotion signal of the user A (or the combination of the emotion signal values of the user A and the user B; see the P coordinate shown in FIG. 11) is larger or smaller than the ideal value. Appropriate advice can be obtained from the above advice rules.

(Display example 2)
FIG. 13 is a diagram showing an example of the display screen 102 for the usage “negotiation” according to the present embodiment. As shown in FIG. 13, the display screen 102 includes a graph screen 111b, a usage selection screen 113b, and an advice notification screen 114b. In the example shown in FIG. 13, “negotiation” is selected as shown in the use selection screen 113b.

  On the graph screen 111b, the current value of each emotion signal of the user A and the ideal value of each emotion signal in the use “negotiation” are shown in a radar chart. By viewing the graph screen 111b, the user A can intuitively grasp that his / her mimicry is larger than the ideal state and his influence, consistency, and activity level are smaller than the ideal state. it can. In the case of “negotiation”, the other party is often in a strong position. Therefore, the user A listens to the other party's thoughts without interrupting the other party's story, and the user A's attitude is made consistent. The ideal value of influence is small and the ideal value of consistency is large.

  Further, on the advice notification screen 114b, advice information indicating how the user A should perform in order to approach the ideal state from the current state is displayed. The advice information according to the present embodiment may be generated according to an advice rule as shown in Table 2 below, for example.

<< 5. Modification >>
In the embodiment described above, the case where both the user A and the user B have the sensor device 10 and can acquire emotion signals of both users has been described. However, the present embodiment is not limited to this, and either one of them can be obtained. Only the user may have the sensor device 10. Hereinafter, a case where only the user A who is the system user has the sensor device 10 and a case where only the user B who is the communication partner has the sensor device 10 will be described.

<5-1. When only the user A (person) has the sensor device 10>
The information processing system according to the first modification of the present embodiment can generate advice for approaching an ideal state based on the emotion signal of only the user A. Specifically, in the information processing system according to this modification, the partner (user B) does not have the sensor device 10 and is in a state where emotion signals cannot be acquired, so the default value for each application is used to generate advice. To do. The default value is calculated based on, for example, the opponent's past influence, mimicry, activity level, and consistency values for each application (for example, the highest appearance rate). Hereinafter, a specific description will be given with reference to FIG.

  FIG. 14 is a flowchart illustrating pre-processing of advice generation processing when only the user A has the sensor device 10. The process shown in FIG. 14 is performed before the advice generation process shown in FIG. 9 when an emotion signal from the partner (user B) cannot be acquired (step S201 / No).

  In this case, when the set use is “negotiation” (step S203 / Yes), the advice generation server 3 extracts an advice set of “negotiation”, and influence, mimicry, activity level, and consistency in “negotiation”. Each sex default value is set as the value of the other party (step S206).

  Further, when the set usage is “listening” (Yes in step S209), the advice generation server 3 extracts an advice set of “listening”, and influence, mimicry, activity level, and consistency in “listening” Each default value is set as the value of the other party (step S212).

  Further, when the set use is “collaboration” (step S215 / Yes), the advice generation server 3 extracts an advice set of “collaboration”, influence, mimicry, activity level, and consistency in “collaboration”. Each default value is set as the value of the other party (step S218).

  In addition, when the set use is “guidance” (step S221 / Yes), the advice generation server 3 extracts an advice set of “guidance”, and influence, mimicry, activity level, and consistency in “guidance”. Each default value is set as the value of the other party (step S224).

  As described above, when the information processing system according to the first modification generates an advice with reference to the advice set as illustrated in FIG. 10, when the emotion signal of the partner (user B) cannot be acquired, It is possible to generate advice by adopting the Default value.

<5-2. When only user B (partner) has the sensor device 10>
Further, the information processing system according to the second modification of the present embodiment can generate advice for approaching an ideal state based on the emotion signal of only the communication partner (user B). Specifically, the information processing system according to this modification is in a state in which the person (user A) does not have the sensor device 10 and cannot acquire an emotion signal, and therefore the other person's emotion signal is monitored, Compare with the standard value (normal value) of to generate advice. Hereinafter, a specific description will be given with reference to FIG.

  FIG. 15 is a diagram illustrating a display screen example according to the second modification of the present embodiment. As shown in FIG. 15, the display screen 103 includes a graph screen 111c, a usage selection screen 113c, and an advice notification screen 114c. Here, on the graph screen 111c according to this modification, the value (current value) of the other party's emotion signal and the normal value of the other party are shown in a radar chart. The advice generation server 3 compares the other person's emotion signal value with the other party's normal value, and generates advice based on, for example, an advice rule as shown in Table 3 below. Such advice rules exist for each use. In Table 3 below, as shown in the graph screen 111c of FIG. 15, the current “influence” of the opponent is smaller than the normal value, “Mimikuri” is larger than the normal value, and “activity level” is the normal value. And “consistency” is less than normal.

  As described above, the information processing system according to the second modification can generate advice based on the emotion signal of the other party (user B) and the normal value of the other party.

<5-3. Emotional signal scramble processing>
Then, the information processing system by the modification 3 of this embodiment is demonstrated. In the above-described embodiment and each modification, advice for constructing a more ideal communication state is generated. However, the present embodiment is not limited to this, for example, for constructing a non-ideal communication state. It is also possible to generate advice. For example, when doing business with a partner in a business environment, express an emotional signal that is different from the original intention by explicitly showing an attitude that is contrary to your own thought so that this thought can not be read by the partner. It may be useful to do so. In the third modification of the present embodiment, an emotion signal scramble process for generating an advice for scrambling the original emotion signal so that it cannot be read by the other party is proposed. Hereinafter, a specific description will be given with reference to FIG.

  FIG. 16 is a diagram illustrating a display screen example according to the third modification of the present embodiment. As shown in FIG. 16, the display screen 104 includes a graph screen 111d, a usage selection screen 113d, and an advice notification screen 114d. In the example shown in FIG. 16, “listen” is selected as shown in the application selection screen 113d.

  In addition, when user A wants to get advice to leave the ideal communication state so that his / her own emotional signal is not read by the other party, for example, when dealing with the other party in a business situation, the scramble ON The / OFF button 112d is set to “ON”. Thereby, for example, a display for prompting the user to leave the ideal state is displayed on the graph screen 111d, and an advice for leaving the ideal state is displayed on the advice notification screen 114d.

  Specifically, as shown in the graph screen 111d of FIG. 16, the value of the emotion signal of the user A (the system user) (“your current value” shown in FIG. 16) and the ideal value for the usage “listening” Are displayed on the axis of each emotion signal (on the 4th axis). As a result, the user A objectively grasps what emotion signal is currently being expressed from the user to the other party, and leaves the ideal state so that the other person cannot read the real emotion signal. Can intuitively recognize whether each emotion signal should be strengthened or weakened.

  For example, since the current value of “influence” is slightly larger than the ideal value, an arrow that prompts the influence to be further increased is displayed so as to further move away from the ideal state. Also, “Mimikuri”, “Activity level”, and “Consistency” are “Mimikuri”, “Activity level” and “ Arrows prompting you to weaken each "consistency" are displayed.

  Further, advice for leaving the ideal state displayed on the advice notification screen 114d is generated by the advice generation server 3 based on, for example, advice rules for each application as shown in Table 4 below.

(Other display screen examples during emotion scramble)
Further, the display screen example according to the present modification is not limited to the example shown in FIG. For example, in this modification, a state away from the ideal state may be indicated as the scrambled ideal value. Hereinafter, a description will be given with reference to FIG.

  FIG. 17 is a diagram showing another display screen example according to this modification. As shown in FIG. 17, the display screen 105 includes a graph screen 111e, a usage selection screen 113e, and an advice notification screen 114e. In the graph screen 111e of FIG. 17, the value of the emotion signal of the user A (the system user) (“your current value” shown in FIG. 17), the ideal value for the usage “listening”, and the ideal scramble value are shown. Radar chart is displayed. As a result, the user A objectively grasps what emotion signal is currently being expressed from the user to the other party, and leaves the ideal state so that the other person cannot read the real emotion signal. In addition, it is possible to intuitively recognize whether each emotion signal should be strengthened or weakened. Further, by displaying the scrambled ideal value together, the user A can intuitively grasp how far from the ideal state it should be.

(When only the other party has the sensor device 10)
Further, the present modification can also be applied to the case where only the user B who is the communication partner of the user A has the sensor device 10 and the user A does not have the sensor device 10. Hereinafter, a description will be given with reference to FIG.

  FIG. 18 is a diagram illustrating another display screen example according to the third modification of the present embodiment. As shown in FIG. 18, the display screen 106 includes a graph screen 111f, a usage selection screen 113f, and an advice notification screen 114f. In the graph screen 111f of FIG. 18, a radar chart indicating the value of the emotion signal of the user B (“current value of the other party” shown in FIG. 18) and the scrambled ideal value for the usage “negotiation” is displayed. As a result, the user A grasps what emotion signals are currently being expressed from the other party, and in order not to make “negotiation” communication in an ideal state, the other person's emotion signal is set to the scrambled ideal value. It is possible to intuitively recognize that it is necessary to approach. The advice notification screen 114f displays advice indicating what kind of response should be taken in order to make the other party's emotion signal the scrambled ideal value.

  Here, the advice displayed on the advice notification screen 114f is based on the emotion signal calculated by the advice generation server 3 based on the sensor data of the user B, for example, for each application as shown in Table 4 below. Generated with reference to a rule.

<< 6. Summary >>
As described above, in the information processing system according to the embodiment of the present disclosure, it is possible to generate advice for building an ideal communication state based on a signal indicating a user's emotion (that is, an emotion signal). .

  In addition, in the information processing system according to the present embodiment, even when only one of the emotion signal of the user himself / herself and the emotion signal of the communication partner can be acquired, advice for constructing an ideal communication state is provided. Can be generated.

  Also, in situations where it is not appropriate for the user's true emotional signal to be transmitted to the other party, it is possible to scramble the emotional signal that is intentionally expressed and generate advice to leave the ideal communication state. is there.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present technology is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

  For example, the sensor device 10, the emotion signal server 2, the advice generation server are installed in hardware such as the CPU, ROM, and RAM incorporated in the sensor device 10, the emotion signal server 2, the advice generation server 3, or the user terminal 5 described above. 3 or a computer program for exercising the function of the user terminal 5 can be created. A computer-readable storage medium storing the computer program is also provided.

  In the embodiment and the modification described above, advice or the like is mainly displayed and output from the display unit 53 of the user terminal 5, but it may be extremely difficult to view the display screen during communication. Therefore, it is conceivable that feedback such as presentation of advice to the user is realized by, for example, a head-mounted wearable device instead of the notebook PC as shown in FIG. The head-mounted wearable device has, for example, a plurality of vibration units, and four axes of emotion signals as shown in FIG. 8 are assigned to the vibration units arranged in the front, rear, left, and right, respectively. It is possible to inform the user of the situation to be taken and how to deal with it by the vibration of the vibration unit. For example, when the “influence” is not large enough, the front vibration part vibrates strongly, and when “Mimikuri” is not large enough, the right vibration part vibrates strongly and the “activity level” is large enough. If not, the rear vibration part vibrates strongly, and if the “consistency” is not large enough, the left vibration part vibrates strongly. The human head is sensitive and can distinguish the vibration of the vibrating part. Further, the head-mounted wearable device may be provided with, for example, a bone conduction speaker, and advice may be output as a voice. By outputting through the bone conduction speaker, the user A can obtain advice without looking at the display screen or being known to the communication partner.

  Further, the user terminal 5 that performs feedback by such vibration is not limited to a head-mounted wearable device. For example, a bracelet type (arm) having a function of vibrating in four directions (front, back, right, and left), respectively. A band) wearable device or a belt-type (trunk band) wearable device may be used.

  In the above-described embodiment, “negotiation”, “listening”, “collaboration”, and “guidance” are given as the usage selection. However, these are examples of usage, and the present disclosure is not limited to this. For example, other situations For example, “date” or “interview” may be used.

  Further, the effects described in the present specification are merely illustrative or exemplary and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

In addition, this technique can also take the following structures.
(1)
A usage setting unit for setting a usage in which the first user communicates with the second user;
A communication unit that receives an emotion signal of at least one of the first user and the second user;
Based on an ideal state according to the use of the received emotion signal, a generation unit that generates advice to the first user,
A control unit that transmits the generated advice to the client terminal corresponding to the first user via the communication unit;
An information processing system comprising:
(2)
The information processing system according to (1), wherein the generation unit generates advice to the first user so that an emotion signal of the first user approaches an ideal state according to the application.
(3)
The information processing unit according to (1) or (2), wherein the generation unit generates advice to the first user based on correspondence information between the state of the emotion signal and advice accumulated for each use. system.
(4)
The generation unit generates advice to the first user based on a comparison result of comparing the received emotion signal with an emotion signal in an ideal state according to the application. The information processing system according to any one of 3).
(5)
The generation unit is based on a comparison result of comparing a combination value of the current emotion signal of the first user and the current emotion signal of the second user with an emotion signal in an ideal state according to the application. The information processing system according to any one of (1) to (4), wherein advice for the first user to approach the ideal state is generated.
(6)
The generation unit is based on a comparison result of comparing a combination value of the current emotion signal of the first user and a default value of the emotion signal of the second user with an emotion signal in an ideal state according to the application. The information processing system according to any one of (1) to (4), wherein advice for the first user to approach the ideal state is generated.
(7)
The generating unit sets the normal value as an ideal state based on the current emotion signal of the second user and the normal value of the second user according to the application, and the second unit approaches the ideal state. The information processing system according to any one of (1) to (4), wherein advice for one user is generated.
(8)
The information processing system according to (1), wherein the generation unit generates advice to the first user so that the emotion signal of the first user is separated from an ideal state according to the application.
(9)
The information processing system according to (1), wherein the generation unit generates advice to the first user so that the emotion signal of the first user approaches a non-ideal state according to the application.
(10)
The control unit performs control so as to transmit the emotion signal of the first user or the second user used for generating the advice and the emotion signal in the ideal state together with the generated advice. The information processing system according to any one of (9) to (9).
(11)
A selection input unit that accepts selection of an application in which the first user communicates with the second user;
Providing advice to the first user based on an ideal state of the emotion signal detected from at least one of the first user and the second user according to the usage selected by the selection input unit A communication unit that transmits the selected usage to an external server that receives the advice to the first user generated according to the usage and emotion signal;
A notification unit for notifying advice to the first user;
An information processing system comprising:
(12)
The information processing system includes:
A detector that detects an emotion signal of at least one of the first user and the second user;
The information processing system according to (11), wherein the communication unit transmits the detected emotion signal to the external server.
(13)
The notification unit notifies the first user of advice so that the emotion signal of the first user approaches an ideal state according to the use selected by the selection input unit, (11) or ( The information processing system according to 12).
(14)
The notification unit notifies the first user of advice so that the emotion signal of the first user leaves an ideal state according to the use selected by the selection input unit, (11) to (11) The information processing system according to any one of 13).
(15)
The notification unit notifies advice to the first user so that the emotion signal of the first user approaches a non-ideal state according to the use selected by the selection input unit, The information processing system according to any one of (13).
(16)
The notification unit notifies the generated advice together with a graph indicating the emotion signal of the first user or the second user used for generating the advice and the emotion signal in the ideal state (11) Information processing system given in any 1 paragraph of-(15).
(17)
The information processing system according to any one of (11) to (15), wherein the notification unit notifies the first user by vibration using a vibration unit mounted on a head-mounted device.
(18)
Setting an application in which the first user communicates with the second user;
Receiving an emotion signal of at least one of the first user and the second user by a communication unit;
Generating an advice to the first user based on an ideal state according to the use of the received emotion signal;
Controlling the generated advice to be transmitted to the client terminal corresponding to the first user via the communication unit;
Including a control method.
(19)
Computer
A selection input unit that accepts selection of an application in which the first user communicates with the second user;
Providing advice to the first user based on an ideal state of the emotion signal detected from at least one of the first user and the second user according to the usage selected by the selection input unit A communication unit that transmits the selected usage to an external server that receives the advice to the first user generated according to the usage and emotion signal;
A notification unit for notifying advice to the first user;
A storage medium storing a program for functioning as a computer.

DESCRIPTION OF SYMBOLS 1 Information processing system 10 Sensor apparatus 11 Sensor part 12 Sensor data extraction part 13 Control part 14 Communication part 15 Storage part 2 Emotion signal server 20 Control part 20a Data processing part 20b Data analysis part 20c Emotion signal calculation part 20d User management part 21 Communication Part 22 Emotion Signal DB
23 User information DB
3 advice generation server 30 control unit 30a application setting unit 30b comparison unit 30c advice generation unit 31 communication unit 32 advice information DB
5 User terminal 50 Control unit 51 Communication unit 52 Operation input unit 53 Display unit 54 Storage unit

Claims (19)

A usage setting unit for setting a usage in which the first user communicates with the second user;
A communication unit that receives an emotion signal of at least one of the first user and the second user;
Based on an ideal state according to the use of the received emotion signal, a generation unit that generates advice to the first user,
A control unit that transmits the generated advice to the client terminal corresponding to the first user via the communication unit;
An information processing system comprising:   The information processing system according to claim 1, wherein the generation unit generates advice to the first user so that the emotion signal of the first user approaches an ideal state according to the application.   The information processing system according to claim 1, wherein the generation unit generates advice to the first user based on correspondence information between the emotion signal state and advice accumulated for each use.   The said production | generation part produces | generates the advice to the said 1st user based on the comparison result which compared the received emotion signal and the emotion signal of the ideal state according to the said use. Information processing system.   The generation unit is based on a comparison result of comparing a combination value of the current emotion signal of the first user and the current emotion signal of the second user with an emotion signal in an ideal state according to the application. The information processing system according to claim 1, wherein advice for the first user to approach the ideal state is generated.   The generation unit is based on a comparison result of comparing a combination value of the current emotion signal of the first user and a default value of the emotion signal of the second user with an emotion signal in an ideal state according to the application. The information processing system according to claim 1, wherein advice for the first user to approach the ideal state is generated.   The generating unit sets the normal value as an ideal state based on the current emotion signal of the second user and the normal value of the second user according to the application, and the second unit approaches the ideal state. The information processing system according to claim 1, wherein advice for one user is generated.   The information processing system according to claim 1, wherein the generation unit generates advice to the first user such that the emotion signal of the first user is away from an ideal state according to the application.   The information processing system according to claim 1, wherein the generation unit generates advice to the first user so that an emotion signal of the first user approaches a non-ideal state according to the application.   The said control part is controlled to transmit the emotion signal of the said 1st user or 2nd user used for the production | generation of the said advice, and the emotion signal of the said ideal state with the said produced | generated advice. Information processing system described in 1. A selection input unit that accepts selection of an application in which the first user communicates with the second user;
Providing advice to the first user based on an ideal state of the emotion signal detected from at least one of the first user and the second user according to the usage selected by the selection input unit A communication unit that transmits the selected usage to an external server that receives the advice to the first user generated according to the usage and emotion signal;
A notification unit for notifying advice to the first user;
An information processing system comprising: The information processing system includes:
A detector that detects an emotion signal of at least one of the first user and the second user;
The information processing system according to claim 11, wherein the communication unit transmits the detected emotion signal to the external server.   The said notification part notifies an advice to the said 1st user so that the said 1st user's emotion signal may approach the ideal state according to the use selected by the said selection input part. Information processing system.   The said notification part notifies an advice to the said 1st user so that the said 1st user's emotion signal may leave | separate from the ideal state according to the use selected by the said selection input part. Information processing system.   The said notification part notifies the advice to the said 1st user so that the emotion signal of the said 1st user may approach the non-ideal state according to the use selected by the said selection input part. Information processing system.   The notifying unit notifies a graph indicating the emotion signal of the first user or the second user used to generate the advice and the emotion signal in the ideal state together with the generated advice. The information processing system described.   The information processing system according to claim 11, wherein the notification unit notifies the first user by vibration using a vibration unit mounted on a head-mounted device. Setting an application in which the first user communicates with the second user;
Receiving an emotion signal of at least one of the first user and the second user by a communication unit;
Generating an advice to the first user based on an ideal state according to the use of the received emotion signal;
Controlling the generated advice to be transmitted to the client terminal corresponding to the first user via the communication unit;
Including a control method. Computer
A selection input unit that accepts selection of an application in which the first user communicates with the second user;
Providing advice to the first user based on an ideal state of the emotion signal detected from at least one of the first user and the second user according to the usage selected by the selection input unit A communication unit that transmits the selected usage to an external server that receives the advice to the first user generated according to the usage and emotion signal;
A notification unit for notifying advice to the first user;
A storage medium storing a program for functioning as a computer.

Images