JP6351692B2 - Display control device - Google Patents

Description

  The present invention relates to a display control apparatus.

There is known a terminal that learns a conversation between a user and a call partner and accumulates a response of the call partner in response to the user's inquiry in a response table (see, for example, Patent Document 1). There is also known an emotion generation device including a neural network that inputs user information, device information, and the current emotion state of itself and outputs the next emotion state (see, for example, Patent Document 2). In addition, a technique for storing a space-time pattern in an associative memory including a plurality of electronic neurons having a layer-neural network relationship having directional artificial synaptic connectivity is known (see, for example, Patent Document 3).
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2011-253389 [Patent Document 2] Japanese Patent Application Laid-Open No. 10-254592 [Patent Document 3] Japanese Patent Application Publication No. 2013-535067

  It is desired to display emotions in an appropriate manner.

  In the first aspect of the present invention, the display control device acquires the information indicating the current emotion of the target object, and the spherical first display object according to the emotion change indicated by the information. And a display control unit that changes the shape or color.

  The display control unit may change the shape or color of the second display object located inside the first display object in accordance with the change in emotion.

  The information may indicate the current main emotion and sub emotion of the target object. The display control unit may change the shape or color of the second display object according to a change in the main emotion, and may change the shape or color of the first display object according to a change in the sub-emotion.

  The information may include strength information indicating the strength of a plurality of emotions. The display control unit combines the colors associated with the plurality of second emotions that can be selected as the main emotion among the plurality of emotions by weighting according to the strengths of the plurality of second emotions indicated by the strength information. The second display object may be displayed with the selected color.

  The display control unit synthesizes the color associated with the plurality of first emotions that can be selected as the sub-emotion among the plurality of emotions by weighting according to the strength of the plurality of first emotions indicated by the strength information. The first display object may be displayed with the selected color.

  The information may include strength information indicating the strength of a plurality of emotions. A plurality of emotions may be mapped to different positions by the emotion map, and colors may be associated with positions where the emotion map maps a plurality of emotions. The display control unit may display the first display object in a color determined from the strength of the plurality of emotions and the color associated with the position where the plurality of emotions are mapped by the emotion map.

  The information may include strength information indicating the strength of a plurality of emotions. The display control unit may display the first display object with a color obtained by combining colors associated with a plurality of emotions with weighting according to the strengths of the plurality of emotions.

  The information may include strength information indicating the strength of emotion. The display control unit may change the shape of the first display object according to the strength of emotion.

  The first display object may be formed by a plurality of points and lines connecting the plurality of points. The display control unit may shift the position of at least one of the plurality of points from a predetermined position on the spherical surface as the emotion is stronger.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

An example of the whole composition of display system 10 concerning this embodiment is shown roughly. 2 is a block diagram schematically showing a block configuration of a user terminal 40 and a server 60. FIG. 1 schematically shows an emotion map 300 to which a plurality of emotions are mapped. 1 schematically shows a part of a neural network used by a display system 10. It is an example of the correspondence information which matches the strength of anger of the user 50 and the endocrine substance. It is an example of binding coefficient correspondence information that correlates the secretion amount of noradrenaline and the binding coefficient BS. An example of the display object 20 which the display part 140 displays is shown typically. An example of the display object which changed the shape and the color is shown typically. 4 is a flowchart showing a processing procedure in a user terminal 40. 5 is a flowchart showing a processing procedure in a server 60. An example of the screen transition of the display part 140 in a conversation application is shown. 1 illustrates an example of a computer 2000 in which embodiments can be implemented in whole or in part.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 schematically shows an example of the overall configuration of a display system 10 according to the present embodiment. The display system 10 includes a server 60 and a user terminal 40. The user terminal 40 is an example of a display device. The user terminal 40 is an example of a target object that specifies emotions in the display system 10.

  In the present embodiment, the user terminal 40 is a mobile terminal. The user terminal 40 is a mobile phone, for example. The user terminal 40 has a communication function. The user 50 is a user of the user terminal 40. The user terminal 40 exchanges conversations with the user 50.

  The user terminal 40 may be a computer such as a portable information terminal or a tablet terminal. The user terminal 40 may be a computer incorporated in a robot. The user terminal 40 may be an embedded computer embedded in a device such as a home appliance. The user terminal 40 may be a computer provided in a store. The user terminal 40 may be a computer incorporated in a vending machine such as a vending machine. The user terminal 40 may be a computer mounted on a vehicle. The vehicle may be an automobile or a motorcycle. The user terminal 40 may be any other computer that can interact with the user.

  The server 60 is provided remotely from the user terminal 40. The server 60 can communicate the user terminal 40 through the communication network 90. For example, the server 60 receives the information acquired by the user terminal 40 through the communication network 90, specifies the emotion of the user terminal 40, processing to be performed by the user terminal 40, and the like based on the received information. Information indicating emotions and processing is transmitted to the user terminal 40 through the communication network 90. In the present embodiment, only one user terminal 40 is shown, but the display system 10 may include a plurality of user terminals having the same function as the user terminal 40. When the display system 10 includes a plurality of user terminals, users of at least some of the plurality of user terminals may be different from users of other user terminals.

  In the display system 10, the server 60 executes a process of specifying the emotion of the user terminal 40 or determining the process of the user terminal 40. However, the user terminal 40 may execute part or all of the processing in the server 60.

  When detecting the voice input from the user 50, the user terminal 40 extracts text information from the voice by voice recognition. Further, the user terminal 40 analyzes the input voice and specifies state information including emotion information of the user 50. The state information indicates the state of the user 50, the state of the user terminal 40, and the like. The user terminal 40 transmits the extracted voice input text information and the user 50 status information to the server 60 via the communication network 90. The server 60 determines the response content based on the text information received from the user terminal 40 and transmits text information indicating the response content to the user terminal 40. Further, the server 60 specifies the emotion of the user terminal 40 based on the state information received from the user terminal 40. The server 60 transmits emotion information indicating the identified emotion to the user terminal 40 through the communication network 90.

  When the user terminal 40 receives the text information indicating the response content from the server 60, the user terminal 40 generates a voice from the user terminal 40 based on the received text information. Thereby, the user terminal 40 has a conversation with the user 50. Moreover, when the user terminal 40 receives emotion information from the server 60, the user terminal 40 displays the display object 20 representing emotion on the display unit 140 included in the user terminal 40 based on the received emotion information.

  The server 60 generates the latest emotion information at a predetermined time interval and transmits it to the user terminal 40. For example, the server 60 generates the latest emotion information at 1 second intervals. The user terminal 40 changes the color or shape of the display object 20 in accordance with a change in emotion based on the latest received emotion information. Thereby, the change of emotion can be appropriately presented to the user 50.

  In the present embodiment, it is assumed that the target object that is the target for specifying the emotion is the user terminal 40. However, the target object is not limited to the user terminal 40. The target object may be a virtual object. For example, the target object may be a virtual character. For example, the virtual character may be virtually provided by an application program. For example, in a conversation application on the user terminal 40, a virtual object recognized as a conversation partner may be regarded as a target object.

  FIG. 2 is a block diagram schematically showing a block configuration of the user terminal 40 and the server 60. First, the functional block configuration of the user terminal 40 will be described. The user terminal 40 includes a sensor unit 120, a processing unit 130, a display unit 140, an audio output unit 180, and a communication unit 102. The processing unit 130 functions as a display control device. The processing unit 130 may be a processor such as an MPU. The display unit 140 includes a display device such as a liquid crystal display. The communication unit 102 is responsible for communication with the server 60. The communication unit 102 may be a communication device such as a network IF. The audio output unit 180 includes, for example, a speaker. The user terminal 40 may include a storage medium that stores data for functioning as the user terminal 40.

  The sensor unit 120 includes various sensors such as a microphone, a gyro sensor, a camera, and a battery remaining amount sensor. The microphone of the sensor unit 120 acquires ambient sound. For example, the microphone of the sensor unit 120 acquires the voice of the user 50. The camera of the sensor unit 120 shoots with visible light and generates image information of moving images and still images. The gyro sensor of the sensor unit 120 detects the angular velocity of the entire user terminal 40 and each part of the user terminal 40. The battery remaining amount sensor of the sensor unit 120 detects the remaining battery capacity of the user terminal 40.

  The sensor unit 120 outputs, to the processing unit 130, various sensor data such as voice data acquired by the microphone, an image captured by the camera, an angular velocity detected by the gyro sensor, and a remaining capacity detected by the battery remaining amount sensor. To do. The processing unit 130 supplies the acquired sensor signal to the communication unit 102 and transmits it to the server 60. Further, the processing unit 130 causes the voice output unit 180 of the user terminal 40 to output based on the text information received from the server 60. Further, the processing unit 130 causes the display unit 140 to display the display object 20 based on the emotion information received from the server 60.

  Next, the functional block configuration of the server 60 will be described. The server 60 includes a processing unit 270, a communication unit 202, and a storage unit 280. The processing unit 270 includes an emotion processing unit 272 and a conversation processing unit 274. The emotion processing unit 272 includes a secretion information generation unit 200, an input information generation unit 210, an adjustment unit 220, a determination unit 240, and an emotion identification unit 260.

  The communication unit 202 is responsible for communication with the user terminal 40. The communication unit 202 may be a communication device such as a network IF. The storage unit 280 includes a storage medium such as a hard disk device or a flash memory. The storage unit 280 includes a volatile storage device such as a RAM. The storage unit 280 stores data necessary for executing the processing of the processing unit 270, in addition to program codes read by the processing unit 270 during execution and various kinds of temporary data.

  The text information from the user terminal 40 received by the communication unit 202 is output to the conversation processing unit 274. The conversation processing unit 274 determines the content of the response to the user 50 based on the received text information. The conversation processing unit 274 outputs text information indicating the determined response content to the communication unit 202. The communication unit 202 transmits text information indicating the response content to the user terminal 40.

  The state information received by the communication unit 202 from the user terminal 40 is output to the emotion processing unit 272. In the emotion processing unit 272, the input information generation unit 210 generates input information to the neural network for specifying emotion based on the state information. The input information includes emotion information of the user 50, sensor information acquired by various sensors of the user terminal 40, and the like. The input information generated by the input information generation unit 210 is output to the NN calculation unit 230.

  The secretion information generation unit 200 generates secretion information indicating the secretion amount of one or more endocrine substances based on the input information generated from the input information generation unit 210. Examples of endocrine substances include noradrenaline and dopamine. The secretory information indicating the secretory amount of the endocrine substance is generated in a pseudo manner as the internal information in the display system 10 and does not actually secrete the endocrine substance.

  The adjustment unit 220 adjusts calculation parameters for the NN calculation unit 230 to determine emotion from the input information based on the secretion amount of the endocrine substance indicated by the secretion information generated by the secretion information generation unit 200. The calculation parameter adjusted by the adjustment unit 220 may be a coupling coefficient of an artificial synapse included in the neural network.

  The storage unit 280 stores correspondence information that associates the input information with the secretion amount of the endocrine substance. Secretion information generation unit 200 changes the secretion amount of the endocrine substance associated with the input information generated by input information generation unit 210. The storage unit 280 stores correspondence information that associates each of the plurality of artificial synapses with an endocrine substance. The adjustment unit 220 changes the coupling coefficient of each of the plurality of artificial synapses associated with the endocrine substance according to the correspondence information, according to the secretion amount of the endocrine substance indicated by the secretion information generated by the secretion information generation unit 200.

  The NN calculation unit 230 performs a neural network calculation using the input information output from the input information generation unit 210. The determination unit 240 determines an artificial neuron based on the calculation result of the neural network by the NN calculation unit 230. The emotion identification unit 260 identifies an emotion based on the determination result of the artificial neuron by the determination unit 240. The emotion identifying unit 260 identifies the strength of emotion based on the firing state of the artificial neuron and the secretion amount of the endocrine substance.

  The neural network includes a plurality of emotional artificial neurons, which are artificial neurons in which emotions are defined. The determination unit 240 determines the current firing state of each of the plurality of emotional artificial neurons. The emotion identification unit 260 determines the current emotion based on the firing state of the emotion artificial neuron determined by the determination unit 240. For example, the emotion identification unit 260 may identify one or more emotions assigned to one or more emotion artificial neurons that have ignited as the emotion of the user terminal 40. Emotion specifying unit 260 may specify a stronger emotion as the amount of endocrine secretion increases. In addition, the emotion identification unit 260 may identify a stronger emotion as the output of the emotion artificial neuron in the fired state is larger. Emotion information including information indicating the emotion specified by the emotion specifying unit 260 and information indicating the strength of the emotion is supplied to the communication unit 202 and transmitted from the communication unit 202 to the user terminal 40. The communication unit 202 is an example of an acquisition unit that acquires information indicating the current emotion of the user terminal 40.

  Next, in the user terminal 40, the process regarding the information which shows the emotion which the communication part 202 received is demonstrated. The processing unit 130 changes the shape or color of the spherical first display object in accordance with the change in emotion indicated by the emotion information. Specifically, the processing unit 130 changes the shape or color of the second display object located inside the first display object in accordance with the change in emotion.

  The emotion information indicates the current main emotion and sub emotion of the user terminal 40. The processing unit 130 changes the shape or color of the second display object according to the change in the main emotion, and changes the shape or color of the first display object according to the change in the secondary emotion. As described above, the processing unit 130 changes the shape or color of one display object among the plurality of display objects according to the change in the main emotion, and among the plurality of display objects according to the change in the sub-emotion. The shape or color of the other display object is changed. Note that the processing unit 130 may change the color of the second display object according to a change in the main emotion, and change the color of the first display object according to a change in the sub emotion.

  The emotion information may include strength information indicating the strength of a plurality of emotions. In this case, the processing unit 130 corresponds to the strength of the plurality of second emotions indicated by the strength information, with the color associated with the plurality of second emotions that can be selected as the main emotion among the plurality of emotions. The second display object is displayed with the color synthesized by weighting. In addition, the processing unit 130 weights the colors associated with the plurality of first emotions that can be selected as the secondary emotion among the plurality of emotions according to the strengths of the plurality of first emotions indicated by the strength information. The first display object is displayed with the color synthesized in step (b).

  A plurality of emotions may be mapped to different positions by the emotion map, and colors may be associated with positions where the emotion map maps a plurality of emotions. In this case, the processing unit 130 may display the first display object in a color determined from the strength of the plurality of emotions and the color associated with the position where the plurality of emotions are mapped by the emotion map. In this way, the processing unit 130 may display the first display object with a color obtained by combining colors associated with a plurality of emotions with weighting according to the strengths of the plurality of emotions.

  The processing unit 130 changes the shape of the first display object according to the emotion strength. For example, the processing unit 130 causes a part of the first display object to protrude more greatly as the emotion is stronger. The processing unit 130 may increase the degree of deformation of the first display object from the spherical shape as the emotion strength increases.

  As an example, the first display object is formed by a plurality of points and lines connecting the plurality of points. In this case, the processing unit 130 shifts the position of at least one of the plurality of points from a predetermined position on the spherical surface as the emotion is stronger. The processing unit 130 may reduce the angle formed by the two lines connecting the shifted point and another point.

  FIG. 3 schematically shows an emotion map 300 to which a plurality of emotions are mapped. In the emotion map 300, the emotions are arranged concentrically radially from the center. The closer to the center of the concentric circles, the more primitive emotions are placed. Outside the concentric circles, emotions representing the state and behavior born from the emotional state are arranged. Emotion is a concept that includes emotions and mental states. In the emotion map 300, emotions that express a pleasant feeling are generally arranged above the concentric circles. In addition, below the concentric circles, an emotion representing an unpleasant feeling is generally arranged. On the left side of the concentric circles, emotions generated from reactions that occur in the brain are generally arranged. On the right side of the concentric circles, emotions induced by situation judgment are generally arranged.

  The first circumference includes emotions such as “like” and “relief”. The first circumference includes emotions related to emotion. The second circumference includes emotions such as “love” and “joy”. The second circumference includes emotions related to emotional emotions. The third circumference includes emotions such as “love” and “fun”. The third circle includes emotions related to emotional expression. The fourth circumference includes emotions such as “great” and “proud”. The fourth circumference includes emotions representing emotional state expressions.

  The neural network to be calculated by the NN calculation unit 230 includes an artificial neuron assigned to each emotion shown in the emotion map 300. In the neural network, an artificial neuron for input is assigned to each input located on the innermost side of the concentric circle in the emotion map 300. The input information generated by the input information generation unit 210 is input to the input artificial neuron assigned to the input. Then, artificial neurons are generally connected by artificial synapses from the inside to the outside to form a neural network.

  The NN calculation unit 230 repeatedly calculates a neural network based on the input information, and the determination unit 240 determines the firing state of each artificial neuron. The emotion identifying unit 260 identifies the emotion of the user terminal 40 from the firing state of each artificial neuron. For example, the emotion identifying unit 260 identifies the emotion to which the fired artificial neuron is assigned as the emotion that the user terminal 40 has. If there are a plurality of fired artificial neurons, the user terminal 40 determines that the user terminal 40 has a plurality of emotions.

  A color is associated with a position where the emotion map 300 maps an emotion. As an example, the region 310 included in the emotion map 300 is associated with yellow or a color whose main color component is yellow. In the area 310, emotions of “love”, “happy”, “fun”, “joy”, “love”, “joy”, “like” are located. A region 310 is a region where emotions that generally express happiness are located.

  The region 320 is associated with red or a color whose main color component is red. In the region 320, emotions of “dislike”, “spicy”, “anger”, “sad”, and “don't forgive” are located. Area 320 is an area where emotions that generally indicate discomfort are located.

  The region 330 is associated with a green color or a color whose main color component is green. Within the region 330, emotions of “relief”, “relax”, and “reassuring” are located. The region 330 is a region where an emotion that generally indicates stability is located.

  The region 340 is associated with blue or a color whose main color component is blue. In the region 340, emotions of “anxiety”, “fear”, “scary”, “pain”, and “pain” are located. The region 340 is a region where emotions generally expressing anxiety are located.

  In FIG. 3, as an example, only the colors associated with the area 310, the area 320, the area 330, and the area 340 are illustrated, but the colors are associated with all the emotion positions mapped by the emotion map 300. ing.

  The processing unit 130 displays the display object 20 in a color determined from the strength of the plurality of emotions and the color associated with the position where the plurality of emotions are mapped by the emotion map 300. The emotions included in the first circle, the second circle, and the three circles of the emotion map 300 correspond to the main emotion. Emotions included in the fourth circumference of the emotion map 300 correspond to secondary emotions. The processing unit 130 may determine the color of the display object 20 using the color associated with the emotion corresponding to the main emotion and the color associated with the emotion corresponding to the secondary emotion. The color of the display object 20 determined by the processing unit 130 will be described later.

FIG. 4 schematically shows a part of the neural network used by the display system 10. Part of the illustrated neural network includes artificial neurons N ¹ , N ² , N ³ , N ⁴ , N ⁵ and N ⁶ and artificial synapses S ¹² , S ¹⁴ , S ²³ , S ²⁵ , S ³⁶ , S ^42. , S ⁴³ , S ⁴⁵ , S ⁵³ , S ⁵⁶ . Artificial neurons correspond to neurons in the living body. An artificial synapse corresponds to a synapse in a living body.

E ¹ shows the input information. Artificial neuron N ¹ is an artificial neuron for input. The n pieces of input information E ₁ ¹ ... Input information E _n ¹ are input to the artificial neuron N ¹ .

Artificial Synapse ^{S 12} is an artificial synapses for connecting the neurons and ^{N 1} and the artificial neuron ^{N 2.} In particular, the artificial synapse ^{S 12,} the output of the artificial neuron ^{N 1,} an artificial synapse inputs to the artificial neuron ^{N 2.} The artificial synapse S ¹⁴ is an artificial synapse that connects the artificial neuron N ¹ and the artificial neuron N ⁴ . In particular, the artificial synapse S ¹⁴ is an artificial synapse that inputs the output of the artificial neuron N ¹ to the artificial neuron N ⁴ . Note that an artificial synapse in which j and k are integers and the output of the artificial neuron N ^j is input to the artificial neuron N ^k is denoted as an artificial synapse S ^jk .

Here, the i is an integer, it is assumed that each artificial neuron is denoted by N ^i. N ⁱ has a ^{S i} representing the status of the ^{N i,} and ^{V i} m which represents the internal state of the artificial neuron represented by ^{N i,} and ^{T i} which represents the threshold of the ignition of the ^{N i} as a parameter. The artificial synapse S ^jk has a coupling coefficient BS ^jk as a parameter. In the present embodiment, the artificial neuron may be collectively referred to as an artificial neuron N with its subscript omitted. In some cases, the artificial synapse is collectively referred to as an artificial synapse S with its subscript omitted. Similarly, the parameters of the artificial neuron may be collectively referred to as internal information Vm, threshold value T, and status S with their subscripts omitted.

  The status S, the internal state Vm, and the threshold T of the artificial neuron N are parameters that can be updated as time progresses. The status S is information regarding the firing state of the neuron, and indicates at least whether the artificial neuron N is in a firing state or a non-firing state. The internal state Vm is information relating to the membrane potential of the neuron, and is an example of a parameter representing the internal state or output of the artificial neuron N.

  In addition, the coupling coefficient BS, which is a parameter of the artificial synapse S, is a parameter that can be updated as time progresses. The coupling coefficient BS is information relating to synaptic plasticity, and indicates the coupling strength between the artificial neurons N to which the artificial synapse S is coupled.

  The NN calculation unit 230 updates the above-described parameters in the neural network from the input information, and calculates the internal state Vm of each artificial neuron N. In the present embodiment, when the internal state Vm exceeds the threshold T, the artificial neuron N has the status S in the firing state. In the firing state, the artificial neuron N outputs a predetermined value over a predetermined time. The output of the fired artificial neuron N may drop. The value output by the fired artificial neuron N may increase and then decrease. The value output from the firing artificial neuron N may be constant. When a predetermined time has elapsed, the status S of N returns to the last ignition.

Here, the contents of calculation by the NN calculation unit 230 will be described more specifically with N ² taken up. NN arithmetic unit 230, an input ^{I 2} to ^{N 2,} is calculated by ^{BS 12 × Vm 1 × f (} S 1) + BS 42 × Vm 4 × f (S 4). Here, f (S) is a function that returns 0 if S is a value indicating unfired, and returns 1 if S is a value indicating firing. This f (S) corresponds to a model in which the synapse transmits the action potential only when the neuron fires. It may be f (S) = 1. This corresponds to a model that transmits a membrane potential regardless of the firing state of neurons. As f (S), a function corresponding to another transfer model of the membrane potential may be applied.

Generally, NN calculation unit 230, an input ^{I i} to ^{N i,} is calculated by _{^{Σ j BS ji × Vm j ×}} f (S j) + Σ j E j i. The NN operation unit 230 calculates the inputs I ⁱ and S ⁱ to the N ^{i at the} next time using the BS ^ji , Vm ^j , S ^j , and E ^j at the current time. The NN calculation unit 230 repeats this in time to determine the status S of each artificial neuron N in real time. Then, the determination unit 240 determines the status S of each artificial neuron N, and the emotion specifying unit 260 specifies the emotion of the user terminal 40 from the determination result of the determination unit 240. For example, the emotion specifying unit 260 specifies “happy” as the emotion of the user terminal 40 when an artificial neuron assigned with the emotion “happy” in FIG.

  Here, the adjustment unit 220 adjusts the coupling coefficient BS of the artificial synapse S based on the information acquired from the user terminal 40. For example, when it is detected that the remaining capacity that is the remaining amount of the storage battery of the user terminal 40 is 50% or less, the secretion information generation unit 200 increases the secretion amount of “noradrenaline” as an internal variable. Then, the adjustment unit 220 adjusts the coupling coefficient BS of the artificial synapse S associated with “noradrenaline” based on the secretion amount of “noradrenaline”. As will be described later, the generation of “noradrenaline” is set so as to enhance the coupling coefficient BS of the artificial synapse S on the path where the artificial emotional neuron corresponding to the emotion such as “anxiety” or “anger” fires, for example. ing. As a result, “noradrenaline” acts in such a direction that emotions such as “anxiety” and “anger” are likely to be generated.

  The secretion amount of the endocrine substance is associated with the binding coefficient BS of a specific artificial synapse S. Thereby, the ease of transmission of signals at various artificial synapses S in the neural network can be changed by the information acquired by the user terminal 40 through the secretion amount of the endocrine substances. Therefore, various emotions can be generated from the information acquired by the user terminal 40.

  FIG. 5 is an example of correspondence information that associates the intensity of anger of the user 50 with an endocrine substance. The storage unit 280 stores information indicating noradrenaline in association with a plurality of values indicating the strength of anger. More specifically, the storage unit 280 stores information indicating an increase in the amount of noradrenaline secreted in association with each anger intensity. The strength of anger is shown as a percentage of the maximum strength. Further, the increase amount of the secretion amount is shown as a ratio to the upper limit value 1 of the secretion amount represented by the internal variable used by the NN calculation unit 230. Thereby, the amount of noradrenaline secretion increases as the anger of the user 50 included in the input information increases. As a result, emotions of “anxiety” and “anger” are easily generated as emotions of the user terminal 40.

FIG. 6 is an example of binding coefficient correspondence information that correlates the secretion amount of noradrenaline and the binding coefficient BS. Storage unit 280, in association with total amount of secreted noradrenaline, and increase the coefficient of coupling coefficient ^{BS 14} of the artificial synapse ^{S 14,} the increase factor of the coupling coefficient ^{BS 45} of the artificial synapse ^{S 45,} the coupling coefficient of the artificial synapse ^{S 43} Information for associating the increase coefficient of the BS ⁴³ is stored. It is assumed that the artificial synapse S taken up here is connected to the artificial neuron N by strong coupling.

As shown in the figure, the increase coefficient of BS ^{14 and} the increase coefficient of BS ⁴⁵ are associated with larger values as the amount of noradrenaline increases. On the other hand, the increase coefficient of BS ⁴³ is associated with a smaller value as the amount of noradrenaline increases. Thus, for example, in a neural network shown in FIG. 4, from the direction from N ¹ to N ^3, towards the direction from N ¹ to N ⁵ it is likely to occur a signal is transmitted by the input information. Therefore, the artificial neuron arranged in the direction from N ¹ to N ⁵ becomes easier to fire. Therefore, for example, in the emotion map shown in FIG. 3, as noradrenaline increases, emotions arranged in a specific direction with respect to the center of the concentric circle, for example, emotions such as “anxiety” and “scary” are more likely to ignite. Therefore, it is possible to make it easier for the user terminal 40 to generate emotions similar to those that occur when a person is hungry.

  Here, a case has been described where the coupling coefficient BS of the artificial synapse S is adjusted in a direction that facilitates firing the output destination artificial neuron N. However, the increase coefficient may be set so that the coupling coefficient BS of the artificial synapse S can be adjusted in a direction that makes the output destination artificial neuron N difficult to fire. For example, when the artificial synapse S is strongly coupled, it is possible to make the output destination artificial neuron N difficult to fire by reducing the increase coefficient. On the other hand, when the artificial synapse S is connected to the artificial neuron N by inhibitory connection, it is possible to make the output artificial neuron N difficult to fire by increasing the increase coefficient, and by reducing the increase coefficient, It is possible to facilitate firing of the output destination artificial neuron N.

  The adjustment unit 220 refers to the coupling coefficient correspondence information and adjusts the corresponding coupling coefficient BS by an amount corresponding to the total secretion amount of each endocrine substance. Thereby, from the information acquired by the user terminal 40, the adjustment amount of the coupling coefficient BS can be adjusted in a complicated manner. As a result, emotional artificial neurons can be fired in various combinations. In addition, the relationship between the information acquired by the user terminal 40 and the endocrine substance, and the relationship between each endocrine substance and the binding coefficient BS are replaced with humans to make them correspond to each other. Can generate emotions.

  In addition, in relation to FIG. 5, the correspondence relationship between the intensity of anger of the user 50 as information acquired from the user terminal 40 and noradrenaline is illustrated. In addition, in relation to FIG. 6, the correspondence relationship between noradrenaline and the binding coefficient BS is illustrated. However, the correspondence between these pieces of information is illustrated for easy understanding of the action of endocrine substances in the neural network. It goes without saying that a correspondence relationship other than the correspondence relationship described with reference to FIGS. 5 and 6 may be determined.

  FIG. 7 schematically illustrates an example of the display object 20 displayed by the display unit 140. The display object 20 includes a display object 21 and a display object 22. The display object 21 has a spherical shape. The display object 21 is an example of a spherical first display object. The display object 22 is located inside the display object 21. The display object 22 is an example of a second display object located inside the first display object.

  The first display object 21 includes a point 710, a point 720, and a point 730. The display object 21 includes a straight line 712 that connects the points 710 and 720. The display object 21 includes a straight line 713 connecting the points 710 and 730. The display object 21 is formed by a plurality of points and straight lines connecting the plurality of points. In the display object 21, a line connecting a plurality of points may not be a straight line.

  The spherical display object may be a display object on a two-dimensional surface that is recognized as a substantially spherical shape by human eyes. The spherical shape includes a perfect spherical shape, but corresponds to a spherical shape if it can be regarded as a substantially spherical shape. A display object having an approximately spherical shape can be regarded as a spherical display object. When a perfect spherical display object is projected onto a two-dimensional surface, the outer shape of the display object is a circle. When the outer shape of a display object projected onto a two-dimensional surface is represented by a polygon that is a pentagon or more, the display object may be regarded as a spherical shape. In the case where the outline of a display object projected onto a two-dimensional surface is represented by a closed curve, when the rate of change in the rotation angle direction of the tangent to the closed curve is smaller than a predetermined value, the display object is a spherical shape. Can be considered.

  The processing unit 130 displays the display object 21 and the display object 22 with a color corresponding to the current emotion of the user terminal 40. Specifically, the processing unit 130 displays the display object 22 in a color corresponding to the current main emotion of the user terminal 40 and displays the display object 22 in a color corresponding to the current sub emotion of the user terminal 40. Let The main emotion and the secondary emotion are as described with reference to FIG.

  For example, the processing unit 130 is a color obtained by combining colors associated with a plurality of emotions that can be selected as main emotions among a plurality of emotions by weighting according to the strengths of the plurality of emotions indicated by the strength information, The display object 22 is displayed. For example, when the secondary emotion includes emotions of “love” and “anxiety”, the processing unit 130 displays the display object 22 in a color obtained by combining yellow and blue associated with the emotion map 300. For example, when the internal state Vm of the “like” artificial neuron is twice the internal state Vm of the “anxiety” artificial neuron, the processing unit 130 sets “like” and “anxiety” in a ratio of 2: 1. The display object 22 is displayed with the color synthesized in step (b). Similarly, the processing unit 130 combines a color associated with a plurality of emotions that can be selected as a sub-emotion among a plurality of emotions by weighting according to the strength of the plurality of emotions indicated by the strength information. Thus, the display object 21 is displayed.

  As a result, when the “like” emotion is high, the display object 22 is displayed in a yellow color, and when the “anxiety” emotion is high, the display object 22 is displayed in a blue color. Thereby, the type of emotion at the time of the exchange user terminal 40 can be presented to the user 50 in an easily understandable manner.

  In FIG. 7, an object 750 is a character object generated from text information obtained by the processing unit 130 analyzing the voice of the user 50. After the text information is transmitted to the server 60, the user terminal 40 receives the text information of the response content from the server 60. An object 751 is a character object generated by the processing unit 130 from the text information of the response content. Thereafter, the processing unit 130 generates an object 752 from the text information newly obtained by analyzing the voice of the user 50. After the text information is transmitted to the server 60, the user terminal 40 receives text information of new response content from the server 60. An object 753 is a character object generated by the processing unit 130 from the text information of the response content. As described above, the processing unit 130 causes the display unit 140 to display the character object indicating the conversation content of the user 50 together with the display object 20.

  FIG. 8 schematically shows an example of a display object whose shape and color are changed. A display object 25 in FIG. 8 is an example of a display object in which the shape of the display object 20 shown in FIG. 7 is changed. The display object 25 includes a display object 23 and a display object 24. The display object 24 is an example of a display object in which the color of the display object 22 illustrated in FIG. 7 is changed. Here, the display object 24 is a display object in which the color of the display object 21 is changed to yellow.

  The display object 23 is a display object in which the shape of the display object 21 is changed. The display object 23 includes a point 810, a straight line 812 connecting the points 720 and 810, and a straight line 813 connecting the points 730 and 810. Here, the change in the shape of the display object 21 will be mainly described.

  When the emotion strength exceeds a predetermined value, the processing unit 130 shifts the point 710 on the display object 21 to the position of the point 810. Point 810 is at a position shifted from point 710 in the direction indicated by arrow 880. The direction indicated by the arrow 880 is a direction outward from the center of the spherical shape indicated by the display object 21.

  As described above, when the emotion intensity exceeds a predetermined value, the processing unit 130 shifts the position of the point 710 toward the outside of the spherical shape. Further, the processing unit 130 shifts the point 810 and displays a line 812 connecting the point 720 and the point 810 instead of the line 712. Further, instead of the line 712, a line 812 connecting the points 730 and 810 is displayed. The angle formed by the line 812 and the line 813 is smaller than the angle formed by the line 712 and the line 713. The display object 23 has a shape protruding outward at the position of the point 810.

  The strength of emotion may be an index of the total amount of endocrine secretion. The strength of emotion may be based on the sum of the internal state Vm of the fired artificial neuron. The strength of emotion may be an index of the number of fired artificial neurons. The strength of emotion may be determined by using an arbitrary combination of the sum of secreted amounts of endocrine substances, the sum of internal states Vm of fired artificial neurons, and the number of fired artificial neurons.

  As described with reference to FIG. 8, the processing unit 130 shifts the position of at least one of the plurality of points from a predetermined position on the spherical surface as the emotion is stronger. Thereby, when the strength of emotion exceeds a predetermined value, a part of the display object 21 is projected and displayed. In addition, the process part 130 may protrude a part of display object 21 largely, so that the intensity | strength of several emotions is strong. The processing unit 130 may cause a part of the display object 21 to protrude more greatly as the strength of a plurality of emotions is stronger. Thereby, the strength of emotion can be presented to the user 50 in an easy-to-understand manner.

  The shape or color of the display object is an example of the display mode of the display object. That is, the processing unit 130 may change the display mode of the display object 20 according to the change of emotion.

  In FIG. 8, an object 754 is a character object generated from text information obtained by the processing unit 130 analyzing the voice of the user 50. After the text information is transmitted to the server 60, the user terminal 40 receives the text information of the response content from the server 60. An object 755 is a character object generated by the processing unit 130 from the text information of the response content. Even while the user 50 is talking, the user terminal 40 receives emotion information from the server 60 and changes the shape and color of the display object 20 as described with reference to FIGS. To go. Thereby, as shown in FIG. 8, it is possible to appropriately present to the user 50 that the feeling of happiness has increased in response to the word “thank you” being issued.

  FIG. 9 is a flowchart showing a processing procedure in the user terminal 40. The subject of this flowchart is mainly the processing unit 130.

  When this flowchart starts, the processing unit 130 waits for an event to occur (S902). Here, a description will be given by taking a speech detection event, conversation information reception event, emotion information reception event, and end event as events. The sound detection event occurs when a sound signal is input from the microphone in the sensor unit 120. The conversation information reception event occurs when the communication unit 102 receives text information of a response content from the server 60. An emotion information reception event occurs when the communication unit 102 receives emotion information from the server 60. The end event occurs when the operation of the user terminal 40 is ended. The end event may also occur when the operation of the application software according to this flowchart is ended.

  When the voice detection event occurs, the processing unit 130 performs voice analysis (S904), and specifies the content of the voice input and the emotion of the user 50 (S906). Then, the processing unit 130 causes the communication unit 102 to transmit the text information indicating the voice input content and the state information indicating the emotion of the user 50 (S908), and proceeds to S902.

  When a conversation information reception event occurs, the processing unit 130 outputs a voice from the voice output unit 180 based on the received text information of the response content (S920), and proceeds to S902.

  When an emotion information reception event occurs, the processing unit 130 determines the shape and color of the display object 20 based on the emotion information (S930). Then, the processing unit 130 deforms the display object 21 and the display object 22 with the determined shape and color (S932), and proceeds to S902.

  When the end event occurs, the processing unit 130 performs the end operation of the user terminal 40.

  FIG. 10 is a flowchart showing a processing procedure in the server 60. The main part of this flowchart is the processing unit 270.

  When this flowchart starts, the processing unit 270 waits for the occurrence of an event (S1002). Here, a text information reception event from the user terminal 40, a state information reception event from the user terminal 40, an emotion information generation event, and an end event will be described as events.

  The text information reception event occurs when the communication unit 202 receives text information of voice input from the user terminal 40. The state information reception event occurs when the communication unit 202 receives state information from the user terminal 40. The emotion information generation event occurs when emotion information is generated in the emotion processing unit 272. Specifically, the emotion information generation event occurs when an emotion is specified by the emotion specifying unit 260. The end event occurs when the operation of the server 60 is ended.

  When a text information reception event occurs, the received text information is input to the conversation processing unit 274 in the processing unit 270 (S1004). Then, the conversation processing unit 274 determines the content of the response to the text information based on the input text information (S1006). Then, the processing unit 270 generates text information based on the response content determined by the conversation processing unit 274, transmits the text information from the communication unit 202 (S1008), and proceeds to S1002.

  When the reception event of the state information occurs, the received state information is input to the emotion processing unit 272 in the processing unit 270 (S1020), and the process proceeds to S1002. In the emotion processing unit 272, calculations for specifying emotions are periodically performed. In the emotion processing unit 272, at the next calculation timing after the state information is input, the input information based on the received state information is supplied to the NN calculation unit 230, and the calculation of the neural network for specifying the emotion is performed.

  When the emotion information generation event occurs, the processing unit 270 causes the communication unit 202 to transmit the generated emotion information to the user terminal 40 (S1030).

  When the end event occurs, the processing unit 270 performs the end operation of the server 60.

  FIG. 11 shows an example of screen transition of the display unit 140 in the conversation application. Screen 1100 is an initial screen of the conversation application. A display object 1102 indicating the emotion of the user terminal 40 is displayed at the center of the screen 1100.

  When the user 50 makes a voice saying “What is Mr. X's contact?”, On the screen 1110, an object 1114 obtained from the analysis of the voice acquired by the microphone of the user terminal 40 is displayed near the center of the screen. . A display object 1112 indicating emotion is displayed at a position above the screen.

  The processing unit 130 searches the address book data for a person whose name includes “XX” obtained from the voice analysis. When only one person whose name includes “OO” is searched, an object 1124 indicating detailed information of the searched person is displayed on the screen 1120. A display object 1122 indicating emotion is reduced and displayed at an upper position on the screen.

  When a plurality of people whose names contain “XX” obtained from voice analysis are searched, if the number of searches is equal to or less than a predetermined number, an object 1134 indicating summary information of the searched people is displayed on the screen 1130. Is done. A display object 1132 indicating emotion is reduced and displayed at a position above the screen.

  In the case where a plurality of people whose names contain “XX” obtained from speech analysis are searched, if the number of searches exceeds a predetermined number, an object indicating the content of the response to narrow down the searched people on the screen 1140 1144 is displayed below the object 1114. The contents to be heard back are determined by the server 60. A display object 1142 indicating emotion is reduced and displayed at a position above the screen.

  Subsequently, when the user 50 speaks “sales headquarters”, on the screen 1150, the object 1154 obtained from the analysis of the voice acquired by the microphone of the user terminal 40 is displayed under the object 1144. A display object 1152 indicating emotion is displayed at a position above the screen.

  The processing unit 130 searches the address book data for a person whose attribute information includes “sales headquarters” obtained from the voice analysis. When only one person including “sales headquarters” in the attribute information is searched, the screen transitions to a screen 1120. When a plurality of people whose attribute information includes “sales headquarters” are searched, if the number of searches is equal to or less than a predetermined number, the screen transitions to a screen 1130. When a plurality of people whose attribute information includes “sales headquarters” is searched, if the number of search items exceeds a predetermined number, the person is narrowed down by another information as in the process described in relation to the screen 1140. A rehearsal may be made.

  As described above, according to the display system 10, emotions can be presented in an appropriate display mode.

  Various embodiments have been described with reference to procedures or block diagrams and the like. Each block in the block diagram may represent (1) a stage of the process in which the operation is performed or (2) a section of the device responsible for performing the operation. For example, in the server 60, each of the processing unit 270, the communication unit 202, and the storage unit 280 may represent one section of the server 60. In the processing unit 270, the emotion processing unit 272 and the conversation processing unit 274 may represent one section of the processing unit 270. In the emotion processing unit 272, the input information generation unit 210, the secretion information generation unit 200, the adjustment unit 220, the NN calculation unit 230, the determination unit 240, and the emotion identification unit 260 may represent one section of the emotion processing unit 272. In the user terminal 40, the communication unit 102 and the processing unit 130 may represent one section of the user terminal 40.

  Certain stages and sections are implemented by dedicated circuitry, programmable circuitry supplied with computer readable instructions stored on a computer readable medium, and / or processor supplied with computer readable instructions stored on a computer readable medium. It's okay. Dedicated circuitry may include digital and / or analog hardware circuitry and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuits include memory elements such as logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGA), programmable logic arrays (PLA), etc. Reconfigurable hardware circuitry, including and the like.

  Computer-readable media may include any tangible device capable of storing instructions to be executed by a suitable device so that a computer-readable medium having instructions stored thereon is designated by a procedure or block diagram. Constitutes at least part of the product including instructions that may be executed to provide a means for performing the performed operations. Examples of computer readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Electrically erasable programmable read only memory (EEPROM), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (RTM) disc, memory stick, integrated A circuit card or the like may be included.

  Computer readable instructions can be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or object oriented programming such as Smalltalk, JAVA, C ++, etc. Including any source code or object code written in any combination of one or more programming languages, including languages and conventional procedural programming languages such as "C" programming language or similar programming languages Good.

  Computer readable instructions may be directed to a general purpose computer, special purpose computer, or other programmable data processing device processor or programmable circuit locally or in a wide area network (WAN) such as a local area network (LAN), the Internet, etc. The computer-readable instructions may be executed to provide a means for performing the operations provided in and described in the procedure or specified in the block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

  FIG. 12 illustrates an example computer 2000 in which embodiments may be implemented in whole or in part. A program installed in the computer 2000 causes the computer 2000 to function as the server 60 according to the embodiment or one or more sections of the server 60, and operations associated with the server 60 or one or more sections of the server 60. And / or a process according to an embodiment or a stage of the process can be executed. Such a program is executed by the CPU 2012 to cause the computer 2000 to perform certain operations associated with some or all of the processing procedures and block diagram blocks for the server 60 described herein. It's okay.

  A computer 2000 according to the present embodiment includes a CPU 2012, a RAM 2014, a graphic controller 2016, and a display device 2018, which are connected to each other by a host controller 2010. Computer 2000 also includes ROM 2030. The ROM 2030 is connected to the host controller 2010 via the input / output controller 2020. The computer 2000 also includes input / output units such as a communication interface 2022, a hard disk drive 2024, a DVD-ROM drive 2026, and a memory card drive 2028, which are connected to the host controller 2010 via the input / output controller 2020. Yes. The computer 2000 also includes legacy input / output units such as a keyboard 2042, which are connected to an input / output controller 2020 via an input / output chip 2040.

  The CPU 2012 operates in accordance with a program stored in the ROM 2030 and / or the RAM 2014, thereby controlling each unit. The graphic controller 2016 acquires a frame buffer or the like provided in the RAM 2014 or image data generated by the CPU 2012 in the RAM 2014 and causes the display device 2018 to display the image data.

  The communication interface 2022 communicates with other electronic devices via a network. The hard disk drive 2024 stores programs and data used by the CPU 2012 in the computer 2000. The DVD-ROM drive 2026 reads a program and / or data from the DVD-ROM 2001 and provides the program and / or data to the hard disk drive 2024 via the RAM 2014. The memory card drive 2028 reads programs and / or data from the memory card 2003, and provides the programs and / or data to the hard disk drive 2024 via the RAM 2014. The memory card drive 2028 may write a program and / or data to the memory card 2003.

  The ROM 2030 stores therein a boot program executed by the computer 2000 at the time of activation and / or a program depending on the hardware of the computer 2000. The input / output chip 2040 may also connect various input / output units to the input / output controller 2020 via a parallel port, serial port, keyboard port, mouse port, and the like.

  The program is provided by a computer-readable medium such as a DVD-ROM 2001 or a memory card 2003. The program is read from a computer-readable medium, installed in the hard disk drive 2024, the RAM 2014, and / or the ROM 2030, which are also examples of the computer-readable medium, and executed by the CPU 2012. Information processing described in these programs is read by the computer 2000 to bring about cooperation between the programs and the various types of hardware resources. The apparatus or method may be configured by implementing information operations or processing in accordance with the use of computer 2000.

  For example, when communication is performed between the computer 2000 and an external device, the CPU 2012 executes a communication program loaded in the RAM 2014 and performs communication processing on the communication interface 2022 based on the processing described in the communication program. You may order. The communication interface 2022 reads and reads transmission data stored in a transmission buffer processing area provided in a recording medium such as the RAM 2014, the hard disk drive 2024, the DVD-ROM 2001, or the memory card 2003 under the control of the CPU 2012. The transmission data is transmitted to the network, or the reception data received from the network is written in a reception buffer processing area provided on the recording medium.

  Further, the CPU 2012 reads all or a necessary part of a file or database stored in an external recording medium such as the hard disk drive 2024, the DVD-ROM 2001, the memory card 2003 or the like into the RAM 2014, Various types of processing may be performed. Next, the CPU 2012 writes back the processed data to the external recording medium.

  Various types of information such as various types of programs, data, tables, and databases may be stored in the recording medium and subjected to information processing. The CPU 2012 performs various types of operations, information processing, condition determination, conditional branching, unconditional branching, information retrieval / information described in this specification and specified by the instruction sequence of the program for the data read from the RAM 2014. Various types of processing, including replacement, may be performed, and the result written back to the RAM 2014. Further, the CPU 2012 may search for information in files, databases, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 2012 specifies the attribute value of the first attribute. The entry matching the condition is searched from the plurality of entries, the attribute value of the second attribute stored in the entry is read, and the first attribute satisfying a predetermined condition is thereby read. The attribute value of the associated second attribute may be obtained.

  The programs or software modules described above may be stored on a computer readable medium on or near the computer 2000. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the program to the computer 2000 via the network. You can do it.

  Note that, similarly to the server 60, the computer can function as the user terminal 40. For example, similarly to the server 60, a program installed in a computer causes the computer to function as one or more sections of the user terminal 40 or the user terminal 40 according to the embodiment. Operations associated with multiple sections may be performed and / or processes or steps of the processes related to the user terminal 40 according to embodiments may be performed. Such a program is executed by the CPU to cause the computer to perform specific operations associated with some or all of the processing procedures and block diagram blocks for the user terminal 40 described herein. It's okay. When the user terminal 40 is realized by a personal computer or the like, a computer having the same configuration as the computer 2000 can be applied as the computer for the user terminal 40. When the user terminal 40 is realized as a portable terminal, a computer provided with an input device such as a touch input device in place of the keyboard 2042 except for the hard disk drive 2024 and the DVD-ROM drive 2026 in the configuration provided in the computer 2000. It can be applied as a computer for the user terminal 40.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first,” “next,” etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 10 Display system 20, 21, 22, 23, 24, 25 Display object 40 User terminal 50 User 60 Server 90 Network 102 Communication part 120 Sensor part 130 Processing part 140 Display part 200 Secretion information generation part 210 Input information generation part 220 Adjustment part 230 NN operation unit 240 determination unit 260 emotion identification unit 270 processing unit 272 emotion processing unit 274 conversation processing unit 280 storage unit 300 emotion map 180 voice output unit 202 communication unit 310, 320, 330, 340 region 710, 720, 720 point 712 , 713 Line 750, 751, 752, 753, 754, 755 Object 810 Point 812, 813 Line 880 Arrow 1100, 1110, 1120, 1130, 1140, 1150 Screen 1102, 1112, 1122, 1132, 1142, 114, 115 2 Display object 1114, 1124, 1134, 1144, 1154 Object 2000 Computer 2001 DVD-ROM
2003 Memory Card 2010 Host Controller 2012 CPU
2014 RAM
2016 Graphic controller 2018 Display device 2020 Input / output controller 2022 Communication interface 2024 Hard disk drive 2026 DVD-ROM drive 2028 Memory card drive 2030 ROM
2040 input / output chip 2042 keyboard

Claims (6)

An acquisition unit for acquiring information indicating the current main emotion and sub-emotion of the target object;
According to the change of the secondary emotion indicated by the information, the shape of the spherical first display object is changed in accordance with a change of the main emotion said information indicates, the plurality may be selected as the main emotion first The color of the second display object located inside the first display object is changed to a color obtained by synthesizing the color associated with the second emotion with weighting according to the strength of the plurality of second emotions. A display control unit. The display control unit further in accordance with a change in the main emotion, the display control apparatus according to claim 1 for changing the shape of the second display object. The display control unit further wherein in response to a change in secondary emotions, corresponding to the intensity of the secondary emotions first emotion previous SL multiple colors associated with the plurality of first emotion may be selected as color synthesized by weighting, the display control device according to claim 1 or 2 changes the color of the first display object. The information includes the strength information indicating the strength of the plurality of second emotion,
The plurality of second emotions are mapped to different positions by an emotion map, and a color is associated with a position where the emotion map maps the plurality of second emotions,
The display control unit is weighted according to the strength of the plurality of second emotions indicated by the strength information indicating a color associated with a position where the plurality of second emotions are mapped by the emotion map. The display control apparatus according to any one of claims 1 to 3 , wherein the second display object is displayed in a synthesized color. The information includes strength information indicating the strength of the secondary emotion,
The display controller, the according to the strength of the secondary emotions, the display control device according to any one of the first display 4 claims 1 to change the shape of the object. The first display object is formed by a plurality of points and a line connecting the plurality of points,
The display control according to claim 5 , wherein the display control unit shifts the position of at least one of the plurality of points from a position on a predetermined spherical surface as the strength of the secondary emotion is stronger. apparatus.

Images