JP2022031617A - Advice system and advice method - Google Patents

Abstract

To provide an advice system and an advice method capable of giving advices that users can reasonably accept.SOLUTION: An advice system has a first database that stores user's situation and first advices while associating the relationship therebetween, and a second database that stores user's reaction and second advices while associating the relationship therebetween. This advice system determines the user's situation (S1), selects a first advice from the first database based on the determined user's situation (S3 and S5), determines a user's reaction when presenting the selected first advice to the user (S7), and selects a second advice related to the first advice according to the determined user's reaction when presenting the first advice (S17 to S25).SELECTED DRAWING: Figure 6A

Description

The present invention relates to an advice system and an advice method in which a second advice is selected from a plurality of advices and presented to the user according to the reaction when the first advice is given to the user.

A system has been proposed to provide learning guidance to students through a network. For example, in Patent Document 1, students are asked to answer questions for classifying learning styles by type, and the server objectively classifies the students by type based on the answers. The teaching method that enables the instructor to give the optimum instruction without misunderstanding the characteristics of the person is disclosed.

Japanese Unexamined Patent Publication No. 2005-338300

In the teaching method shown in Patent Document 1, the characteristics of the user are taken into consideration. However, it does not consider how each user feels the advice, and it is not always the case that the user gives reasonably acceptable advice.

The present invention has been made in view of such circumstances, and provides an advice system, an advice method, an advice program, an advice terminal, an advice server, and the like that can give advice that can be reasonably accepted by the user. The purpose is.

In order to achieve the above object, the first invention relates to a user reaction determination unit that determines the reaction of the user when the first advice is presented to the user, and the user when the first advice is presented. It includes an advice selection unit that selects one of the second advice classified into a plurality of categories according to the reaction of the user determined by the reaction determination unit.

In the advice system according to the second invention, in the first invention, the user reaction determination unit determines whether or not the user can be persuaded based on the determined reaction of the user.
In the advice system according to the third invention, in the first invention, the user reaction determination unit separately determines the emotion and the rest, and further, the user's emotion determined by the user reaction determination unit. It has an emotion control unit that controls emotions based on the above.

In the advice system according to the fourth invention, in the first invention, the second advice classified into the plurality of above is divided into the content that appeals to the logical profit and the content that appeals to the sensory temptation. In presenting the second advice to the user, the distribution of the above contents is different.
In the advice system according to the fifth invention, in the first invention, the second advice classified into the plurality of cases is divided into a content for forcing the user and a content for guiding the user, and the user. In presenting the second advice to the above, the distribution of the above contents is different.

In the advice system according to the sixth invention, in the first invention, the second advice classified into the plurality of cases is divided into the content with assistance and the content without assistance, and the above-mentioned first is given to the user. In presenting the advice of 2, the distribution of the above contents is different.
In the advice system according to the seventh invention, in the first invention, the second advice classified into the plurality of cases includes a plurality of proposals having different additional information, and the user can select from among them.

The advice system according to the eighth invention is based on the first invention, when the user reaction determination unit accepts or rejects the second advice when the second advice is presented to the user. Determine information about the user's reaction.

In the eighth invention, the advice system according to the ninth invention outputs information regarding the second advice from the device presented to the user to the succeeding device, and the advice system selects. The third advice related to the second advice given can be output from the advice output unit of the second device related to the action performed by the user based on the second advice.

In the advice system according to the tenth invention, in the first invention, the second advice selection unit determines whether or not the result determined by the user reaction determination unit is likely to persuade the rejection reaction. However, if the result of this determination seems to be convincing, the above-mentioned second advice is selected.
In the tenth invention, the advice system according to the eleventh invention selects alternative advice when the result determined by the user reaction determination unit is unlikely to be convincing. ..

In the advice system according to the twelfth invention, in the first invention, whether the user reaction determination unit rejects the advice mentally or physically by the user. The second database stores the second advice according to whether the mental refusal or the physical refusal is made, and the second advice selection unit makes a determination by the user reaction determination unit. Based on the result, the second advice corresponding to the mental refusal or the second advice corresponding to the physical refusal is selected.

In the first invention, the advice system according to the thirteenth invention presents the first advice to the user by the avatar.
In the thirteenth invention, the advice system according to the fourteenth invention changes the avatar according to the user's preference or operation history.

In the advice system according to the fifteenth invention, in the first invention, the second database is divided into a plurality of types and stored according to the situation of the user.
In the fifteenth invention, the advice system according to the sixteenth invention determines one of the plurality of types based on the determination result by the user reaction determination unit. Select the second advice above.

The advice system according to the seventeenth invention has a first advice presenting unit that presents advice belonging to the first category to the user, a second advice presenting unit that presents advice belonging to the second category to the user, and the above. Among the stress determination unit that determines the stress of the user when the advice is presented to the user by the first advice presentation unit and the second advice presentation unit, and each stress determined by the stress determination unit. It has an advice selection unit that selects advice belonging to the one with less stress as advice to the user.

In the advice system according to the eighteenth invention, in the fifteenth invention, the first advice presenting unit presents advice corresponding to mental refusal as the first category, and the second advice presenting unit presents the advice. As the second category, we will present advice corresponding to physical refusal.

The advice system according to the nineteenth invention associates a user situation determination unit that determines a user's situation, an advice presentation unit that presents advice to the user, and a relationship between the user's situation and the first advice. A first advice selection unit that selects the first advice from the first database based on the first database and the situation of the user determined by the user status determination unit, and the first advice selection unit. When the first advice selected by the user is presented to the user, the user reaction determination unit for determining the reaction of the user and the second advice for storing the relationship between the reaction of the user and the second advice are stored in association with each other. The database and the second advice selection unit that selects the second advice related to the first advice according to the reaction of the user determined by the user reaction determination unit when the first advice is presented. Equipped.

The advice method according to the twentieth invention includes a first database that associates the relationship between the user's situation and the first advice, and a second database that associates and stores the relationship between the user's reaction and the second advice. In the advice method in the advice system having the above, the situation of the user is determined, the first advice is selected from the first database based on the determined situation of the user, and the selected first one When the advice is presented to the user, the reaction of the user is determined, and when the first advice is presented, the second advice related to the first advice is given according to the determined reaction of the user. select.
As for the advice method according to the twenty-first invention, in the twenty-second invention, the advice system further has a third database for storing the second advice and the avatar type in association with each other. Further, when the second advice is selected, the type of the avatar is changed according to the selected second advice.

In the advice method according to the 22nd invention, the situation of the user is determined, the first advice is selected from the first database based on the determined situation of the user, and the selected first advice is given. When presented to the user, the reaction of the user is determined, and when the first advice is presented, the avatars and items having different modes classified into a plurality of types according to the determined reaction of the user are selected. Select one to enable display.
As for the advice method according to the 23rd invention, in the 22nd invention, the avatar and the item can be independently combined.
The advice method according to the twenty-fourth invention, in the twenty-second invention, the item includes at least one of clothes, hairstyle, background, facial expression, voice, and gender.

The program according to the 25th invention includes a first database that associates the relationship between the user's situation and the first advice, and a second database that stores the relationship between the user's reaction and the second advice. This is a program for causing the computer provided in the advice system to give advice to the user, and determines the situation of the user, and based on the determined situation of the user, the first database is described. When the first advice is selected from the above and the selected first advice is presented to the user, the reaction of the user is determined, and when the first advice is presented, the determined user of the user is determined. According to the reaction, the computer is made to select the second advice related to the first advice.

In the advice terminal according to the 26th invention, when the first advice is presented to the user, the user reaction determination unit that determines the reaction of the user, the reaction of the user determined by the user reaction determination unit, and / Alternatively, it includes a transmission unit that transmits information related to the reaction to the outside, and an advice presentation unit that presents a second advice according to the information acquired from the outside.
The advice method in the advice terminal according to the 27th invention determines the reaction of the user when the first advice is presented to the user, and the determined reaction of the user and / or information related to the reaction. Is sent to the outside, and the second advice is presented according to the information obtained from the outside.
When the program according to the 28th invention presents the first advice to the user, the reaction of the user is determined, and the reaction of the determined user and / or the information related to the reaction is transmitted to the outside. Then, the computer is made to perform the second advice according to the information obtained from the outside.

The advice server according to the 29th invention has a user reaction acquisition unit that acquires the user's reaction obtained when the first advice is presented to the user from the outside, and the above-mentioned user reaction acquisition unit acquired by the user reaction acquisition unit. It includes an advice selection unit that selects a second advice based on a user's reaction, and an information transmission unit that transmits information to the outside so that the selected advice can be presented to the user.
In the advice method according to the thirtieth invention, the reaction of the user obtained when the first advice is presented to the user is acquired from the outside, and the second advice is selected from the obtained reactions of the user. , The information is transmitted to the outside so that the above-selected advice can be presented to the above-mentioned user.
The program according to the thirty-first invention acquires the user's reaction obtained when the first advice is presented to the user from the outside, selects the second advice from the acquired user's reaction, and selects the second advice. The computer in the advice server is made to execute the information to be transmitted to the outside so that the above-selected advice can be presented to the above-mentioned user.

The advice method according to the thirty-second invention is a determination step for determining a user situation and the above determination in an advice method that can be selected and displayed from avatars having different modes classified into a plurality of types and items that modify the avatar. It has a step of selecting the above item based on the said user situation.

The advice method according to the 33rd invention includes, in the 32nd invention, the emotional determination step of determining an emotional system reaction based on the user situation, and further, the emotional system reaction is If the item deviates from a predetermined level, a step for changing the above item is provided prior to the language advice.
As for the advice method according to the 34th invention, in the 32nd invention, the predetermined level is determined from the history of the user situation according to the emotional reaction when the user refuses the advice.

As for the advice method according to the 35th invention, in the 32nd invention, the user situation is determined from the coordinate values of the graph having a plurality of axes or the items of the tabular database.
In the advice method according to the thirty-sixth invention, in the thirty-second invention, the avatar and the item can be independently classified and recorded, and the item can be emphasized.
In the thirty-second invention, the advice method according to the thirty-seventh invention is the same as the item, such as clothes, hairstyle, background, style, posture, facial expression, voice color, tone, background, make-up, aura, and halo, which can modify the avatar. Contains at least one of the lights.

According to the present invention, it is possible to provide an advice system, an advice method, an advice program, an advice terminal, an advice server, and the like that can give advice that is reasonably accepted by the user.

It is a block diagram which mainly shows the electrical structure of the advice system which concerns on one Embodiment of this invention. It is a block diagram which mainly shows the electrical structure of the advice system which concerns on one Embodiment of this invention. It is a flowchart which shows the operation of the action assist in the advice system which concerns on one Embodiment of this invention. It is a figure explaining the kind of advice in the advice system which concerns on one Embodiment of this invention. It is a figure explaining the advice from the viewpoint of the user's emotion in the advice system which concerns on one Embodiment of this invention. It is a figure which shows the example of the advice data (DB) in the advice system which concerns on one Embodiment of this invention. It is a figure which shows the example of taking over advice in the advice system which concerns on one Embodiment of this invention. It is a figure which shows another example of taking over advice in the advice system which concerns on one Embodiment of this invention. It is a flowchart which shows the operation of the advice in the advice system which concerns on one Embodiment of this invention. It is a flowchart which shows the operation of the advice in the advice system which concerns on one Embodiment of this invention. It is a flowchart which shows the operation of the action assist in the advice system which concerns on one Embodiment of this invention. It is a flowchart which shows the operation of another example of action assistance in the advice system which concerns on one Embodiment of this invention. It is a flowchart which shows the operation of the health examination / behavior support in the advice system which concerns on one Embodiment of this invention. It is a flowchart which shows the operation of the advice making for trust building in the advice system which concerns on one Embodiment of this invention. It is a figure explaining the advice and the reaction of the person who received the advice in the advice system which concerns on one Embodiment of this invention.

The present invention is a system, method, and method for inspiring (advice, guiding) behaviors that are reasonably beneficial to the user, taking into consideration the situation (psychologically and environmentally) of the user. A terminal (device, device) that is the invention of a program and directly inspires the user, and a device or device such as a server that performs various support controls in cooperation with the terminal, for example, on the cloud. Regarding the improvement of.

Hereinafter, in explaining the advice system as an embodiment of the present invention, first, using FIG. 11, the user's reaction when presenting the advice A that requires action to the user, and the reaction according to this reaction. Explain the advice to be presented to the user.

However, it is better to simplify the classification of this "reaction" and "advice" as a countermeasure against it. If it is rational, it can be used in various situations, and the infrastructure of services can be simplified, leading to merits in terms of maintenance and operating costs. In the examples described below, various "reaction" classifications and countermeasures for each are regarded as two-way communication according to the situation, and are rationally organized and explained. As a result, it is not specialized for a service for a specific purpose, but can be applied in various life scenes, and we aim to build a relationship of trust with users through continuous communication. .. Here, too, first, an example of broadly classifying human reactions will be described. In the explanation of this embodiment, as the reaction when the advice A is presented to the user, (1) absolutely unreasonable (reaction 1), (2) conflict, and do not obey as it is (reaction 2), (3) acceptance. (Reaction 3), the case of being roughly classified into three types will be described.

When the user accepts the advice A and acts according to the advice as in the reaction 3, no particular action is required. On the other hand, if the user refuses to follow the advice, as in Reaction 1, then there is no way to persuade him to follow the advice, at least for the time being, so an alternative to the advice (or to it). It is only necessary to give advice on topic change that will lead to the connection (correspondence 1a).

Or, instead of giving advice that makes full use of logical reasoning such as persuasion, if there is an emotional barrier, emotional control that removes it is performed first. For example, if it is something like an avatar (a part of advice in a broad sense), it is not a logical advice, but it can have a soothing effect by looking at it or listening to the wording. can. Therefore, when the user shows reaction 1, making full use of the avatar may be expressed by the word alternative to advice. When the reaction 1 is shown, the user is in a state of mind that rejects the logical meaning of general advice such as words and letters. It is advisable to take advantage of the intimacy that appeals to the senses, which is unique to Avatar, and to respond to this state of mind by relying on the appearance, the feeling of hearing the voice, BGM, sound effects, and the like.

That is, in the present embodiment, in order to give advice in a broad sense (not only the alternative advice 1a but also the advice 2a given to the reaction 2 described later), the user's emotion is changed by making full use of the sensory healing effect. It includes the concept of emotional content control (avatar control) and more logical language control (advice in a narrow sense) that depends on the language such as conversation with the user, and monitors the user's situation to judge the emotional reaction. An emotion determination step (see, for example, S32 in FIG. 2) is provided. When it is determined by the emotion determination step that the emotional reaction deviates from a predetermined level, the item that modifies the avatar to supplement the effect is changed before activating the verbal advice (advice in a narrow sense). Steps (for example, S41 in FIG. 2, FIGS. 3A (a) and 3B, FIG. 3B, etc.) are provided. When using avatars to convey linguistic advice, we are trying to remove the emotional barriers that occurred to the user at the timing prior to this advice.

Advice is, so to speak, a type of communication, and while communication is an act of communicating feelings (emotions) and opinions to the other party through words, the content of the advice has a stronger element as an opinion than an emotion. Also, while communication is two-way, advice can be one-sided. However, one-sided communication tends to be ineffective because it is not known whether the other party will accept it. In this embodiment, the reaction of the subject at the time of advice is determined, and the effect of the advice is enhanced based on two-way. However, as long as it is communication, it is really useful to consider the emotional things of the other party in the opinion and aim to foster a relationship of trust so that it will be more than physical and mechanical. I try to get my advice trusted.

In addition, when communication is classified into "verbal communication" and "nonverbal communication", it is often said that the latter impression has a strong influence, so it is necessary to control this point accurately. It is said that this "nonverbal communication" has a large cultural difference, but it should be noted that it depends on the personality, tendency, taste and values of the person. With a uniform response, communication aimed at a relationship of trust fails. As an example of "non-verbal communication", there is a so-called "pictogram" that is sandwiched between e-mails and the like. Depending on the person who also receives the emoji, it often makes the feeling of strangeness and the difference in values with the person stand out. Emojis related to emotional elements such as smiles and crying faces are often used in these emojis, and if the scenes to be used are appropriate, they have the effect of giving a sense of familiarity and unity more than the language (text) part. Is excellent. However, in many cases, this is one-way, transmitted exactly as the writer wrote it, and is displayed without considering the emotions of the other party. In other words, the control of emotional barriers here is not taken into consideration.

On the other hand, as in the reaction 2 shown in FIG. 11, when the user conflicts when the advice A is presented and the advice cannot be followed as it is, the advice that can persuade the user is supplemented or expressed. It is better to provide advice such as changing (Correspondence 2a). This embodiment provides an advice system that can give advice that can persuade the user, such as when the user is in conflict, the user is convinced and takes the initiative in the direction indicated by the advice. offer.

That is, the advice system according to the present embodiment detects the user's reaction when giving general advice to the user (see, for example, S31 in FIG. 2, S5 in FIG. 6A, and S31 in FIG. 7), and the user. It is determined whether or not the rejection reaction of FIG. 2 is likely to be persuaded (see, for example, S37 in FIG. 2, S7, S17 in FIG. 6A, S37 in FIG. 7 and the like). If the result of this determination seems to be convincing, appropriate advice is presented in consideration of the background of the user's refusal (for example, S39 in FIG. 2, S17 to S25 in FIG. 6A, S37 to S41 in FIG. 7). See 2a etc. in FIG. 11). On the other hand, if the rejection reaction is shown as absolutely unreasonable, another advice is presented (see, for example, S13, S15 in FIG. 6A, 1a in FIG. 11 and the like).

Considering the classification of "verbal communication" and "nonverbal communication", it is desirable to make effective use of "nonverbal communication", which has a great intuitive influence, when presenting advice. However, as mentioned above, nonverbal communication also depends on a person's personality, propensity, tastes, and values, so as we accumulate communication, we confirm the above-mentioned factors (the person's type). I am doing it.

Further, in this advice system, when the advice A is accepted, the advice information is taken over if there is a takeover device (see, for example, S11 in FIGS. 5A, 5B, and 6A). Of course, even if the advice A is not accepted and is handed over to another hand-over device, the advice information may be handed over. Further, this advice system has a record control unit that records the behavior of the advice presentation target person based on the relationship between the information given to the target person and the reaction (for example, the user behavior history recording unit of FIG. 1A). 16a, see S27 and the like in FIG. 6B).

1A and 1B are block diagrams showing an example of an advice system according to the present embodiment. This advice system includes a mobile terminal 10 and a second information device 20, and both devices can exchange information via the Internet or the like. When exchanging this information, the advice transfer information 30 is transmitted and received between the second information device 20 and the mobile terminal 10.

In many cases, the mobile terminal 10 is used as a device of the advice system because the mobile terminal used daily has more opportunities to retain the user's behavior history and acquire the behavior information, and is more frequent. This is to consider the realistic situation where the service can be enjoyed through communication with various servers via the Internet or the like. The second information terminal 20 may be stationary or not used normally, and may handle only information that is biased from the information of the mobile terminal 10 that is used daily.

Therefore, in the present embodiment, it is considered that it is better to make the information of the mobile terminal 10 available in cooperation with the mobile terminal 10 as necessary with respect to the shortage of information in the second information terminal 20. be. Many Internet services try to manage similar users on the network to deal with similar problems. In that case, it is necessary to assume that the second information terminal 20 is used by the same user. However, if the second information terminal 20 is actually a shared device or a device that does not have a communication function with the Internet, there is a limit to the conventional system.

The second information device 20 is an example of a device that takes over the advice information when the user uses another device after the advice is presented on the mobile terminal 10, and the second information device 20 is shown in the figure. Not limited. If there is no device to take over, the system may consist of only the mobile terminal 10.

In addition, mobile terminals do not always give advice to users standalone, and in many cases, they communicate with or cooperate with external servers that can be connected to the Internet, or upgrade application software. The reality is that a wider variety of services are provided, and the Internet Communication Department 17 makes this possible. Here, illustrations are omitted for various servers linked on a network such as the Internet. That is, even if the mobile terminal has a user reaction determination unit that determines the user's reaction when the advice is presented to the user, it is the server that issues the next advice based on the user reaction determination unit. good.

When generating advice on the server, the server for advice is a user reaction acquisition unit (outside the server) that acquires the user's reaction obtained when the first advice is presented to the user from the outside. The advice selection unit (inside the server) that selects the second advice based on the user's reaction acquired by the user reaction acquisition unit (acquiring the user's reaction from the function equivalent block of the user reaction unit 15a in FIG. 1A) An information transmission unit (a block having a function equivalent to the advice information selection unit 15c) provided in the server and an information transmission unit (advice information output unit provided inside the server) that transmits information to the outside so that the selected advice can be presented to the user. A block having a function equivalent to 15d) is included. Each of these parts and the advice method performed by each part may be realized by a processor such as a CPU in the server according to a program.

Further, the mobile terminal linked with the server may have a communication unit that transmits the user's reaction determined by the user reaction determination unit to the outside, and presents the second advice according to the information from the server acquired from the communication unit. It is assumed that the mobile terminal has the advice presentation unit again. At this time, the mobile terminal can be said to be an advice terminal, but as described above, the second information terminal may substitute a part of the role and function of the mobile terminal.

When generating advice on the server, the advice terminal is equivalent to the user reaction determination unit (user reaction determination unit 15a in FIG. 1A) that determines the user's reaction when the first advice is presented to the user. (Having a function) and a transmission unit (transmitting a user's reaction or the like to a server outside the mobile terminal) that transmits the user's reaction and / or information related to the reaction determined by the user reaction determination unit to the outside. (Blocks corresponding to the advice information output unit 15d, the net communication unit 17, the device communication unit 18, etc.) of FIG. 1A, and the second advice generated according to the information transmitted from the transmission unit are received and presented. It has an advice presentation unit (a block corresponding to the advice information output unit 15d, the display unit 12, and the like in FIG. 1A). Each of these parts and the advice method performed by each part may be realized by a processor such as a CPU in the mobile terminal according to a program.

The mobile terminal 10 is a terminal held and portable by the user, has a small device for carrying and handling information such as a schedule, ToDo, address book, and memo, and has a personal computer function, and has an affinity with the Internet. Includes smartphones, etc., which are highly functional multifunctional mobile phones. The mobile terminal 10 includes an operation unit 11, a display unit 12, a sensor cooperation unit 13, an image pickup unit 14, a control unit 15, a recording unit 16, a net communication unit 17, and a device communication unit 18.

The operation unit 11 has various buttons arranged on the mobile terminal 10, an operation member such as a touch panel, and a sensor for detecting the operation state of these operation members. The operation status signal of the operation member detected by the sensor is output to the control unit 15. The operation unit 11 functions as an interface for inputting user instructions and the like.

The display unit 12 has a display for display and the like. The display has various images such as an image acquired by the imaging unit 14, an advice image output from the advice information output unit 15d, a mail acquired by the net communication unit 17, a photographed image acquired by the second information device 20, and the like. Information is displayed. Further, the display unit 12 is not limited to the visual display by the display for display, and may be provided with a speaker for voice display to perform the display by voice.

When the sensor cooperation unit 13 presents advice to the user, various sensors for detecting the user's reaction (for example, one that projects special light onto the human body to see reflection, one that uses a piezoelectric sensor, etc. Includes those that detect voice, etc.). Examples of the sensor include a microphone for acquiring the user's voice, a sensor for detecting biological information such as the user's blood flow, heartbeat, and blood pressure, and an imaging unit for detecting the user's facial expression, gesture, and the like. (It may also be used as the image pickup unit 14) and the like.

In FIG. 1A, various sensors are shown to be built in the mobile terminal 10, but the sensor cooperation unit 13 may acquire sensor information in cooperation with various wearable terminal information. .. In that case, a communication unit for exchanging information with each device exists as an interface, but it is omitted here. In addition, when effectively using "nonverbal communication", which has a great intuitive influence, the user himself does not respond in language, so he uses biometric information, etc., and uses the user's personality, propensity, tastes, and values ( You may want to check the person's type).

The image pickup unit 14 has an optical lens, an image pickup element, an image pickup element control circuit, an image pickup signal processing circuit, and the like, and converts an image into image data. The image pickup unit 14 may acquire a subject image in response to an instruction from the operation unit 11, and as described above, when giving advice to the user, the facial expression, appearance, and the like of the user may be obtained. An image showing a gesture or the like may be acquired. In recent years, the image pickup unit 14 has tended to be equipped with a plurality of smartphones, and they may be linked with each other, or a single image pickup unit may be used from among the plurality of images, or the image pickup unit 14 may be linked with an external image pickup unit. You may.

The control unit 15 is a processor having a CPU (Central Processing Unit), a memory for storing a program, and peripheral circuits. The control unit 15 controls the entire unit in the mobile terminal 10. The control unit 15 includes a user reaction determination unit 15a, a communication control unit 15b, an advice information selection unit 15c, and an advice information output unit 15d. Each of these parts is realized by the CPU in the control unit 15 in terms of software, and a part of them is realized by the hardware circuit in the control unit 15. However, the present invention is not limited to this, and all may be realized by software, and all may be realized by hardware circuits.

Further, the control unit 15 can determine the situation of the user who is using the mobile terminal 10. The user's situation is posted on the user's SNS, the user's schedule recorded on the mobile terminal 10, movement recording by GPS or the like, the user's conversation acquired by the microphone in the mobile terminal 10, and the image pickup unit 14. The user's situation is determined based on various information such as the image acquired by. The control unit 15 may collect the lifestyle of the user, and may determine and infer the situation of the user by adding the collected lifestyle and other information. The control unit 15 functions as a user status determination unit for determining the user status (see, for example, S1 in FIG. 6A, S71 in FIG. 9 and the like).

Further, the control unit 15 causes the succeeding device to output information regarding the second advice from the device presented to the user, and the user bases the third advice related to the selected second advice based on the second advice. It is possible to output from the advice output unit of the second device related to the action performed (for example, communication between the device communication unit 18 and the communication unit 25 in FIG. 1B, timing 1 and timing 2 in FIG. 5B (a), S11 in FIG. 6A and S29 in FIG. 6B). This second advice is divided into a plurality of types, and the optimum type is selected for each reaction of the user classified according to the idea of the advice system and the method. The third advice is also based on the same idea and is output according to the usage status of other devices and the state in which the user is involved in the device.

When the advice output from the advice information output unit 15d is displayed on the display unit 12, the user reaction determination unit 15a determines the reaction shown by the user based on the data acquired by the sensor cooperation unit 13. As this reaction, for example, it is determined whether the user absolutely rejects the advice, accepts it, or rejects it in conflict (for example, S5, S7, S9, S13 of FIG. 6A, FIG. 7 S33, FIG. 11 and the like). In addition, this determination may be made by "linguistic communication". However, in the relationship between people, rejection is often detected in "nonverbal communication", as it is said that "eyes are as good as mouth". As mentioned above, this also depends on a person's personality, propensity, tastes, and values, so it is necessary to consider the type of person.

Also, when the user rejects advice due to mental background, factors such as reason and instinct (emotion, sensation) (determined or appearing exclusively by nonverbal communication) cannot be ignored. The user reaction determination unit 15a determines, for example, whether or not the user's reaction to the advice is an emotionally biased rejection of the advice (see, for example, S32 in FIG. 2). When a user refuses, he / she may refuse for a rational reason, but in many cases, he / she may reject emotionally. In the case of emotional rejection, depending on the type and strength of emotions, when rational advice is presented, it may cause problems that cannot be logically organized and solved, so to deal with them first. Is. In this case, since the strength of the rejected emotion (emotion bias) differs depending on the situation, the strength of the rejected emotion (emotion bias) is determined, and the advice presented to the user is given according to the determination result. You should change it.

The user reaction determination unit 15a has an emotion determination unit 15aa and a non-emotion determination unit 15ab. The emotion determination unit 15aa determines the user's emotion based on the user's voice, image, or the like detected by the sensor cooperation unit 13. When the user's emotions change, the words expressed by voice, intonation, etc. change, and the facial expressions of the user, gestures by limbs and head, etc. also change. It may be acquired from an image acquisition unit or the like. The emotion determination unit 15aa determines the user's emotion based on this information.

When the emotion determination unit 15aa is input as text, the emotion may be inferred by using AI according to the tone or word of the text, and in the case of voice, the voice picked up by the microphone is converted into text. The same thing may be done, the frequency (change) may be analyzed and inferred, or the tone (input voice signal) may be analyzed and determined. Of course, anger and surprise can be determined by biological signals such as pulse and heartbeat disorder. These items are based on the large impact of nonverbal communication. In addition, it may be judged by linguistic communication, and of course, it may be inquired by a questionnaire. Further, since it is known that emotions can be estimated from the positions and shapes of facial parts, their temporal changes, and the like based on the images obtained by capturing the facial images, the emotion determination unit 15aa uses this technique. In addition, the user's emotions may be determined by using a database, an inference engine, or the like together. Further, when determining a user's emotion, it is possible to determine the depth and strength of the emotion by numerically expressing how far away from the data at the time of normal emotion. Of course, it is also possible to make inferences based on the depth and strength of emotions, or to make judgments in a database. When emotions are deep, they can be classified as severe, and when they are not deep, they can be classified as mild.

The non-emotion determination unit 15ab determines a reaction that is not caused by the user's emotion, based on the user's voice, image, or the like detected by the sensor cooperation unit 13. Among the reactions of the user, the reactions other than the reactions determined by the emotion determination unit 15aa are the reactions other than the emotions. When input as text, depending on the content of the text, AI may be used to infer reactions other than the user's emotions, and in the case of voice, the voice picked up by the microphone is converted into text. Similar inference may be made. Other than emotions, the emotional bias determined by analyzing and inferring the frequency (change) of the voice, or analyzing the magnitude and speed of the tone (input voice signal), or other biological signals, etc., is subtracted. The determination may be determined. Of course, you may inquire by questionnaire.

The user reaction determination unit 15a functions as a user reaction determination unit that determines the user's reaction when the first advice is presented to the user (for example, S7 in FIG. 6A, S33, S37, and 8 in FIG. 7). See S53, S57, S73, S83 in FIG. 9). The user reaction determination unit determines whether or not the user can be persuaded based on the user's reaction (see, for example, S37 in FIG. 2).

The user reaction determination unit determines emotions and others separately (see, for example, S38 in FIG. 7 and S73 in FIG. 9). When the second advice is presented to the user, the user reaction determination unit determines information regarding the user's reaction when the second advice is accepted or rejected (for example, S32 of FIG. 2 and FIG. 6A). See S7 etc.). The user reaction determination unit 15a functions as a user reaction determination unit that determines the user's reaction when the first advice selected by the first advice selection unit is presented to the user (for example, S32 in FIG. 2, FIG. 6A S7, FIG. 7 S33, S37, FIG. 8 S53, S57, FIG. 9 S73, S83, etc.).

The user reaction determination unit can classify the user's reaction into a plurality of types in some way, and the method of classification is required to be a classification that optimizes the subsequent advice. In other words, when the emotions of the person who received the advice are emphasized (especially effective for proposals that are likely to affect the emotions), it is preferable that the emotions can be classified. When the advice is likely to be accepted or rejected depending on the personality, tendency, aesthetics, outlook on life, taste, etc. of the person who received the advice (the tendency is that these types of people dislike or like this kind of advice. (In some cases) is required to be a reaction determination that can be classified in this way. Here, as mentioned above, if mental health is a concern, there may be a means to overcome it physically. According to this idea, the user reaction determination unit determines whether or not the user mentally or physically rejects the advice (for example, S37 in FIG. 7 and S75 in FIG. 9). , See S83).

Further, the user reaction determination unit 15a may determine the stress when receiving advice as the user's reaction (see, for example, S53 and S57 in FIG. 8). Further, the user reaction determination unit 15a determines tension and stress based on changes in biological information such as pulse, heart rate, and respiratory rate (for example, S7 in FIG. 6A, S33, S37, and 8 in FIG. 7). See S53, S55, S73, S83 in FIG. 9). Stress is treated in this embodiment as expressing a psychological conflict with advice. By separating or reducing factors such as the user's situation, surrounding environment, etc., such as being in the scorching sun, after exercising for a long time, eating, before, during, and after bathing, the difference before and after the advice is given. It is a separation of the psychological state of the user caused by it.

The user reaction determination unit 15a functions as a stress determination unit that determines the stress of the user when the advice is presented to the user by the first advice presentation unit and the second advice presentation unit (see, for example, S53 and S57 in FIG. 8). ). As mentioned above, if the rejection is due to physical factors such as lack of time or money, the user will not have to worry from the beginning, and there is no room for negotiation, and the rejection may be reflexive. Is high. However, it is not always an absolute refusal, and if you overcome your mental health, you may be able to take some measures. In addition, the case where you actually want to accept the advice, but when you think about it, you don't have the money, so it's a mental case. If you refuse for mental reasons, your mind will be tested, so you may feel stress and easily appear in biological reactions. In this case, the stress determination mainly determines mental rejection, but this stress determination may be used in other classifications.

The communication control unit 15b controls communication in the net communication unit 17 and the device communication unit 18 in the mobile terminal 10. The net communication unit 17 can be connected to a communication network such as the Internet, and the device communication unit 18 communicates with a device such as a second information device 20 at a short distance.

In a system like this embodiment, in many cases, the contact point with the user is handled by the information terminal, and the information terminal cooperates with the hardware, software, and program of the external device (server, etc.). , Realize the following steps and functions in terms of hardware built into the terminal or in cooperation with software stored in memory or the like. When the mobile terminal presents the first advice to the user, the user reaction determination unit (step, function) for determining the user's reaction and the user's reaction determined by the user reaction determination unit are transmitted to the outside. It functions as an advice terminal including a unit (step, function) and an advice presentation unit (step, function) that presents a second advice according to the information acquired from the transmission unit.

The advice information selection unit 15c selects advice to be presented to the user from the advice information recording unit 16a based on the result of determining the reaction of the user. The first thing to be presented to the user is to select from the advice information group 0, and then to select from the advice information groups 1 and 2 according to the user's reaction.

In the present embodiment, the advice information selection unit 15c can select the advice from various viewpoints when selecting the advice according to the reaction of the user. For example, the advice information selection unit 15c changes the instruction depending on whether the user's "barrier" to the advice presented to the user is mental or physical (S37 to FIG. 7). See S41).

It says that physical problems (time, money, health, physical constraints, etc.) can be compensated for in some way if there are no barriers to mental (accepting advice on human will and emotions). It adopts the idea. Time can be settled financially, money can be settled in time, and depending on your will, you may be able to overcome something, but mental things can be solved without physical restrictions, even if there are no physical restrictions. It rarely helps the negative part of mental health and is difficult to overcome.

There is also a way to reduce mental barriers by assisting and assisting physical obstacles, but the advice presented by mobile terminals, etc. this time first appeals to the mental aspect of the user, and the barriers there. First try to reduce. Therefore, based on such an idea, the advice information selection 15c selects the advice with an idea of overcoming this mental barrier. In other words, I classified them into reason and instinct (or sensation, emotion) that make up the mental elements, and gave advice to work effectively on these.

That is, a plurality of advices are prepared, the advices are classified into rational and sensory ones in advance, and the priority is changed according to the reaction tendency of the user. That is, the advice information selection unit 15c first gives advice in order to remove the mental barrier of the user and to decide whether it should be overcome rationally or in a sensory manner. Based on the reaction result of the user at the time of presentation, the support that does not feel pain (stress caused by psychological disorder, etc.) is selected and reflected in the next instruction.

This is because, in many cases, physical disabilities cannot be solved by themselves and cannot be solved without assistance and support, but they do not cause psychological conflict and have less stress response. Mental disabilities are changed by one's own way of thinking, and the independence is questioned, so it is based on the idea that the pressure of having to make a decision by oneself is likely to appear in the stress response.

Of course, in addition to providing advice by separating mental and physical disabilities, we analyze the factors of disability that we do not want to accept advice, divide it into various types, and lead to disability overcoming advice for each type. There are many other ways of thinking. Also in this specification, "disability" is classified according to the emotion of the user when the advice is presented (Fig. 3B) and the type of the user (personality, taste, aesthetics, etc.) (Fig. 6A). , Explains what kind of advice is preferred and what determines whether it is effective. However, these are intricately intertwined and hinder the acceptance of advice, so they are not always solved by a single method, and may be combined or switched depending on the situation. In presenting advice, "verbal communication" may be used exclusively, but at the same time, "nonverbal communication", which has a large intuitive influence, may be used. As mentioned above, whether or not to accept advice depends on the person's personality, propensity, tastes, and values, so check the person's situation and the person's (user's) reaction. It is desirable to grasp and switch.

As mentioned repeatedly, it is desirable to determine whether or not the user accepts the advice, and to classify the type of advice that the user is effective and leads to the action based on the determination result. In other words, even with the same advice, how to receive it differs depending on the user, such as the environment and situation in which the user is placed, the way of thinking, hobbies and tastes, and the way to overcome the restrictions also differs depending on the user. In the present embodiment, the user's reaction is dealt with in detail, the advice is customized, and the advice according to the user is presented.

The advice information selection unit 15c uses any of the advice types classified in consideration of individuality such as a user's personality and hobbies / preferences. In other words, if the user's reaction is not an absolute refusal, either assist-type advice, vision-type advice, attraction-type advice, imperative-type advice, etc., depending on the personality of the user. (See, for example, S19 to S25 in FIG. 6A).

When the advice information selection unit 15c performs emotion control, it may have an emotion control determination unit 15ca. The function of the emotion control determination unit 15ca does not necessarily have to be possessed by the mobile device, and an external server may have the function. Moreover, the external server to be linked does not necessarily have to be one type, and may or may not have an emotion control function.

When the emotion control determination unit 15ca presents advice to the user, for example, when using an avatar, the clothes, facial expressions, hairstyles, voices, backgrounds, etc. of the avatar (these may be called avatar items) are used. , Choose according to the type of advice. Considering the customization of this avatar, it is desirable to prepare various types, and when selecting the most suitable type for the user, it makes more sense to prepare it externally. Of course, if it is a simple switch, you may choose from the internal memory. For example, when giving assist-type advice (see, for example, FIGS. 3A (a), 4, 5A, etc.), the clothes and facial expressions should be familiar to the user, while the instruction type. When giving advice, authority, honesty, and politeness are more important than familiarity, and if necessary, the avatar should be dressed and facial expressions that make the user consider the direction not to refuse. Control the image of.

An avatar is generally an original character created on the Internet by freely selecting a face, hairstyle, clothes, belongings, etc., and means an avatar that moves exclusively in a virtual space as its own alter ego. In, we assume a displayable two-dimensional character that imitates a person who presents advice to the user, not his own alter ego. This avatar is used because it is easy to make it appear on behalf of a person by computer instructions as appropriate, and it may be 3D depending on the infrastructure that is necessary or acceptable, and something like a robot is actually given to the user. It may be something like talking to you. There may be a virtual character in which the figure or the like is not displayed and only the voice is used, but in this embodiment, such a character is assumed.

This is one of the ways to make the target user like the avatar and want to receive advice by changing the favorite character depending on the hobby and taste of the target user. This avatar is set to a specific gender and personality. However, under the control of the computer, the appearance of the avatar can be changed in various ways, for example, it is easy to mix hairstyles, clothes, voices, facial expressions, postures, gestures, halos, auras, speech bubbles, sound effects, etc. It is possible to fine-tune and display or express the age etc. according to the situation.

If such a secondary item is called an avatar item, it is easy to emphasize it and display it, and facial expressions etc. are displayed by changing not only emotions but also emotions. It is possible to do. An element that can be emphasized in this way may be called an item part of an avatar, or an element including an item may be called an avatar.

Such avatars and items are displayed and expressed in a basic manner (shape, prominence, face, posture) so that they can be switched and displayed and expressed according to the user's hobbies / preferences and specific situations in which the user is placed. , Expressions expressing personality expressed by how to speak, etc.) are data (image data and audio data) organized and classified by a database formula. Further, the database for storing avatars and items is organized and classified in a database format so that it can be switched and displayed according to the user's classification result or situation or the user's emotion.

In addition, the appearance and atmosphere of the avatar correspond to the latter when communication such as advice is classified into "verbal communication" and "nonverbal communication". As mentioned above, nonverbal communication also depends on the environment and culture in which the recipient of the advice was raised, the personality and tendency of the person, and the tastes and values. Therefore, the database for generating avatars and the like may be provided separately for each country, region, cultural area, and the like. Needless to say, even in the same culture, it may change depending on the generation, gender, hobbies and tastes. As information for generating a database, it may be read and classified from various behavior records and changes in biological reactions such as facial expressions. You may choose by yourself, or you can decide by Q & A.

In addition, items may also be selectively recalled from the database so that they can be selected from several patterns so that the character of the avatar can be decorated. However, the present invention is not limited to this, and when emphasizing according to the situation, the design may be made so that the characteristics of color, area, and shape can be emphasized by changing the numerical value of the variable, for example, by a program.

Depending on the type of avatar, the item part may bring different effects, may become unnatural, or may have adverse effects or cancel the effects. For example, even light makeup may be emphasized, and for a character like Our Lady, darkening the makeup will have the opposite effect rather than emphasis. Therefore, when expressing a combination of an avatar and an item, it is better to include prohibition processing or the like. That is, depending on the item, it may or may not be emphasized, and even if a person with a scared shaved head wears glasses, the degree of emphasis does not change. For this reason, avatars may be categorized and the effective items and the meaning of the emphasis effect may be sorted separately.

As explained above, the item to be recognized by the user is, so to speak, a transient item, and the avatar wearing the item maintains the corresponding identity and is an element that can identify some individuality even if it appears repeatedly. It is preferable to have a common design taste having. It is the same as if a famous actor plays a role by changing clothes, hairstyle, makeup, etc., he will not lose track of who is playing. The avatar wearing the item maintains the basic taste of the design, and at the same time, it fosters a sense of familiarity and trust through interaction with the user. However, it may eventually become incompatible, so in this situation it will be possible to identify the avatar (alternate) of a completely different character by changing the facial features, gender, clothing preferences, style, etc. Is preferable.

In addition, when changing the avatar (including item change) according to the situation, basically, the aim is to foster attachment and trust by changing the avatar based on the user's hobbies and tastes. However, the present invention is not limited to this, and switching may be performed according to the environment in which the user is placed, the reaction of the user, the content of advice, and the like. Of course, it may be a single substance or a plurality of groups. In particular, when considering the emotions of the other person, the facial expression, gesture, and voice change function of the avatar is effective, just as a human speaks to the other person by changing the facial expression, gesture, and voice color, thereby understanding and respecting the other person. It is possible to give advice in the above manner. Avatar with different modes classified into multiple types according to such situations and the emotions and reactions of the other party are organized and recorded in the recording unit so that they can be called and adopted by specific program control. .. These avatars (including items) may be appropriately selected and displayed from the recording unit (which may be in the terminal or outside the terminal, for example, in the cloud).

If the user's emotions cannot be determined from the beginning, even a human being may ask a question or talk to the other person to find out the reaction of the other person. Similarly, in the advice method in the present embodiment, the current situation of the user is determined, and based on the situation result, words equivalent to talking or asking from the database (here, since it is an advice method, advice). ) Is selected and the reaction is judged when presented to the user.

Assuming that this is the first advice, when the first advice is presented to the user, by having a step of selecting from different avatars classified into a plurality of types according to the determined reaction of the user. Appropriate response (such as avatar reaction and mode switching, next advice, etc.) is possible according to the user's reaction (emotionally and whether or not to give advice).

Here, since the attachment of the avatar itself of one's taste and the reaction to the gesture etc. are not particularly logical but emotional, it is effectively used for emotion adjustment, mediation, control, etc. It may be supposed to be done. However, since there are many scenes that appeal to other than emotions, the effects of avatars and items are not limited to emotions only. That is, as a reaction of the user, not only the avatar may be switched according to the emotion, but also the user's hobbies and tastes may be selected from the recorded results and utilized in a logical part other than the emotion. That is, the element (character identity) that enables mutual differentiation of avatars is determined by the judgment of the user's hobbies and tastes, and the item may be adjusted according to the user's reaction. Further, the avatar (including the item) to be presented may be changed not only by the reaction of the user's emotion but also by the content that the user mutters or speaks in the reaction.

Further, the expression "following the user's reaction" is expressed because there are many cases where the avatar mode that reduces the strong rejection reaction of the user and calms down and the avatar mode that arouses the positive emotion of the user are different types. For example, for a refusal, a formal or business response that expresses that you are facing the other party is preferable, and for an approval, a slightly broken, casual, friendly response is preferable. There are many. This is because it is easy to express how they agree with each other. The avatar character itself may be formal or casual, while the avatar character may be neutral and it may be possible to switch between formal and casual depending on facial expressions, clothes, hairstyles, etc. Here, the user's reaction is classified as refusal or approval, but other classifications may be used, and the classification may be appropriately changed according to the required specifications, performance, and functions, or a combination of multiple classifications may be used. May be good.

In the advice system that can be displayed by selecting from avatars with different modes classified into multiple types and items that modify the avatar, if the item for modifying the avatar is selected based on the result of judging the user situation. good. Here, the user situation is judged from the coordinate values of the graph having multiple axes or the items of the tabular database, and the ineffective advice is repeated or the effective advice is slightly changed and presented again. It makes it easy to switch such as. That is, the user status is determined from the coordinate numerical values of the graph having a plurality of axes or the items of the tabular database.

The avatar does not need to depend on the user's preference, and may determine the user's reaction when presenting the starting point advice to the user, and may be able to select and display from different avatars and items that attract unexpectedness and attention. These make at least a casual, formal, or personal or safe taste change to the advice in order to change the user's emotions in a positive direction. Since avatars appear many times and build a relationship of trust, it is preferable that, for example, those determined by the judgment of the user's hobbies and tastes appear more frequently than others.

In addition, the item is adjusted according to the reaction of the user, which may be data that can be commonly used by other avatars, and the avatar and the item can be independently classified and recorded in combination. Items can be highlighted, and items effectively decorate their avatars. The item is assumed to include at least one of clothes, hairstyle, background, style, posture, facial expression, voice, tone, background, makeup, aura, and halo that can modify the avatar.

Below, various user reaction classifications such as classification by emotion, acceptance at the time of advice, level of rejection, cause of rejection such as mental rejection, physical rejection, classification by origin, etc., or what kind of advice is easy for the user to accept It exemplifies what kind of response is to be taken according to the type of user.

The emotion control determination unit 15ca is not limited to the avatar, and when presenting advice to the user by voice, the emotion control determination unit 15ca may control the user's emotion by the tone of the voice (intonation or the like). In this case, it is advisable to use a voice synthesizer together. Since the data for the emotion control determination unit 15ca to present these images and the like is stored in the emotion control DB 16bd, an image and the like that can appropriately control the user's emotions according to the user's reaction and the like can be obtained. select.

As described above, the emotion control determination unit 15ca determines the user's propensity, hobbies, and preferences, determines emotions, and determines differences in response to advice. As mentioned above, there are various methods for classifying the difference in user's reaction, and the emotion control determination unit 15ca appropriately combines them according to the purpose and background of the advice, the environment in which the user is placed, and the situation. You can use it. For example, an advice to control emotions is required for a user who is difficult and emotional, but in order to determine the difficulty, the preference, disposition, and personality may be classified and judged, and the emotion for the advice may be determined. It may be judged by the way of change of, and does not have to be uniform. In addition, when emotions become intense, the basis for accepting logical proposals such as advice is impaired, and information transmission from the front may become impossible. In that case, classification such as whether or not emotions are involved, logical proposals after sedation of emotions, and various measures taken are interwoven and explained in the following examples.

The advice information selection unit 15c functions as an advice selection unit that selects one of the second advice classified into a plurality of categories according to the reaction of the user determined by the user reaction determination unit when the first advice is presented. (See, for example, S34 to S41 in FIG. 2, S17 to S25 in FIG. 6, S38 to S41a in FIG. 7, S73 to S85 in FIG. 9, and the like). The advice information selection unit 15c functions as a first advice selection unit that selects the first advice from the first database based on the user's situation determined by the user situation determination unit (for example, S31 in FIG. 2). See S5 in FIG. 6A and S31 in FIG. 7). The advice information selection unit 15c functions as a second advice selection unit that selects the second advice related to the first advice according to the user's reaction determined by the user reaction determination unit when the first advice is presented. (See, for example, S39 and S41 in FIG. 2, S19 to S25 in FIG. 6, S37 to S41 in FIG. 7, S59 to S63 in FIG. 8, and S75 to S85 in FIG. 9).

Again, when selecting advice, it is not necessary to select from uniformly categorized advice, and in the following explanation, approaches from different perspectives will be explained as appropriate with various classification concepts. First of all, as the most basic, it is conceivable to switch the advice depending on whether or not it is likely to be persuaded, but even if it is narrowed down to the case where it is likely to be persuaded, there may be various types of switching of advice.

The above-mentioned second advice selection unit determines whether or not the result determined by the user reaction determination unit is likely to persuade the rejection reaction, and if the result of this determination is likely to be persuasive, the second advice selection unit is used. (For example, S39 in FIG. 2, S13No to S25 in FIG. 6A, S37 in FIG. 7). Further, the second advice selection unit selects alternative advice when the result determined by the user reaction determination unit is unlikely to be convincing (see, for example, S13Yes and S15 in FIG. 6A). In this case, you may put off persuading with the initial advice, and first build a relationship of trust or communicate to see how the other person reacts. The second advice selection unit may select advice based on the above-mentioned mental idea of overcoming the physical condition, and in the case of physical refusal, the idea of not causing mental conflict or stress. The second advice selection unit selects the second advice corresponding to the mental refusal or the second advice corresponding to the physical refusal based on the judgment result by the user reaction judgment unit (for example, FIG. 7). S37 to S41 and S75 to S85 in FIG. 9).

The above-mentioned second advice selection unit determines one of a plurality of types based on the determination result by the user reaction determination unit, and selects the second advice (for example, S39, S41 in FIG. 2, FIG. 6A S19 to S25, FIG. 7 S37 to S41, and FIG. 9 S73 to S85). The advice information selection unit 15c functions as an advice selection unit that selects the second advice according to the user's reaction determined by the user reaction determination unit when the first advice is presented (S19 to S25 in FIGS. 6A, FIG. 7 from S37 to S41, FIG. 8 from S59 to S63, and FIG. 9 from S73 to S83). The advice information selection unit 15c functions as a priority unit that prioritizes one of the classified advices according to the advice presentation target person and the content of the advice (S19 to S25 in FIG. 6A, S37 to S41 in FIG. 7, and S59 to FIG. 8). S63, S73 to S83 in FIG. 9).

As described above, the user reaction determination unit 15a determines the stress of the user, and the address information selection unit 15c may select advice based on the stress determined by the user reaction determination unit 15a. Specifically, as will be described later with reference to FIG. 8, the advice information selection unit 15c selects the advice belonging to the one with the smaller stress among the stresses determined by the stress determination unit as the advice to the user. It functions as an advice selection unit (see S59 to S63 in FIG. 8).

The advice information output unit 15d outputs the advice information selected by the advice information selection unit 15c to the display unit 12. Alternatively, instead of the display unit 12, the advice information output unit 15d outputs to a device used by the user, such as the second information device 20, through a communication unit such as the device communication unit 18.

Further, the advice information output unit 15d has an emotion control output unit 15da. The emotion control output unit 15da presents advice and the like to the display unit 12 and the like based on the information (data) for controlling the emotion of the user selected by the emotion control determination unit 15ca. That is, when the advice is output, the avatar may also be displayed, and it may be possible to display the user as if he / she is interacting with the user. When displaying the avatar, the avatar may be changed based on the user's preference, operation history, and the like. In the present embodiment, the advice is visually displayed by the display unit 12, but the advice may be presented to the user by another method such as voice. Of course, controlling the user's emotions by advice is not limited to visual advice, but may be performed by voice.

The advice information output unit 15 functions as an advice presentation unit that presents advice to the user (for example, S5, S21 to S24 in FIG. 6A, S31, S39, S41 in FIG. 7, S51, S55, S61 in FIG. 8). , S63). Further, the advice presenting unit presents the first advice to the user by the avatar (see FIGS. 4B and 5B). The above-mentioned avatar changes the above-mentioned avatar according to the user's preference or operation history.

Further, the advice information output unit 15 functions as a first advice presentation unit that presents advice belonging to the first category to the user and a second advice presentation unit that presents advice belonging to the second category to the user (for example). , See S51 and S55 in FIG. 8). The first advice presentation unit presents advice corresponding to mental refusal as the first category, and the second advice presentation unit presents advice corresponding to physical refusal as the second category (for example,). See S59 to S63 in FIG. 8). The advice information output unit 15d (or the advice information selection unit 15c, the emotion control output unit 15ca) functions as an emotion control unit that controls emotions based on the emotions determined by the user reaction determination unit (for example, FIG. 3). See emotional control).

The recording unit 16 is a memory for recording various information, and has a user action history recording unit 16a and an advice information recording unit 16b. The user behavior history recording unit 16a records the behavior of the holder of the mobile terminal 10. User actions include, for example, a history of displaying advice by the mobile terminal 10, a history of reactions to advice, a history of actions recorded in a schedule, a history of detection results detected by the sensor cooperation unit 13, and a history of detection results. For example, when equipped with GPS or the like, there is a user's movement record or the like. In "nonverbal communication", which has a great intuitive influence, what kind of effect is effective depends on the personality and tendency, taste and values of the recipient. Therefore, the behavior history analysis may be accumulated to determine the factors that affect the person (the type of the person). However, it should be noted that there are some parts that cannot be decided uniformly depending on the situation.

The advice information recording unit 16b records a plurality of advices to be presented to the user. As described above, the advice information selection unit 15c selects an appropriate piece of advice information recorded in the advice information recording unit 16b. The advice information recording unit 16b has an advice information group 0 (16ba), an advice information group 1 (16bb), an advice information group 2 (16bc), and also has an emotion control database DB 16bd. These advice information groups and the like function as an advice database in which specific advice is classified into an efficacy display type and a sensory appeal type in advance and stored.

In the advice information group 0 (16ba), a plurality of general advices (referred to as advice 0) presented to the user by the mobile terminal 10 according to the user's situation are recorded (S31 in FIG. 2, FIG. 4, FIG. 5A, FIG. 5B, S5 of FIG. 6A, S31 of FIG. 7). That is, general advice is recorded in association with the user's situation. The advice recorded in the advice information group 0 is first presented when it is determined that the user needs advice (see S3 and S5 in FIG. 6A). Further, the advice recorded in the advice group 0 is also presented as an alternative to the advice when the advice is presented to the user and an absolute refusal is shown.

When presenting advice to the user, the emotion control DB 16bd has data used to control the user's emotions, for example, the clothes, facial expressions, and hairstyle of the avatar, in addition to the advice selected from the advice information groups 1, 2, and so on. It is a database that records data related to such things. The emotion control determination unit 15ca and the emotion control output unit 15ca use the advice selected from the advice information groups 1 and 2 in combination with the emotion control data selected from the emotion control DB.

The advice information group 0 functions as a first database that associates the relationship between the user's situation and the first advice (for example, advice 0 in FIGS. 4A and 5B, S5 in FIG. 6A, FIG. 7).

In the advice information group 1 (16bb) and the advice information group 2 (16bc), a plurality of advice 1s to be presented to the user are recorded next according to the reaction of the user when the advice 0 is presented to the user. (See FIGS. 4 and 5A, 5B assisted advice, vision type advice, attraction type advice, imperative advice, S21 to S25 in FIG. 6A, and S39 in FIG. 7). Although only the advice information group 1 and the advice information group 2 are displayed as the memory for storing the advice 1 in FIG. 1A, three or more advice information groups may be provided for each type of the advice 1.

The advice information group 1 (16bb) and the advice information group 2 (16bc) function as a second database that stores the relationship between the user's reaction and the second advice in association with each other (for example, FIGS. 4A and 5A). , Assisted advice in FIG. 5B (a), vision type advice, attraction type advice, imperative advice, S19 to S25 in FIG. 6A, S39, S41 in FIG. 7, S61, S63 in FIG. 8, S79, S81 in FIG. , S85, etc.). The second database stores the second advice depending on whether it is a mental refusal or a physical refusal (see, for example, S37 to S41 in FIG. 7 and S75 to S85 in FIG. 9). The second database is divided into a plurality of types and stored according to the user's situation (for example, S19 to S25 in FIGS. 4, 5A3 and 6A, S37 to S41 in FIG. 7, and S59 in FIG. 8). -S63, see S73-S83 in FIG. 9).

The second advice stored in the advice information group 1 (16bb) and the advice information group 2 (16bc) is divided into the content that appeals to the logical profit and the content that appeals to the sensory temptation. In presenting the advice of 2, the distribution of contents is different (see, for example, sensory and rational in FIG. 3A (a)).

The second advice stored in the advice information group 1 (16bb) and the advice information group 2 (16bc) is divided into a content for forcing the user and a content for guiding the user, and the second advice is given to the user. In presenting, the distribution of content is different (see, for example, compulsory and inductive in FIG. 3A (a)).

The second advice stored in the advice information group 1 (16bb) and the advice information group 2 (16bc) is divided into the content with assistance and the content without assistance, and presents the second advice to the user. In doing so, the distribution of content is different (see, for example, with and without assistance in FIG. 3A (b)).

The second advice stored in the advice information group 1 (16bb) and the advice information group 2 (16bc) includes a plurality of proposals having different additional information, and the user can select from among them.

The net communication unit 17 has a communication circuit and communicates with an external server or the like through various communication networks such as a mobile phone network and WiFi. The device communication unit 18 has a short-range communication circuit and communicates with a device existing in the vicinity of the mobile terminal 10. In this communication, for example, the advice information in the mobile terminal 10 is transmitted from the mobile terminal 10 to the second information device 20 as the advice transfer information 30 for handing over to the second information device 20.

The second information device 20 is an example of a device that inherits advice information from the mobile terminal 10, and is, for example, a camera. The second information device 20 includes an image pickup unit 21, an operation unit 22, a display unit 23, a control unit 24, a communication unit 25, and a recording unit 26.

The image pickup unit 21 has an optical lens, an image pickup element, an image pickup element control circuit, an image pickup signal processing circuit, and the like, and converts an image into image data. The image pickup unit 21 may acquire a subject image in response to an instruction from the operation unit 22. The second information device 20 may acquire an image in cooperation with an image pickup unit or the like in an external device such as a mobile terminal 10. Further, when the second information device 20 does not require an image pickup unit, such as a device in a hospital or an inspection institution, the image pickup unit 21 may be omitted.

The operation unit 22 has various switches and buttons provided in the second information device 20, operation members such as a touch panel, and sensors for detecting the operation state of these operation members. The operation status signal of the operation member detected by the sensor is output to the control unit 24. The operation unit 22 functions as an interface for inputting user instructions and the like.

The display unit 23 has a display for display and the like. On this display, a menu screen for setting an image acquired by the image pickup unit 21, a reproduced image based on the recorded image recorded in the recording unit 26, a mode of the second information device 20, and the like is displayed. Further, various information such as takeover advice information output from the mobile terminal 10 is displayed on the display unit 23.

The control unit 24 is a processor having a CPU (Central Processing Unit), a memory for storing a program, and peripheral circuits. The control unit 24 controls the entire unit in the second information device 20. The control unit 24 includes an advice information selection unit 24a, a display control unit 24b, a communication control unit 24c, and a recording control unit 24d. Each of these parts is realized by software by the CPU in the control unit 24, and a part is realized by the hardware circuit in the control unit 24. However, the present invention is not limited to this, and all may be realized by software, and all may be realized by hardware circuits.

The advice information selection unit 24a selects appropriate information from the advice information 30b in the transfer information 30 inherited from the mobile terminal 10. That is, the advice information selection unit 24a selects advice related to the advice presented to the user selected in the mobile terminal 10. The display control unit 24b controls the image to be displayed on the display unit 23. The advice selected by the advice information selection unit 24a is displayed on the display unit 23. The communication control unit 24c controls communication with the mobile terminal 10 by the communication unit 25. The recording control unit 24d controls the recording of image data and advice information in the recording unit 26.

In the control unit 24, like the control unit 15 in the mobile terminal 10, the user reaction determination unit 15a, the emotion determination unit 15aa, the non-emotion determination unit 15ab, the advice information selection unit 15b, the emotion control determination unit 15ba, Of course, the same operation as that of the mobile terminal 10 may be performed by providing all or a part of the advice information output unit 15d, the emotion control output unit 15da, and the like.

The communication unit 25 has a short-range communication circuit and communicates with a device existing in the vicinity of the mobile terminal 10. In this communication, as described above, for example, the advice information in the mobile terminal 10 is transmitted from the mobile terminal 10 to the second information device 20 as the advice transfer information 30 for handing over to the second information device 20.

The recording unit 26 is an electrically rewritable non-volatile memory. The recording unit 26 has an image recording unit 26a and an advice information recording unit 26b. The image recording unit 26a records the image data acquired by the image pickup unit 21. The advice information recording unit 26b records the advice transfer information 30 received from the mobile terminal 10. When each unit similar to the control unit 15 of the mobile terminal 10 is provided in the control unit 24, the advice information recording unit 16b in the mobile terminal 10 may be included in the advice information recording unit 26b as necessary. Record similar data.

The hand-over information 30 is information for handing over the advice information provided to the user in the mobile terminal 10 to another device (in the example shown in FIGS. 1A and 1B, the second information device 20). At first, even if the user is using the mobile terminal 10 and is providing advice, the user then uses another device, and in that case, the mobile terminal 10 is used. It may be better to continue the advice as an extension of the case. The advice transfer information is information for continuously providing advice even in such a usage state. As described above, the advice information selection unit 24a selects the advice to be presented to the user based on the inheritance information 30.

The advice takeover information 30 has a user state reaction history 30a and advice information 30b. The user state reaction information 30a is a user reaction history recorded in the user action history recording unit 16a. The user reaction history is a history related to the user's reaction when the advice information is presented to the user. The advice information 30b is the advice information provided to the user, and may further include the advice information that may be provided to the user.

In the present embodiment, the second information device may be a camera having an imaging unit, or may be a medical device such as an endoscope, an X-ray imaging device, a blood test device, a test device, or the like. After presenting the advice to the user on the mobile terminal 10, any device capable of taking over this advice and presenting another advice to the user can be used as the second information device. Further, the user who presents the advice by the second information device 20 is not limited to the user who uses the mobile terminal 10, and may be a user who uses the second information device 20.

When the second information device 20 is a camera, it may be a digital single-lens reflex camera, a mirrorless camera, a compact digital camera, a video camera such as a video camera or a movie camera, and a personal computer ( PC), tablet computer, camera built into game equipment, medical camera (for example, medical endoscope), camera for scientific equipment such as microscope, industrial endoscope, camera mounted on automobile, monitoring It doesn't matter if it's a camera.

Next, the behavioral assisting operation of presenting advice to the user will be described using the flowchart shown in FIG. In this flow, the advice to be presented next is changed according to the reaction when the mobile terminal 10 presents the advice 0 to the user. This flow is mainly realized by the control unit 15 in the mobile terminal 10 according to the program stored in the memory. However, if the advice information is taken over by a device such as the second information device 20 after the operation on the mobile terminal 10, the flow after the takeover is the control unit in the device that has taken over the information such as the second information device 20. Will be realized.

Through the action assistance flow shown in FIG. 2, first, advice 0 is presented to the user (S31). Here, the advice information selection unit 15c selects the advice 0 to be presented to the user first, and the advice information output unit 15d displays the selected advice 0 on the display unit 12.

Next, the emotional bias in the user reaction is determined (S32). Here, what kind of emotion is expressed by the emotion determination unit 15aa in the user reaction determination unit 15a among the user's reactions when the advice 0 is displayed, and the strength (emotion bias) of the emotion is determined. Determine how much it is. That is, when the mobile terminal 10 presents the advice 0 to the user (see step S31), it may not be easily accepted depending on the situation and the content of the advice. In this case, some intention of refusal is made. Is shown. In this step, the user reaction determination unit 15a determines whether or not some kind of refusal is made, what kind of emotional bias is applied if the refusal is made, and whether or not advice 0 is accepted. In addition, if any refusal is made, advice on the cause of the refusal, such as whether the refusal is an emotional refusal or a rational refusal, will be given smoothly and continuously from the next time onward. To be able to do.

If the user's behavior pattern and behavior content are determined on a daily basis, the user's values, thinking style, behavior style, hobbies, preferences, etc. can be determined to some extent. The user's reaction to the advice 0 in step S31, or the reaction under the situation where the advice 0 is received, is important information from the viewpoint of specificity and situation detection when the next advice is given following the same advice 0. It turns out that.

The reaction to this advice 0 includes the voice and complexion of the user when the advice 0 is seen, various biological reactions, and the like. Further, the questionnaire may be presented at the same time as the advice 0 is presented, and the user's answer at that time can also be used as the answer to the advice 0. In this way, various types of information can be considered for the reaction to advice 0, but in this flow, the type of "emotion" that particularly influences human behavior, serves as a criterion for judgment, and is easy to explain. The parameters are taken up in particular, and they are measured and reflected in subsequent responses. Research on the detection of "emotions" is progressing, and by using "emotions" as a representative example of "nonverbal communication" or as an element of the background in which communication occurs, a system with specifications that are easy to explain will be created. Can be done. That is, when the user refuses (in the case of Yes in step S33), the subsequent response can be smoothly proceeded by determining what kind of emotion the user refused.

After determining the reaction of the user in step S32, it is next determined whether or not the reaction is rejected based on this determination (S33). In this step, based on the determination in step S32, it is determined whether or not the user rejects the advice 0.

If the result of the determination in step S33 is not rejected, the action assistance flow is terminated (S35). In this case, since the user has accepted the advice 0 presented in step S31, it is not necessary to continuously give advice, so that the action assistance flow is terminated. Before ending the flow, a supplementary explanation of Advice 0 may be given.

On the other hand, if it is rejected as a result of the determination in step S33, the mode reflects the emotional bias when the advice is presented (S34). Since the user's emotional bias is determined in step S32, selection is made to give advice in a manner that reflects this emotional bias. For example, when using an avatar to present advice to a user (see FIGS. 4B and 5B), the avatar's clothing, facial expression, hairstyle, voice, background, gender, etc. (these are). It may be called an item) depending on the user's emotional bias (and user type). You may also select the type of advice as described below with reference to FIG. 3A.

After selecting an aspect that reflects emotional bias, it is next determined whether or not the rejection reaction is likely to be persuaded (S37). Here, based on the determination result in step S32, it is determined whether or not the user is likely to be persuaded to the rejection reaction. As described above, the emotion when the user refuses is determined, and based on this determination result, it is determined whether or not the user is likely to be persuaded.

In addition, the user's reaction may include elements other than emotions, for example, elements that can be judged by converting the user's words into text (elements of linguistic communication), gestures, etc. (emotional expressions). (A certain language communication) is also included, so such information may also be used.

It is also better to determine whether the emotion is due to advice or something else. By comparing the reactions before and after giving advice, it is possible to determine whether or not it is caused by emotions. For example, whether or not there was a severe refusal can be judged by whether it is left to anger or irritation or embarrassed.

It is also possible to determine whether or not it is rejected on a text basis. If you are embarrassed, you may be persuaded by presenting the correct information next, and you may accept Advice 0 by giving advice that encourages the user's behavior. However, in many cases, even a conversation cannot be established if the severe irritation does not subside. That is, in step S37, the advice sequence branching is performed by seeing this rejection reaction.

As a result of the determination in step S37, if the rejection reaction is likely to be persuaded, the advice of the advice series 1 is presented to the user (S39). On the other hand, if it is unlikely that the rejection reaction can be persuaded, the advice of the advice series 2 is presented to the user (S41). As described above, in the advice information recording unit 16, in addition to the advice information groups 1 and 2, the advice information groups 3, 4, ... May be stored. In this case, the advice series 1 The advice information group may be appropriately distributed according to the advice series 2. Further, in the flow shown in FIG. 2, the advice series has two branches, but the branch is not limited to this, and may be three, four, or more branches. Further, when presenting the advice, the advice may be presented not only by simply displaying the text but also by using an avatar or the like (see, for example, FIGS. 4B and 5B). At this time, the hairstyle, facial expression, etc. (item) of the avatar may be changed according to the reaction of the user.

In the flow shown in FIG. 2, it was simplified and branched into two, but for example, in step S39, since it seems to be persuasive, it is a type that does not require a procedure such as calming the feeling of anger, and further explanation is given. It may be text or voice. Alternatively, you may continue to give advice with gestures such as avatars. On the other hand, in step S41, since it is unlikely to be persuaded, first of all, it may be a little detour advice such as calming emotions. Feelings of anger may be caused by some misunderstanding. In this case, the advice may be similar to the advice series 1, and more safely, the advice 0 itself may be withdrawn. In many cases, it is better to change to the story and give advice according to this revised content.

In addition, if the emotion is positive (plus), advice is given in the direction of continuing and maintaining that emotion, while if the emotion is negative (minus), it is accompanied by an effect that alleviates the negative (minus) emotion as much as possible. Is preferable. According to the emotion detected in step S32, the emotion control mode may be reflected in step S34. This can also be considered as advice output (or advice selection).

Further, in the flow shown in FIG. 2, it is described so as to simplify and end the flow when the advice by the advice series 1 or the advice series 2 is presented to the user. However, instead of ending the flow, starting from the presentation of the advice in step S39 or S41, the advice becomes 0 in step S31, and the next advice may be continued.

By repeating the presentation of such advice and the subsequent determination of emotional bias, it is possible to determine what kind of advice should be presented and what kind of emotion the user will have. By repeating these steps, it becomes possible to create a database of the relationship between individual advice and emotional changes until the user accepts the originally intended advice. If you make this database searchable or use it as teacher data for learning, it will be helpful for presenting advice. For example, it is possible to create an inference model that outputs optimal advice for the user's current emotions.

Further, the advice (second advice) in step S39 or S41 is classified into a content that appeals to profit logically and a content that appeals to sensory temptation, and the distribution may be different. Further, the above-mentioned second advice is classified into the content for forcing the user and the content for guiding the user, and the distribution thereof may be different. Further, the above-mentioned second advice is classified into the content with assistance and the content without assistance, and the distribution thereof may be different. In addition, the above-mentioned second advice includes a plurality of proposals having different additional information, and the user can select from among them.

Next, with reference to FIGS. 3A and 3B, the advice to be presented again (also referred to as the second advice) will be described according to the reaction of the user after the advice is presented.

In the present embodiment, for example, when presenting event starting point advice, which is an event starting point that does not start unless the user takes an action, to the user, a user reaction determination for determining the user's reaction is performed, and the reaction is determined. Based on this, the advice selection is made to select one of the following advices classified into multiple categories, so it is desirable to classify the advice reasonably. For example, after giving ineffective advice, the user rejects it, but even if you give similar advice, it is not accepted, so select the type of effective advice and guide further users. Must. Therefore, from the advice group classified by extremely thoughtful rationality, select the one that best suits the user, and the advice does not cause the user to feel a psychological burden, and the user takes action without difficulty and also acts. After the occurrence of the above, we have devised ways to satisfy the user.

Rational classification means that it is possible to respond to a user's reaction in consideration of the user's taste, tendency, personality, aesthetics, etc., and also in consideration of changes in the user's emotions. Because of this, we treat them together and approach them from both emotional and rational perspectives. In addition, emotions and reason have a large mental element, but in reality, there are restrictions and restrictions (physical elements) due to physical factors, and it may not be possible to respond to advice, so in such a user's situation By conducting various analyzes and responding to it, it is possible to select an effective advice method.

However, even if there are physical restrictions, it may be possible to move on to the next step of taking various alternative measures with mental motivation, so first of all, let's raise the user's independent awareness. I attach great importance to it. Of course, in the event achieved by the user's actions, various conditions may be required to reach the event, and in this case, a measure to move forward even if the detour is considered. Some of these measures take a long time, and physical constraints often change during that time. However, even in this case, if the mental motivation can be maintained, the probability that the target event can be voluntarily participated in the event will increase.

Therefore, in the explanation here, when divided into physical and mental, there are many explanations of how to approach the mental part such as personality, way of thinking and emotion. In addition, depending on the service mode to be directed, the infrastructure available, and business collaboration, we may give advice on building physical strength to those who are not confident in their physical strength, or act on their behalf for those who do not have time. When there is no money, it is possible to propose physical support as appropriate, such as giving advice on saving money.

First of all, when considering the personality of the user, if the person has the same personality in any situation in daily life, he / she does not behave in the same way, and he / she must also consider the current situation of the person. .. What kind of advice should be presented to the user so that the user can accept it reasonably is not uniformly determined depending on the individuality of the user and the situation of the user, but it is shown in FIGS. 3A and 3B. Explain the idea of deciding advice from the perspective shown.

That is, in FIG. 3A (a), it is considered that what kind of information presentation is easy to attract the person according to the personality, tendency, way of thinking, values, etc. of the user, and reason vs. sense, forced vs. I am deciding on advice from the perspective of guidance. In the part represented by the vertical axis, it is classified as to whether it has high empathy for theory or whether it is easier to take action with feeling than theory. Depending on which part of this axis it corresponds to, decide whether to include the logical part or the sensory enjoyment in the content of the advice.

The above-mentioned tendency can be inferred from the user's occasional behavior to determine which tendency is stronger, and when giving specific advice, the question "what is the meaning?" Is asked. It turns out that the type is naturally different between a person and a person who asks "what is fun?" Of course, it may vary depending on the content and circumstances of the advice.

For example, a scene such as "I know the theory but dislike it" is not the user's tendency or personality, and any person does not need any further theory and needs to take care of the feeling of "dislike". .. What you should be aware of here is that if you give a more rational explanation to a person who understands the reasoning, you will get a rejection reaction. That is the meaning of rationally classifying and providing correct and qualified information.

In other words, in order to reduce "dislike", the theory that appeals to the user's left brain is sufficient, and rather attractive information that overturns the sensory prejudice that can be said to be right-brain is useful. That is, on the vertical axis in FIG. 3A (a), advice conscious of the direction displayed as "sense" is useful. In the opposite case, rational and rational advice is useful. Of course, if you don't understand the reason and somehow dislike it, the advice that balances the time will work, so you can present the advice that is positioned in the middle of this axis.

Another way of looking at and using the graph shown in FIG. 3A (a) is to resemble a user-approach necessity graph in which both sides of the axis represent the strength of the need for some kind of allowance. There is also a method. In this case, the idea is that the user located in the "forced" does not need the "forced" by the "forced" and shifts to the coordinates closer to the center. Similarly, users who need "guidance" can plot on the right side and repeat the "guidance" advice to gradually approach the center where guidance is not needed. Not only the horizontal axis but also the vertical axis can be used in the same way. It is assumed that the user who is in a state where the theory is already good or the charm is well communicated approaches the center.

By plotting the user state in this way, if the user is monitored appropriately and the user state is brought to the state in the middle of the graph, the reasoning and emotions will be neutral, and there will be no coercion or incentive. It means that you may decide the coordinates, which is considered to be a situation where you act voluntarily.

By monitoring various advices and their effects, users can be asked to take specific actions. At this time, the guidance type avatar and the compulsory type avatar are different, and therefore, an element capable of switching the degree of the appealing aspect by quantification is required. For example, the avatar may be modified by an item whose numerical value can be set (including presence / absence, 1 or 0). In the compulsory aspect, the facial expression is serious and angry, and in the inductive aspect, the facial expression is bright and smiling.

This can generally be expressed as a different number. You can set 10 for a big smile and 0 for terrible anger, for example, 8 basic emotions known as the "Prutic's Circle of Emotions", irritation, tiredness, melancholy, absentmindedness, anxiety, forgiveness, peace, and interest from 1. It may be associated with the number 8. It is said that if these eight emotions are strengthened, it becomes anger, disgust, sadness, surprise, fear, trust, joy, and expectation, and this is closer to what is called emotions. This may be changed from 1 to 8. By increasing this number, negative emotions can be made positive, and based on this, the avatar will support the user and provide guidance. Here, various tactics are required while observing the reaction of the user, and the concept thereof will be described later with reference to FIG. 3B.

Simply, you can decide the next advice from the previous reaction. Even when presenting avatars, avatars that explain the meaning of advice should be in a reliable business style, and avatars that explain fun should be in a casual style in order to bring out the fun. You can devise ways to create an atmosphere and environment that makes it easier to hear the content of the advice. By performing such control, the effect of the advice becomes large.

In the example shown in FIG. 3A (a), another axis (horizontal axis) is prepared from a different perspective from reason and sensibility, and the user can also use the inductive and compulsory axes as in the metaphor of the north wind and the sun. We are devising ways to deal with the rejection of the above, considering how it can be overcome. With this ingenuity, the user's reaction can be classified in more detail from another perspective, so that a finer response or a more diverse and effective approach becomes possible. In other words, it is possible to guide by reason, to boost by reason, to attract sensuously, and to give a sensuous boost that something unpleasant is likely to occur if you do not act. Again, what works may be determined by the personality of the user, and the effect may change depending on the situation at that time.

If the user's reaction is, for example, just wanting something to trigger, then the advice to support it is good, and if it doesn't feel like it at all, a method that naturally induces it is preferable. In addition, there are stages in causing the same action, and it is better to be able to grasp which coordinate of this graph the advice corresponds to each time. This facilitates feedback, and the advice that can be used can be devised such as giving advice corresponding to similar coordinates or changing the coordinates slightly from there. It is also possible to control not to repeat the advice that a bad reaction appears by accurately grasping the coordinates.

Further, in this way, in addition to the taste of the basic advice to the user, another axis may be created and the three-dimensional coordinates may be used to classify the appropriate advice to the user. Here, there is also a way of whether the person who gives advice (the service provider who appears as an avatar) provides some kind of assistance to the user. This method assumes that the user may be more likely to take action with actual help. Since this method involves some active action, it cannot be determined only by the user's isolation. In FIG. 3A (a), at each place on the graph, the expression "assist type (or assist presentation type, assist accompanying type)" is used to induce assist (what can be done to get more) or force. Assist (if you don't do anything, there will be some further penalties), rational assist (such additional tests will make the need clearer), sensory assist (what makes you happy), etc. A cross-shaped arrow suggesting that there may be assistance with assistance that emphasizes the axis is also shown. This type imposes a burden on the assisting side, so it is a different approach. Depending on the specifications of the service, it may be a natural burden, and the user may be charged a fee.

This assistive action can be said to be more customized advice, and it is easier to understand if you can see the symbolic figure of the subject who responds and assists individually, so there is a lot of room for avatars to play an active role. In addition, when this assistant starts an action, boosting, withdrawing, or guiding is a continuous action that is flexible and difficult to separate finely. In other words, the cut with or without assistance is the more dominant factor than the separation between guidance and coercion. Considering that there is a big difference between the case where assistance is possible and the case where assistance is not possible, as shown in Fig. 3A (b), there is no assistance for sensory and rational separation (this is because the horizontal axis in Fig. 3A (a) is assistance It is a rational advice classification to organize them in a two-dimensional way and show what to do with these formulations. For example, in the case of an optional paid customization service, it is easier to explain with the coordinates of (b) than with (a). Thus, in FIG. 3A (b), advice is determined from the viewpoint of with assistance vs. without assistance.

In this way, the avatar character that can be said to be a symbol of the virtual personality of the subject who presents the advice, and the item that modifies it, needs to switch the impression given to the user depending on the situation, how to treat the user, and the content of the advice. Therefore, in FIG. 3A (c), items such as the personality (character) and clothes of the avatar to be presented can be decided from the viewpoint of business vs casual and individuality vs safe, and can be presented according to the above-mentioned situation. It illustrates that.

This can show the degree of deviation from formal and formal things such as casual and business (formal), which are often problematic in workplace clothes, and those with the same sense of unity like uniforms (safety). The deviation from (individuality) is represented by two axes, but of course, the taste of finer deviation may be considered.

In FIG. 3A (c), the solicitation system should be more attractive in the casual and individual direction, and the rational system should be more formal and uniform-like to increase the persuasiveness and reliability. .. However, depending on the person, and in some cases, it may be better to consider the case where it is not always possible to follow a casual person. Such adjustment may be performed according to the reaction of the user. If you are open to the first advice, you may want to show your personality.

Such a concept of finer division is illustrated in FIG. 3B by isolating the user's reaction and emotion. In this example, as a demanding technique for controlling emotions, the effect of an avatar that gives an intuitive first impression is further effectively utilized and illustrated. That is, advice that avatars classified into multiple types and items that modify the avatars are recorded in the recording unit, and the avatars (including item modification) can be displayed by appropriately selecting from the recording unit. Can provide a method. In addition, selection is made based on the result of determining the user situation. Further, as described above, the user situation is determined as the coordinate numerical value of the graph having a plurality of axes, and advice is given so that the situation can be changed in a positive direction, and an avatar is used for that purpose.

First, the type of advice presented to the user will be described with reference to FIG. 3A (a). When the mobile terminal 10 presents advice 0 to the user, the reason for rejecting this advice may be that it may not be possible to act according to the advice presented for physical reasons, and it is mental (emotional). ) You may not be able to follow the advice for some reason. Physical and emotional reasons can be intricately intertwined.

In the case of a physical problem, it is often possible to solve the problem or by alternatives. On the other hand, in the case of an emotional problem, it is highly possible that the user can be persuaded by changing the advice according to the mental state of the user.

When presenting advice to the user, in the present embodiment, as shown by the cross-shaped arrow A in FIG. 3A (a), the types of advice are divided into four indexes and classified according to these indexes. .. That is, take instinctive and sensory advice for rational advice in the vertical direction of arrow A, and take inductive advice for compulsory advice in the horizontal direction of cross arrow A. There is.

Reasonable advice is left-brained, rational and rational, such as "should" and "should not". On the other hand, instinctive and sensory advice is right-brained and appeals to human sensibilities such as "good" and "no". Also, compulsory advice is coercive and threatening advice. On the other hand, inductive advice is profit-inducing or soothing advice, and includes voluntary behavior of the user.

Further, the cross-shaped arrow B in FIG. 3A indicates the clothes, hairstyle, background of the person presenting the advice such as an avatar, and other than these, for example, the facial expression, voice, gender, etc. may be changed. Since these visual elements have not only mental aspects but also physical aspects, they should be considered separately from the dimensions shown by the cross-shaped arrow A.

In this embodiment, various types of advice as shown in FIG. 3A (a), that is, assist-type advice, vision-type advice, attraction-type advice, and imperative-type advice are prepared to respond to the user's reaction. Depending on the situation, I choose from these advices as appropriate.

Assisted advice is advice that is close to the user and assists in achieving the purpose. This advice suggests that the user cannot think of or cannot think of it individually, and is effective for people who "relieve when they meet someone". Assisted advice is rather rational in FIG. 3A (a), and is neutral or somewhat compulsory in terms of compulsory or inductive advice, but depending on the content of the advice. It can also be inductive.

Vision-based advice is advice that shows future success stories and makes users imagine a successful vision. This advice embodies what happens when you respond to a proposal and is useful advice for realists. Vision-type advice is rather rational and inductive advice in FIG. 3A (a).

The attraction-type advice is advice that stimulates emotions, and for example, makes a proposal that eliminates the anxiety felt by the user. This advice is useful for anxious people. In FIG. 3A (a), the attraction-type advice is rather instinctive, sensory, and inductive advice.

Imperative advice is high-pressure advice and is effective advice for people with weak will. In FIG. 3A (a), imperative advice is rather neutral in terms of rational or instinctive / sensory, and is compulsory advice.

FIG. 3A (b) may assist the user in giving advice. In this case, as shown in FIG. 3A (b), it is sensory or rational in the horizontal direction, and with or without assistance in the vertical direction. The advice may be classified according to this position. However, both with and without assistance are not limited to mental elements, but also have strong physical elements, and advice cannot be generally classified from the viewpoint of mental or physical.

As shown in FIG. 3A (a), the advice consists of a plurality of classified advices (second advice). Here, this second advice is divided into content that appeals to profit logically and content that appeals to sensory temptation, and the distribution is different. For example, as shown in FIG. 3A (a), it is divided into vision-type advice, attraction-type advice, assist-type advice, and imperative-type advice depending on the position of rational, instinctive, and sensory in the vertical direction.

Further, the second advice is divided into a content for forcing the user and a content for guiding the user, and the distribution thereof is different. For example, as shown in FIG. 3A (a), it is divided into vision-type advice, attraction-type advice, assist-type advice, and imperative-type advice depending on the lateral compulsory and inductive positioning.

Further, the second advice is divided into the content with assistance and the content without assistance, and the distribution thereof is different. For example, as shown in FIG. 3A (b), it is divided into vision-type advice, attraction-type advice, assist-type advice, and imperative-type advice depending on the positioning with and without assistance in the vertical direction.

Further, the second advice is divided into the content corresponding to the physical refusal and the content corresponding to the mental refusal, and the distribution contents are different. If you are rejected for physical reasons and if you are rejected for mental reasons, be prepared with advice to eliminate the reasons for each refusal.

FIG. 3A (c) is a diagram showing the classification of types when the user's emotions are controlled by an avatar, wording, or the like. The horizontal direction of FIG. 3A (c) indicates the degree of business or casual, and the vertical direction indicates the degree of individuality or safety. Taking clothing as an example, business clothing corresponds to a combination of jacket and pumps, while casual clothing corresponds to a combination of T-shirt and petanko shoes. In addition, unique clothes correspond to bright colors, patterns, and designs, while safe clothes correspond to basic colors and basic design clothes. When presenting advice to the user, it is advisable to select the avatar's clothing according to the user's emotions while considering the contrast between business vs. casual and individual vs. safe.

Next, with reference to FIG. 3B, what kind of advice is presented according to the emotion of the user will be described. Humans have eight major emotions (introduced as the emotional circle of psychologist Robert Plutchik), which are identified by user reactions such as facial expressions and voices, biological reactions, and questionnaire results. It can be classified by the logic and inference model of. In addition, the strength of emotions (emotion bias) can also be determined.

The chart shown in FIG. 3B organizes and lists the controls performed by the advice system for each emotion. In the second row of the chart, eight human emotions, that is, negative emotions "irritated", "sick", "sorrow", "daze", "anxiety", and positive emotions "interest", "peace", and "acceptance" are listed. Display side by side. Of the emotions described in the second row, those in parentheses are those with strong emotions (strong bias), and those in brackets are those with weak emotions (weak bias). , Close to neutral emotions).

These emotions include the user's voice acquired by the microphone in the mobile terminal 10, the user's face, appearance, gesture, etc. acquired by the imaging unit 14, and the user's biological information detected by the biological detection unit. , The sensor cooperation unit 13 collects, and the user reaction determination unit 15a determines the above-mentioned emotion based on these information. For example, when the microphone acquires "stop" as the user's voice, it is determined that the user's emotions are "irritated", "angry", etc., depending on the voice of the user, and the depth and strength of the emotions are further determined. (Severe or mild) is also determined.

In FIG. 3B, when the user reaction is a positive emotion, the user's emotion is basically kept as it is. On the other hand, when the user reaction is a negative emotion, the negative emotion is weakened or switched to another emotion. Specifically, this device is described as "emotion control" in the 4th to 6th stages of FIG. 3B, and works on the user according to this emotion control.

The description in the fourth row (the row described as "severe") in FIG. 3B shows what kind of advice is given when the degree of emotion described in the second row is strong (strong bias). There is. In the example shown in FIG. 3B, when the emotions are negative "irritability", "sickness", and "sorrow", it is a "roundabout operation", that is, the feeling of giving advice by looking at the timing, not the timing that should be advised now. Take control. This "roundabout operation" may include preparations for switching to other less difficult advice and a change of mood by changing the topic.

In addition, when the emotion is negative "daze" and "anxiety" and the emotion is positive "interest", and the emotion is strong, the user is persuaded. The method of persuasion at this time is almost the same as the case where the emotions described later are mild, and the degree is strengthened. Details of the style of persuasion will be given in the mild case.

In addition, when the emotion is positive "peace" and "acceptance" and the emotion is strong, the presentation of advice is terminated. It is considered that the advice presented above is good or necessary for the user. If this advice is acceptable, then no further emotional control is needed, as persuasion is of course unnecessary, and the process ends. It should be noted that the avatar may display an appropriate aizuchi, etc., without making a call to actively control emotions as in the "persuasion" system.

The description in the fifth and sixth stages of FIG. 3B, that is, "mild", indicates what kind of advice is given when the degree of emotion described in the second stage is weak (weak bias). In the example shown in FIG. 3B, when the emotions of "irritability", "sickness", "anxiety", and "interest" are mild, the visual elements such as the appearance and hairstyle of the avatar or robot output on the mobile terminal are used. , Voice, wording, etc. make it responsive to the user's emotions.

In other words, if the "depth" of negative emotions is deep and the "strength" is strong (severe), it is highly likely that the person is in a state of mind that refuses the logical meaning of the advice. In this case, it is preferable to control the emotions at a glance or at a glance so as not to suddenly cause entanglement of emotions, which is difficult to convey by letters or texts. In the present embodiment, as a visual / auditory appeal and a means of expression, an avatar, in particular, an item that modifies the effect (here, facial expression, gesture, standing behavior, pose, voice, wording, background, It makes effective use of air vibration, aura expression that conveys concentration distribution, halo, etc.).

The "wording" here is not a written word, but an exclamation mark, a sigh, a sigh, a sound expressing a smile, an iconography, a letter, and the like. By making full use of the sensory healing effects such as the appearance, the feeling of hearing the voice, BGM, and sound effects, the emotions are returned to the neutral state.

That is, in the present embodiment, control of emotional content that changes the user's emotions (change in the mode of the avatar or change in the emphasis intensity of the effect of the item that modifies it) and language-dependent communication such as conversation with the user. It includes the concept of logical language system control, and is provided with an emotional determination step (see, for example, S32 in FIG. 2) for monitoring the user's situation and determining the emotional system reaction. When this emotional reaction is determined, if the emotional reaction deviates from a predetermined level, the step of modifying the avatar to change the item that supplements the effect (for example, before activating the verbal advice) (for example, S41 of FIG. 2, FIGS. 3A (a), (c), FIG. 3B, etc.) are provided. When giving linguistic advice, we are trying to remove the emotional barriers that have arisen for the user at the timing prior to that. The predetermined level may be something like a predetermined threshold value, but a level at which the user does not respond to the dialogue may be monitored and used as a reference.

Therefore, at the above-mentioned predetermined level, various advices and changes in the user's facial expressions are recorded in an environment where advice is frequently given, and when the user refuses the advice from the recorded user situation history. It may be decided according to the emotional reaction. This emotional reaction is based on the emotional depth judgment, but there are cases where the terminal such as an advice device is turned off without changing the facial expression, and the emotional reaction is considered to include such a situation. It is also possible to record the facial expressions and reactions when the power is turned off in the recording unit without responding to the advice, and use that as a reference.

In the case of an avatar or a robot whose appearance can be changed, the posture, clothes, facial expressions, and voices are classified into, for example, clothes as shown in FIG. 3A (c), such as "business", "casual", and "". Appropriately select the one that can be changed in the manner obtained by the positioning by the axis such as "individuality" and "safety". Since the heart is healed just by the character that the user likes, we will make full use of such a method to change the user's emotions in a direction that makes it easier to accept advice.

In addition, the mode of avatars and the like is switched as shown in this table, and the part written as "persuasion", including words (text), assists in calming, healing, and promoting emotions. In other words, for anxious people, they choose words that are full of empathy, such as "I understand that you will be anxious." If you need an understanding of reasoning, it is safe to use a business system, and if it is rambling, it is better to change your mood with a casual system.

In the 4th and 5th stages of FIG. 3B, when the emotions are positive "calm" and "acceptance" and the emotions are mild, the advice is accepted as it is, but since the emotions are mild, We encourage the acceptance of advice so that it is easier to accept.

When working on the user according to such emotional control, record what emotional state the user was in and what emotional control was effective, and then check the accuracy when performing emotional control. You may try to improve. You may dare to try various control patterns and accumulate information about the user's reaction at this time, or you may reflect the accumulated records in the emotional control of another person.

In the present embodiment, when the emotional control shown in the 4th to 6th stages of FIG. 3B is performed, the advice is further accepted by the linguistic control described in the 7th to 8th stages. After taking care of the emotional side (or at the same time), we control the language system, which is basically controlled according to the content that appeals to the logical benefit and the content that appeals to the sensory temptation. It may be controlled according to the content that forces the user (boosting system) and the content that guides the user (guidance system), or the content with accessibility (support) and the content without assistance. It may be well controlled according to physical and emotional content. In addition, we will continue to give more specific advice and persuade, such as changing the distribution in the classification according to the personality and taste of the user.

In the example shown in FIG. 5B, two types of language control, the presence / absence of the support described in the seventh stage and the induction system or boost system control described in the eighth stage, are shown. As shown in FIG. 3A (b), the user may accept the advice as long as it assists the user. Therefore, when the user's emotions are "daze", "anxiety", and "interest", the advice is presented to assist the user in executing the advice.

Further, as for the induction system and the boost system, as shown in FIG. 3A (a), the advice includes an inductive (induction system) and a compulsory (boosting system). Depending on the user's emotions, the inductive one may be easier to accept, and the compulsory one may be easier to accept. Therefore, the guidance system and the boost system advice are selected according to the user's emotions. Of the compulsory advice, the boost-type advice has a low degree of coercion.

Assuming that "advice" is one of communication, effective use of "nonverbal communication", which has a large intuitive influence, is effective when there is a refusal due to "verbal communication". As mentioned above, nonverbal communication also depends on a person's personality, propensity, tastes, and values, so the type of the person is grasped and the response is made according to the type. However, this countermeasure should be simplified to some extent as explained here, and should be rationally classified and treated. Without such ingenuity, services, their systems, programs, etc. will become bloated, making it difficult to upgrade, etc., and it will not be possible to respond quickly in the event of a failure. Also, when using an inference model together, the test data is diverse and it takes time to develop.

In the example shown in FIG. 3B, the method of advice is classified by using the user's emotion as a key, but the advice does not necessarily have to be based only on the emotion. In addition, since it may be possible to know what kind of personality and preference the user has depending on what kind of emotion the user is likely to express, the emotions are mainly illustrated. In other words, for those who are frustrated, tired, or sad, the support and boost feels great to them, and it is better to control them with an inductive system so as not to negate the effect of emotional suppression. ..

In addition, if the user sees the advice presented by the mobile terminal, etc., and has "absent-mindedness", "anxiety", or "interest" but does not meet expectations, he / she can offer support. , It is likely that it will be easier to follow the advice. Also, with feelings of "peace" and "acceptance", it is better not to do extra things and water the positive feelings, so support is unnecessary. However, depending on the situation, or depending on the person's personality, tastes, and past performance records, support is not superfluous and, of course, may be expected.

Further, in the chart of FIG. 3B, the distinction between the guidance system advice and the boost system advice is also described, but similarly, "boost" may lead to disgust and irritation of the user, so caution is required. In addition, some of the guidance system advices are logical and appeal to the senses, and this may be switched depending on the type of user and the type of advice at that time.

In the chart shown in FIG. 3B, in the case of positive emotions, the state is used to give priority to boosting. This is because my heart is already moving, so after that, it will be effective to give more specific information, for example, let me choose from multiple options and have independence and take action. .. Alternatively, it may have a bargaining element such as a good deal (no waiting time, benefits, etc.) now. In addition, even if it is a negative emotion, in the case of "absent-mindedness" or "anxiety", it may be boosted by presenting a plurality of options in the same manner. If the advice has options that the user can freely choose, or if the possibility of negotiation is suggested, the user may become passive and active and dispel anxiety. That is, since the additional information of the advice includes a plurality of different proposals and the user can select from them, the user's involvement becomes independent.

In addition, in the chart shown in FIG. 3B, in the case of negative emotions, there are merits such as information that it is theoretically beneficial for the future, information that it is not scary sensuously, and that there is such a good thing now. It is stated that priority is given to inductive advice such as emphasis. In reality, multiple emotions are often mixed, so each control may be decided by trial and error based on this way of thinking. For example, the correspondence shown in FIG. 3B may be set as the default, and the change may be made by trial and error while observing the reaction of the user.

Further, in the chart shown in FIG. 3B, although it depends on the personality of the user, it is assumed that it is expressed in emotions, and what to do with the advice in that case is described. However, apart from emotions and personality, advice may be set independently for each.

Also in the example shown in FIG. 3B (the same applies to other figures), the user's situation is determined (for example, see S1 in FIG. 6A described later), and the determined user's situation is used from the first database. When the first advice is selected and the selected first advice is presented to the user (see, for example, S31 in FIG. 2 and S5 in FIG. 6A described later), the reaction of the user is determined (for example, FIG. 2). S32, S7 of FIG. 6A), when the first advice is presented, it is possible to select and display from different avatars and items classified into a plurality of types according to the reaction of the determined user (for example,). (See S32 to S41 in FIG. 2 and S17 to S25 in FIG. 6A).

Here, "following a reaction" means, for example, changing depending on whether the user's reaction is positive or negative. It is to classify avatars into business style and casual style so that you can make changes such as casual if positive and business style if negative. You may make it possible to select the ones that have been classified as safe and individual. In addition, avatar modification items such as clothes, hairstyles, backgrounds, voices, accessories, makeup, appearance, age, afterglow, characters such as humans, animals, gods, games, animations, gender, etc. In the same way, the mode may be adaptively changed by appropriately selecting or adjusting.

Also in the example shown in FIG. 3B (the same applies to other figures), advice can be displayed by selecting from avatars having different modes classified into a plurality of types and items that modify the avatars. In this case, the user situation is determined (see, for example, S32 and S34 in FIG. 2 and S1 and S7 in FIG. 6A described later), and an item is selected based on the determined user situation (for example, in FIG. 2). S39, S41, see S17 to S25 in FIG. 6A). The user's situation in this case is not limited to the reaction of the user's emotions when the advice is presented, the environment in which the user is placed when the advice is not presented, the objective situation such as the user's schedule, etc. , Including various states.

In addition, it is the best type to detect the user's feelings for advice as follows, and to follow the advice from avatars with different modes classified into multiple types according to the corresponding emotions as follows. You may choose.

Example 1: If you are angry at the advice, change to an older boss-like avatar that will be less likely to hit you.
Example 2: If you feel sad about your advice, change to a lighter make-up, halo-like Virgin-like avatar.
Example 3: If you are ridiculed for advice, change to a scary adult avatar with a shaved head and sunglasses.

Also, avatars and items can be combined independently. As mentioned above, when expressing an avatar and an item in combination, it may be better to include kinsoku processing, etc., but basically, it depends on the user's reaction and the situation at that time. Therefore, the system can be simplified if the avatar and the item can be combined independently.

The item also includes at least one of clothing, hairstyle, background, facial expression, voice, and gender. These are examples, and in addition to these, style, posture, facial expression, tone, make-up, aura, halo, and the like may be used. As described above, the item is attached to the avatar, and the impression of the avatar can be changed by changing the item.

Next, an example of the advice recorded in the advice information recording unit 16b will be described with reference to FIG. In the advice system according to the present embodiment, the situation of the user is determined (for example, see S1 in FIG. 6A), and as a result of this determination, if advice is required, advice 0 is given to the display unit 12 of the user's mobile terminal 10. (See, for example, S31 in FIG. 2, S3, S5, and S7 in FIG. 6A). Further, the reaction of the user who sees this advice is determined, and the advice is displayed according to the result of this reaction (see, for example, S32 in FIG. 2, S7, S9, S13, S21 to S25 in FIG. 6A). In the chart shown in FIG. 4A, advice to be presented to the user is shown for two examples (situation 1 and situation 2).

In situation 1 in FIG. 4A, the time for the user's health diagnosis is approaching. When the mobile terminal 10 determines that the situation is 1, it is determined that it is necessary to present advice for recommending the user to go for a medical examination. When this determination is made, the mobile terminal 10 displays an advice message as advice 0 on the display unit 12, for example, "○ × inspection is recommended for your age." In addition, the avatar may interactively present advice as shown in FIG. 4 (b) in order to give a feeling of thinking together at the time of this display.

When the user sees the advice 0 displayed on the display unit 12, as described above, the reaction differs depending on the situation of the user and the like. Therefore, according to the situation of the user, personality, etc., the obstacles are reduced and the advice that the user can easily understand is displayed.

From the user's reaction when displaying advice 0, if the assisted advice is appropriate, for example, advice such as "I will help you prepare together without strain (meal management the day before)." The message 41 is displayed (see 4A (b)). In addition, when the vision type advice is appropriate, for example, an advice message 41 such as "It seems that 70% of the people who have undergone this test are in good physical condition" is displayed. If the attraction-type advice is appropriate, "This teacher is a well-reputed teacher" is displayed as an advice message 41 as the voice of the person who has undergone this inspection. At this time, not only the text message may be displayed on the display unit, but also the voice may be transmitted from the speaker in the mobile terminal 10. Further, when imperative advice is appropriate, an advice message 41 such as "I'll do it!" Or "Tell me to do it." Is displayed.

Further, the situation 2 in FIG. 4A is a state in which the photographing equipment has not been used because the photographer has not been out for a long time. If the mobile terminal 10 determines that the situation is 2, it is determined that advice is needed to recommend the user an opportunity to go out for shooting with the shooting equipment. When this determination is made, the mobile terminal 10 displays an advice message 41 as advice 0 on the display unit 12, for example, "Tomorrow the cherry blossoms in Komiya Park are in full bloom", as shown in FIG. 4 (b). .. Further, in order to give a feeling of thinking together at the time of this display, the avatar 42 may interactively present the advice as in the case of the situation 1.

For example, as shown in FIG. 4 (b), when advice 0 is displayed on the mobile terminal 10 together with the avatar 42 and the advice message 41 is displayed, the user sees this display and says, "That's right, isn't it?" It may be issued (see vocalization 43 in FIG. 4 (b)). Analyzing this utterance 43, it is highly likely that the user is willing to go out for shooting if there is assistance such as help, although it is sudden. Therefore, the mobile terminal 10 presents the user with assisted advice, for example, "Would you like to help prepare the lunch box?". With such advice, the user is likely to go out for shooting with the shooting equipment with the help of the mobile terminal 10.

In addition, for example, if going out for shooting is a meaningful or effective action, the mobile terminal 10 provides vision-type advice, for example, "one day" to a user who is willing to go out for shooting. The higher the number of steps, the lower the maximum and minimum blood pressure. "Is displayed as an advice message 41. Also, for example, for users who are easily convinced when solicited, the mobile terminal 10 uses attraction-type advice, for example, an avatar, so that the person who went there will want to go, "No way, it's so beautiful. ... ”is displayed as an advice message 41. Such users are likely to do it when they hear their experiences. In addition, in the case of an emotional user, there is a high possibility that the user will execute the advice by presenting advice with the voice or avatar of an idol who has a good sensibility and likes.

Furthermore, the mobile terminal 10 is for users who have a lot of excuses, who speak well, who are reluctant to follow advice, and who have a weak will and can execute when there is a voice that encourages them. Imperative advice, such as "I'm going" or "Say I'm going", is presented to the user.

Next, an example of advice in the case of inheriting the advice information will be described with reference to FIGS. 5A and 5B. In the case of situation 1 shown in FIG. 4, when the user receives advice 0 and goes out for a health diagnosis, a doctor or the like performing the health diagnosis tells the user according to the history of advice and changes in the situation. It is convenient to give advice. Further, in the case of the situation 2 shown in FIG. 4, after the user receives the advice 0 and the subsequent advice by the mobile terminal 10, the second information device 20 or the like may be carried and photographed. In such a case, it is convenient that a device such as the second information device 20 can give advice to the user according to the history of advice and changes in the situation. Therefore, in the present embodiment, the advice transfer information 30 is transmitted to a device such as the second information device 20 so that the advice according to the situation can be given on the premise of the history of the advice given to the user in the mobile terminal 10. is doing.

The example shown in FIG. 5A provides advice when a user undergoes endoscopy. This example is a situation similar to situation 1 in FIG. 4, in which endoscopy is incorporated at the time of health examination. Timing 1 is a situation in which a user who has already undergone endoscopy is approaching the time to undergo endoscopy. Further, the timing 1b is a situation in which a user who has undergone endoscopy for the first time is approaching the time to undergo endoscopy. Timing 2 is a situation after the user of timing 1b applies for endoscopy and before receiving a medical examination. Timing 3 is a situation in which an endoscopy is performed at a clinic regardless of whether or not the patient has undergone endoscopy.

Using the chart shown in FIG. 5A, advice 0 and advice to be presented to the user according to the reaction when the advice 0 is presented to the user will be described.

At timing 1a, the display unit of the user's mobile terminal 10 displays "It has been 0 years since the last examination. Would you like to have an endoscopy again?" If the assist-type advice is appropriate from the user's reaction, and if the vision-type advice, attraction-type advice, and imperative-type advice are appropriate, the advice shown in the chart of FIG. 5A can be given. It becomes easier to accept advice 0. In presenting the advice, the advice may be simply displayed as a text, but the advice may be presented by an avatar as shown in FIG. 5B (b). In this case, the hairstyle, clothes, background, etc. of the avatar may be changed according to the type of advice.

Also, when advice 0 is displayed at timing 1b, if the assisted advice is appropriate from the user's reaction, "I will help you prepare without any hassle. For example, meal management the day before. I will give you some advice. " When the user sees this display, he / she is assisted by the mobile terminal 10 and has a sense of security, so that he / she can easily accept the advice 0. Then, at timing 2 (before the examination), when advice 0 is presented, "Would you like to help me skip the meal?" Is displayed. Furthermore, at timing 3 (at the time of medical examination), a doctor or the like is displayed with "This person has worked hard to skip the meal, so please praise him." In this way, if the type of assisted advice is appropriate, it will be easier to accept the advice if the user is close to the user and gives advice that assists in achieving the purpose. As an advice, it is suggested that the user cannot / cannot think of it individually as illustrated in FIG. 5A. This vision-type advice is effective for people who "relieve when they meet someone."

In addition, when advice 0 is displayed at timing 1b, if vision-type advice is appropriate from the user's reaction, "If cancer can be detected early, nearly 90% of patients will be 5 years after diagnosis. It seems that he is still alive after that. " Then, at timing 2 (before the examination), when advice 0 is presented, "If you take the examination here (if there is no abnormality), it looks good in 3 years." Furthermore, at timing 3 (at the time of medical examination), a message such as "If there is something that needs improvement in your life, please tell us the risk." Is displayed. In this way, if the type of vision-type advice is appropriate, it is easier to accept advice 0 by showing future success stories and giving advice that makes the user imagine a successful vision. In this case, it is advisable to specify what will happen if the proposal is responded to. Effective for realists.

In addition, when advice 0 is displayed at timing 1b, if attraction-type advice is appropriate from the user's reaction, "This teacher has a good reputation" is displayed as the voice of the recipient. do. Then, at timing 2 (before the examination), when advice 0 is presented, "I will explain the examination flow of the day in advance." "Shall I play relaxing music?" Is displayed. Furthermore, at timing 3 (at the time of medical examination), a message such as "I feel uneasy, please give me a reassuring music or tell me that it is okay." Is displayed. In this way, if the type of attraction-type advice is appropriate, if the advice is given to stimulate emotions, it becomes easier to accept the advice 0. In this case, for example, a proposal is made to eliminate the anxiety felt by the user. Effective for anxious people.

Further, when the advice 0 is displayed at the timing 1b, if the imperative advice is appropriate from the viewpoint of the user's reaction, "Please consult!" Is displayed. Then, at timing 2 (before the examination), when advice 0 is presented, "Prepare for tomorrow!" Is displayed. Furthermore, at timing 3 (at the time of medical examination), a message such as "Please relax. Please do not speak." Is displayed to the doctor. In this way, if imperative advice is appropriate for the type, high-pressure advice makes it easier to accept advice 0. In this case, it is effective for the weak-willed type.

As described above, in the example shown in FIG. 5A, the advice 0 as shown is presented to the user at the timings 1a, 1b, and 2, and the assist type, the vision type, the attraction type, and the instruction are presented according to the mode of refusal of the user. Type advice is presented. Further, when the user applies or undergoes an examination, the advice transfer information up to that point is transmitted to the second information device 20 such as a doctor, and at these timings, the assist type, the vision type, the attraction type, and the command type. Advice is displayed. That is, the advice according to the type, individuality, and emotion of the user is taken over not only by the mobile terminal 10 but also by the second information device 20, and appropriate advice is given. In recent years, it has been desired to manage the physical condition of patients themselves outside the hospital, and a technique for transferring such information outside the hospital to medical information devices is effective.

Next, an example in which the advice information is inherited in a situation similar to the situation 2 in FIG. 4 will be described with reference to FIG. 5B. In FIG. 5B (a), the timing 1 is a case where the user has not been out for a long time, as in the situation 2 of FIG. 4 (a). At this timing 1, as described above, the display unit 12 of the mobile terminal 10 displays "The cherry blossoms in Komiya Park are in full bloom tomorrow" as advice 0. In addition, attraction advice and imperative advice are given according to the user's reaction. Assisted advice or vision type advice may be given in the same manner as in situation 1 of FIG. 4 (a).

Timing 2 in FIG. 5B (a) is after the user receives advice 0 and actually goes out for shooting. Upon going out for shooting and arriving at the shooting point, the mobile terminal 10 displays the message 41a on the display unit 12 as the advice 0 following the attraction-type advice, as shown in FIG. 5B (b). This message 41a displays, for example, "This is the shooting point." When the user sees this message 41a, he / she responds, "Well, this is enough" (see vocalization 43a).

Further, at the timing 2, the user displays, for example, "Would you like to help the composition determination?" When the assist type advice is appropriate from the viewpoint of the user's reaction when the advice 0 is displayed. In addition, if the vision-type advice is appropriate, for example, "I would be pleased to announce the photograph taken to XX." Also, if attraction-type advice is appropriate, say "Wow, in full bloom !!". Furthermore, if imperative advice is appropriate, a display such as "Take a picture anyway!" Is displayed. In the case of this imperative advice, the hairstyle of the avatar 42a may be scared, or the avatar itself may be changed according to the user's reaction, such as changing to a hairstyle that seems to be gentle when the command is followed. good. Similarly, the background of the avatar 42a may be changed. The avatar 42a when giving assist-type advice, vision-type advice, and attraction-type advice also has the avatar's hairstyle, clothes, background, facial expression, voice, and gender according to the user's reaction so that the user can easily accept the advice. Etc. should be changed.

Next, the operation of presenting advice to the user will be described with reference to the flowcharts shown in FIGS. 6A and 6B. As described above, in this flow, the advice presented next is optimized according to the reaction when the mobile terminal 10 presents the advice 0 to the user. This flow is mainly realized by the control unit 15 in the mobile terminal 10 according to the program stored in the memory. However, as described with reference to FIGS. 5A and 5B, when the advice information is taken over by the device such as the second information device 20, the flow after the takeover takes over the information such as the second information device 20. The control unit inside the device is realized.

When the flow shown in FIG. 6A starts, the control unit 15 first determines the user's situation (S1). The control unit 15 is a posting on the user's SNS, a user's schedule recorded on the mobile terminal 10, a movement record by GPS or the like, a user's conversation acquired by a microphone in the mobile terminal 10, and an image pickup unit 14. The user's situation is determined based on various information such as the image acquired by. In the example shown in FIG. 4A, the user has not been out for a long time, or the time for the health examination is approaching. This determination may be determined in advance based on logic, or may be determined using inference using an inference model or the like.

Subsequently, the control unit 15 determines whether or not advice is necessary (S3). Here, the control unit 15 determines whether or not advice is necessary based on the situation of the user determined in step S1. For this determination, for example, the situation of the user determined in step S1 may be classified, and the necessity of advice may be set in advance for each classification. In addition, teacher data is created by adding the necessity of advice to the user situation as an annotation, and using this teacher data, an inference model for determining the necessity of advice is generated, and this inference model is used. The determination in step S3 may be executed. As a result of this determination, if no advice is required, the process returns to step S1.

As a result of the determination in step S3, if advice is required, advice 0 is output (S5). Here, the advice information selection unit 15c of the control unit 15 selects appropriate advice according to the situation of the user determined in step S1 from the advice information group 0 (16ba) of the advice information recording unit 16b. The advice information output unit 15d displays the selected advice 0 on the display unit 12.

When the advice 0 is output, the control unit 15 next confirms the user's reaction (S7). Here, the sensor cooperation unit 13 in the mobile terminal 1 detects the user's reaction. For example, a microphone picks up a user's voice, a sensor detects blood flow, a heartbeat, a blood pressure, and the like, and an imaging unit 14 detects a user's facial expression and the like. Based on these sensor outputs, it is determined whether the user accepts the content of the advice 0, absolutely rejects it, accepts it depending on the method of advice, and the like.

After confirming the user's reaction, it is next determined whether or not the reaction was accepted (S9). For example, if the user says "OK" or makes a nodding gesture, or if the user shows an acceptance reaction, it is predicted that the advice 0 will be accepted and the user will act according to the advice 0. In this case, it is determined to be accepted.

If the result of the determination in step S9 is acceptance, the user reaction history information and the advice takeover information are transmitted to the takeover device if there is a takeover device (S11). As described with reference to FIGS. 5A and 5B, the user may use another device such as the second information device 20 from the mobile terminal 10 according to the advice. In this case, the user reaction history information and the advice takeover information (see the advice takeover machine information 30) recorded in the user action history recording unit 16a up to that point in the other device are transferred to the takeover device in the device communication unit. Send through 18. This transfer information may also include information about the user's skills. When this process is performed, the process returns to step S1.

If the result of the determination in step S9 is not acceptance, the control unit 15 determines whether or not it is an absolute rejection (S13). For example, if the user says "I absolutely don't like it" or shakes his face to make a refusal gesture, etc., if he shows a strong intention of refusal, he will not accept advice 0 and will follow this advice 0. It is expected not to act. In this case, it is determined to be an absolute refusal.

As a result of the determination in step S13, in the case of absolute rejection, advice 0 is output (S15). In this case, since it is difficult to persuade the user, the advice information output unit 15d only outputs the alternative advice 0. When the advice 0 is output, the process returns to step S1.

If the result of the determination in step S13 is not absolute rejection, the background at the time of rejection is determined (S17). In this case, the rejection reaction seems to be persuasive (see, for example, reaction 2 and correspondence 2a in FIG. 11). Therefore, the control unit 15 determines the situation in which the user is placed, the personality and characteristics of the user, and the like, based on the reaction of the user when the advice 0 is presented in step S7 and the action history of the user up to that point. ..

Then, it branches to the optimum advice type (S19). Here, the control unit 15 gives the advice described with reference to FIGS. 4, 5A, 5B, etc., based on the background at the time of rejection determined in step S17, the reaction of the user shown in steps S5, S7, and the like. From the types, determine the best advice for the user.

As a result of the determination in step S19, when the user desires assistance to act according to the advice, assisted advice is given (S21). Further, when the user wants to know the meaning and effect of the action according to the advice, the vision type advice is given (S23). Further, when the user wants to invite the user to take an action according to the advice, the attraction type advice is given (S24). Further, when the user seems to obey when giving advice in an imperative tone, imperative advice is given (S25).

Further, in addition to the advice in steps S21 to S25, other advice (not shown) may be given. In addition, when giving these advices, the avatars 42, 42a, etc., the clothes, hairstyles, background images, etc. of the avatars may be changed according to the type of advice. In addition to displaying on the display unit 12, a speaker or the like may be provided to give advice by voice.

When the control unit 15 gives advice in steps S21 to S25, the control unit 15 then confirms the user's reaction, digitizes the reaction, and records it (S27). Here, the control unit 15 determines the reaction of the user when the advice is displayed in steps S21 to S25 based on the detection output from the sensor cooperation unit 13. Then, the determined reaction of the user is converted into standardized data and recorded in the user action history recording unit 16a. At this time, the control unit 15 also records the displayed advice in association with it.

Subsequently, if there is a takeover device, the user reaction history information and the advice takeover information are transmitted to the takeover device (S29). After displaying the advice in steps S21 to S25, the user may use another device such as the second information device 20 from the mobile terminal 10 according to the advice. In this case, as in the case of step S11, the user reaction history information and the advice takeover information (see the advice takeover machine information 30) are transmitted to the takeover device through the device communication unit 18. When this process is performed, the process returns to step S1.

Next, the operation of the action assist will be described with reference to the flowchart shown in FIG. This behavioral assistance flow shows another example of the operation corresponding to steps S5 to S25 in FIG. 6A. Further, in this flow, in FIG. 2, step S37 is replaced with step S38, and similar processing is executed in other steps. Although steps S32 and S34 are omitted in FIG. 7, these steps may be executed. Therefore, this flow is also realized mainly according to the program stored in the memory by the control unit 15 in the mobile terminal 10. However, as described with reference to FIGS. 5A and 5B, when the advice information is taken over by the device such as the second information device 20, the flow after the takeover is the control unit of the device in the second information device 20 or the like. Will be realized.

When the flow shown in FIG. 7 starts, advice 0 is first displayed (S31). Here, similarly to step S31 of FIG. 2 and step S5 of FIG. 6A, appropriate advice 0 is selected from the advice information group 0 (16ba) of the advice information recording unit 16b according to the user's situation. The advice information output unit 15d displays the selected advice 0 on the display unit 12. Before displaying the advice 0, the control unit 15 may determine the user's situation and determine in advance whether or not the advice is necessary, as in FIG. 6A.

Subsequently, the control unit 15 determines whether or not the user rejects the advice 0 (S33). Here, the control unit 15 detects the user's reaction and determines whether or not the advice 0 is rejected based on the detected reaction, as in step S33 of FIG. 2 and step S7 of FIG. 6A. ..

If the result of the determination in step S33 is not rejection, this flow is terminated (S35). In this case, since the user has accepted the advice 0 and no new advice is required unless the situation changes, the flow of this action assistance is terminated.

If the result of the determination in step S33 is rejection, it is determined whether or not the rejection reaction is mental (S38). Here, the reason why the user rejects the advice 0 based on the user's voice etc. from the collected voice data by collecting the user's voice by the microphone or the like in the sensor cooperation unit 13. , Whether it is a mental reason or a physical reason, etc. are determined. Further, this determination is not limited to the vocabulary, and the reason for refusal may be determined based on the facial expression, gesture, etc. of the user acquired by the imaging unit 14. Further, various biological reactions such as blood flow, heartbeat, and blood pressure may be detected by the biological detection sensor in the sensor cooperation unit 13 to determine the reason for refusal. Further, in these determinations, machine learning may be performed using words, facial expressions, biological reactions, etc., and the determination may be made using the generated inference model.

Further, when the reason for refusal is a mental reason, the feeling of refusal may be alleviated depending on how the user feels. On the other hand, if the reason for refusal is a physical reason, it is difficult to persuade the refusal reaction depending on how you feel, so give other advice.

In determining whether it is a mental refusal or a physical refusal, the type of stress may be determined based on a biological signal such as a fluctuation of the heartbeat. When there is mental stress, changes are seen in biological signals and the like. If there is no mental stress such as lack of time or money, biological signals do not change. In addition to the biological signal and the like, it may be determined from the result of filling out the questionnaire filled out by the user and the tendency of the user's behavior history and the like.

As a result of the determination in step S37, if the rejection reaction seems to be persuasive, or if the rejection reaction is mental, the advice series 1A is output (S39a). Here, the advice information selection unit 15c obtains the optimum advice 1 from the advice information group 1 (16bb) and the advice information group 2 (16bc) in the advice information recording unit 16b based on the reason for refusal determined in step S37. select. The advice information output unit 15d outputs the selected advice 1 to the display unit 12 and the like.

On the other hand, if the rejection reaction is physical as a result of the determination in step S37, the advice sequence 2A is output (S41a). Here, the advice information selection unit 15c provides the optimum advice 2 from the advice information group 1 (16bb) and the advice information group 2 (16bc) in the advice information recording unit 16b based on the reason for refusal determined in step S37. select. The advice information output unit 15d outputs the selected advice 1 to the display unit 12 and the like.

As a result of the determination in step S38, if the reason for refusal is mental, select from the advice series 1A and output advice 1 as described above (see S39a), while if the reason for refusal is physical, Select from the advice series 2A and output the advice 2 (see S41a). In presenting this advice, the advice may be presented not only by simply displaying the text but also by using an avatar or the like (see, for example, FIGS. 4B and 5B). At this time, the hairstyle, facial expression, etc. (item) of the avatar may be changed according to the reaction of the user. Also, when refusing for mental reasons, rational advice is effective, while when refusing for physical reasons, sensory advice is effective.

For example, when "Eat XX" is presented as advice 0 and the mental refusal is made, the nutritional content of XX is presented as advice series 1A (see S39a). On the other hand, in the case of physical refusal, it is effective to emphasize the deliciousness of XX as the advice series 2A (see S41a). In this case, it is also good to use an image or the like that shows the deliciousness of XX.

Further, as another example, when "Go to △△" is presented as advice 0 and the mental refusal is made, the actual benefit of △△ is presented as the advice series 1A (see S39a). On the other hand, in the case of physical refusal, it is effective to emphasize the fun of Δ△ as the advice series 2A (see S41a). In this case, it is also good to make full use of images, sounds, etc. that show the fun of △△.

Further, for example, as advice 0, it is assumed that "stretching after taking a bath for 20 minutes" is displayed. As a user's reaction, remarks such as "I'm sorry", "Eh", and "Muri" may be made. In such a case, not only the voice (word spirit) but also the facial expression and the heartbeat change. Since this reaction is a mental refusal, the advice series 1 or the advice series 2 is output in consideration of the user's action history, for example, "do it if two people" or "never do it".

Next, another example of the action assisting operation will be described with reference to the flowchart shown in FIG. Similar to the case of FIG. 7, this action assist is also an operation corresponding to steps S5 to S25 in FIG. 6A. Therefore, this flow is also realized mainly according to the program stored in the memory by the control unit 15 in the mobile terminal 10. However, as described with reference to FIGS. 5A and 5B, when the advice information is taken over by the device such as the second information device 20, the flow after the takeover is the control unit of the device in the second information device 20 or the like. Will be realized.

In the flow shown in FIG. 7, the advice 1 selected from the advice series 1A is presented or the advice 1 selected from the advice series 2A is presented according to the reaction when the advice 0 is shown to the user. On the other hand, in the flow shown in FIG. 8, the advice 1 selected from the advice series 1B for dealing with mental refusal and the advice 2 selected from the advice series 2B for dealing with physical refusal are presented to the user, respectively. The stress at the time is judged, and the advice series with the smaller stress is output. In this flow, it is assumed that the advice information group 1 (16bb) stores the advice 1 for mental refusal response, and the advice information group (16bc) stores the advice 2 for physical refusal response.

When the flow shown in FIG. 8 starts, the control unit 1 first presents the advice sequence 1B to the user (S51). Here, the advice information selection unit 15c selects the advice 1 for mental refusal from the advice information group 1 (16bb) in the advice information recording unit 16b, and the advice information output 15d is output so that it can be presented to the user. do.

When the advice 1 is presented, the stress or the like determination 1 is performed next (S53). Here, the control unit 1 determines the stress of the user or the like when the advice for dealing with mental refusal is presented to the user based on the detection result of the sensor in the sensor cooperation unit 13.

Subsequently, the control unit 1 presents the advice sequence 2B to the user (S55). Here, the advice information selection unit 15c selects advice for physical refusal from the advice information group 2 (16bc) in the advice information recording unit 16b, and the advice information output 15d outputs it so that it can be presented to the user. ..

After presenting the advice 2, the stress or the like determination 2 is performed next (S57). Here, the control unit 1 determines the stress and the like of the user when the advice for dealing with the physical refusal is presented to the user based on the detection result of the sensor in the sensor cooperation unit 13.

The determination of stress or the like in steps S53 and S57 can be determined based on changes in biological signals such as fluctuations in heartbeat. Alternatively, the determination may be made based on a tendency such as a questionnaire result to the user or a user's behavior history. Furthermore, "About objective stress evaluation method (http://www.jsomt.jp/journal/pdf/066050330.pdf)", "Biological information sensing technology aiming at health care monitoring (https://www.jstage. You may judge stress etc. by using the method described in the document such as "jst.go.jp/article/essfr/12/1/12_30/_pdf)".

When the stress or the like determination 2 is performed in step S57, it is next determined whether or not the stress 1 is smaller (S58). Here, the control unit 15 compares the result of the stress or the like determination 1 in step S53 with the result of the stress or the like determination 2 in step S57, and determines which stress is smaller.

If the stress 1 is smaller as a result of the determination in step S59, the advice sequence 1B is output (S61). In this case, since the advice for mental refusal is less stressful for the user than the advice for physical refusal, the advice for mental refusal is displayed to the user. Specifically, the advice information selection unit 15c selects the optimum advice 1 from the advice information group 1 (16bb) in the advice information recording unit 16b. The advice information output unit 15d outputs the selected advice 1 to the display unit 12 and the like.

If the stress 1 is larger as a result of the determination in step S59, the advice sequence 2B is output (S63). In this case, since the advice for physical refusal is less stressful for the user than the advice for mental refusal, the advice for physical refusal is displayed to the user. Specifically, the advice information selection unit 15c selects the optimum advice 2 from the advice information group 2 (16bc) in the advice information recording unit 16b. The advice information output unit 15d outputs the selected advice 2 to the display unit 12 and the like. Further, when presenting the advice, the advice may be presented not only by simply displaying the text but also by using an avatar or the like (see, for example, FIGS. 4B and 5B). At this time, the hairstyle, facial expression, etc. (item) of the avatar may be changed according to the reaction of the user.

The advice sequence 1B and the advice sequence B of the advice sequence 2 may be the same as or different from the advice in steps S39 and S41, respectively. For example, when "Eat XX" is presented as advice 0, the nutritional content of XX is presented in the advice series 1B, and the deliciousness of XX is emphasized in the advice series 2B. In this case, it is also good to make full use of an image or the like that shows the deliciousness of XX. Further, as another example, when "Go to △△" is presented as advice 0, the actual benefit of △△ is presented in the advice series 1B, and the fun of △△ is emphasized in the advice series 2B. In this case, it is also good to make full use of images, sounds, etc. that show the fun of △△.

Next, the operation of the health diagnosis / behavioral assistance will be described using the flowchart shown in FIG. This flow shows an example of a case where the mobile terminal 10 assists the user in the case of performing a medical examination. In this flow, the takeover device is assumed to be a device for health diagnosis (for example, a blood pressure measuring device, a blood test device, an X-ray imaging device, an endoscope (see FIG. 5A), etc.). This action assist is an action corresponding to steps S1 to S25 in FIG. 6A. Therefore, this flow is also realized mainly according to the program stored in the memory by the control unit 15 in the mobile terminal 10. However, when the advice information is inherited by the health examination device, the flow after the transfer is realized by a mobile terminal of a doctor or the like performing the health examination, a control unit in the health examination device, or the like.

When the flow shown in FIG. 9 starts, it is first determined whether or not the time for the health examination is approaching (S71). Here, the control unit 15 determines whether or not the time for the health examination is approaching based on the user's schedule or the like. As a result of this determination, if the time for the health examination is not approaching, the standby state is set in step S71.

As a result of the determination in step S71, when the time for the health examination is approaching, it is next determined whether or not the muri factor (the factor rejected) is physical (S73). Here, the sensor cooperation unit 13 detects the reaction when the mobile terminal 10 indicates to the user that the time for the health diagnosis is near. Then, the control unit 15 determines whether or not the user's rejection reaction is physical, as in step S37 (see FIG. 7). In addition to this, for example, when the user always feels sick when the health examination approaches, or when the user does not have time to undergo the health examination at work, the stuffiness factor is determined to be physical.

As a result of the determination in step S73, if the stuffiness factor is physical, it is determined whether or not it can be overcome mentally (S75). Here, when the user does not want to undergo a health diagnosis due to physical factors, the control unit 15 determines whether or not he / she is willing to undergo a health diagnosis by mental advice or the like. .. This determination is made based on the user's behavior history, the user's speech, facial expression, biological reaction, etc. detected by the sensor cooperation unit 13.

As a result of the determination in step S75, if it is possible to overcome mentally, a vision guide is performed (S79). Here, as in step S23, vision-type advice is presented. The vision guide is a guide that informs the user of the significance and effect of the advice. For example, there is a guide such as "If you do not have a medical examination, this will happen later." The advice or guide in this step may be, for example, the vision-type advice in FIGS. 4 (a) and 5A. Further, as described above, various emotions of the user are taken into consideration, and linguistic communication and non-verbal communication are taken into consideration. In addition, as nonverbal communication, avatars may be used to present advice. The same applies to steps S81, S85, and S87 described later.

On the other hand, if the result of the determination in step S75 cannot be overcome mentally, physical assistance is provided or another proposed guide is provided (S81). Since the factors that the user rejects the advice are physical and cannot be overcome by mentality, in this step, the control unit 15 assists to lower the physical obstacle. Further, the control unit 15 may make another proposal. For example, the doctor in charge of the medical examination may display the experience story that he is so good.

Returning to step S73, if the result of the determination in this step is that the muri factor is not physical, then it is determined whether or not the muri factor is mental (S83). Here, as in step S73, it is determined whether or not the user's rejection reaction is mental based on the detection result by the sensor cooperation unit 13. For example, if the patient refuses to take the endoscope because he / she can drink the endoscope but does not like to skip the meal, the stuffiness factor is determined to be mental.

As a result of the determination in step S83, if the stuffiness factor is mental, a physical needs emphasis guide is performed (S85). In this case, since the stuffiness factor was not physical but mental, it can be said that the physical guide is a guide that is easy to understand without causing great stress to the user. Therefore, the control unit 15 displays the physical needs emphasis guide. For example, it's just a matter of simple things to show that it's so effective.

On the other hand, as a result of the determination in step S83, if the stuffiness factor is not mental, a normal guide is performed (S87). In this case, since there are no physical and mental stuffiness factors, the control unit 15 may perform a normal guide. For example, all you have to do is display "It's time for a medical examination."

When the guide is performed in steps S79, S81, S85, and S87, the process returns to step S71.

In this way, in this flow, it is rationally simplified by dividing it into mental and physical. This is because we want to provide meaningful services to various types of target people and to as many people as possible with a simple system configuration. Primarily, with positive advice for mental rejection, wisdom overcoming physical rejection, and minimal physical assistance, this can also be a low-priced service. Assistance for mental change can be covered by relatively low-cost technology (some areas are not always so). Assistance for physical rejection is often physical and economical and often costly. Costly services are biased to reduce the number of users, so it is better to make them low-priced services as described above.

In the explanation of FIGS. 6A to 9, the operation in the mobile terminal 10 is mainly performed, but as described above, the operation screen of the second information device 20 and the device for health diagnosis is displayed from the mobile terminal 10. The guide information may be inherited and set, and the guide display may be performed based on the information inherited by another device. In particular, since the mobile terminal 10 is always held and carried by the user, the mobile terminal 10 has the most information about the user's behavioral characteristics, preferences, and the like. Therefore, the guide may be performed based on the information possessed by the mobile terminal 10, and the takeover device may also be guided based on the information possessed by the mobile terminal 10. The inherited devices such as the mobile terminal 10 and the second information device 20 are packed with functions and have many functions that the user does not know. Therefore, these devices may display guides by utilizing unknown functions.

Further, although the flow of FIG. 9 describes the behavioral assistance in the case of the health examination, if the determination in step S71 is the determination of the general guide correspondence status, it can be applied to the general behavioral guide. That is, it is determined in step S71 whether or not it is a “guide correspondence status”.

For example, suppose that the user is recommended to exercise such as walking or marathon. In this case, if the user refuses because he / she gets tired due to exercise, it is determined that the stuffiness factor is physical in the determination in step S73. If it can be overcome mentally in step S75, a logical guide such as "If you get fat, it's not cool" or "If you lack exercise, you will have to pay for medical expenses" is recommended as a vision guide in step S79.

Also, assuming that when a user serves a dish that uses corn at a restaurant or the like, the dish may not be eaten but he dislikes it, in this case, the stuffiness factor is not physical but mental. It can be said that it is. In this case, a guide that emphasizes physical needs may be provided (see S87). For example, a guide such as "The texture of corn, which is the ingredient used in cooking, is appetizing."

In this way, when presenting advice or a guide to the user, if the user's reaction is negative, it is determined whether the rejecting factor is physical or mental, and this determination result is used. Based on this, you can select advice and guides. Such advice and guides are convincing and useful information for the user.

Next, the operation of the trust building advice will be described with reference to the flowchart shown in FIG. Up to this point, an example of calling a user for a single action (that is, presenting advice) from a mobile terminal or the like has been described. In reality, each call and communication connection affects the user's mentality and leads to more effective advice. This is because the user gains trust in the source of advice by realizing that the advice is accumulated and that it is good for him / her. If the advice from the source of advice raises the expectation that something will be satisfied, even advice that was not accepted in the past will be accepted. The flow shown in FIG. 10 aims to improve such reliability.

When the flow of trust building advice shown in FIG. 10 starts, first, information is input (S91). Here, the user's basic information, for example, age, gender, nationality (environment in which he grew up), personality, etc. are input. This input may be input to the user in the form of a questionnaire when applying for building trust, or may be acquired by acquiring the user's history information or the like posted on the Internet.

Then, the avatar is determined (S93). Exchanging information through avatars displayed on mobile terminals and the like is very useful for building trust with users. Exchanging information through dialogue with avatars creates the illusion of talking to a real human being and deepens the relationship of trust. In this case, the appearance of the avatar and the like are largely related to the generation of the trust relationship. The avatar may be determined according to the user's preference, for example, a person whom the user longs for, a person who trusts, a hero, an idol, or the like.

Next, a trust relationship is built (S95). Here, the mobile terminal or the like presents advice to the user, and determines whether trust is built based on how much the user has acted according to the advice. To improve trust, daily communication is planned, for example, mirroring and listening to user complaints. In addition, mirroring means to give the other party a sense of intimacy or a favorable impression by imitating the other party's words and actions like a mirror. In addition, daily communication includes greetings and the avatar interacting with the user about the topic at that time.

Subsequently, actions are encouraged to build a relationship of trust (S97). Here, the method of advice is determined according to the reliability cultivated up to that point and the type of user. As a method of advice, change the background, clothes, facial expressions, etc. of the avatar. You may also change the type of advice. In addition, the tone, for example, command tone, recommendation, begging, etc. when presenting advice to the user may be considered in consideration of the user's reaction. When an action is urged in step S97, the process returns to step S95, feedback is given while observing the user's reaction, and the advice method or the like is changed. By repeatedly executing these processes, a relationship of trust with the user can be built.

In this way, by executing the flow of trust relationship building advice, it is possible to build a trust relationship with the user. If the relationship of trust is low, the appearance of the avatar to be used should be a business style with a decent make-up, and the advice to be presented should be low in difficulty. On the other hand, when the relationship of trust with the user is high, the appearance of the avatar to be used may be a rough casual style, and the difficulty of the advice to be presented may be high.

In addition, if the personality of the user can be grasped and advice can be given in consideration of the personality by executing the flow of trust relationship building advice, the customer probably thinks that the real needs and wants (the customer "wants" a certain product / service". You can feel that you understand the state of being. That is, according to this flow, before generating the next advice, the trust relationship built so far is judged in relation to the acceptance behavior for the advice, and the next advice is changed based on the trust relationship judgment result. It is possible to provide a method of building a relationship of trust.

As mentioned above, various methods of presenting advice that is easy for users to accept have been described. The following points should also be considered when presenting advice. In the present embodiment, an emotional determination is performed, and advice that can persuade the user is presented based on the emotional determination. In this case, the user may appear to have accepted the advice, but may not be completely convinced. Therefore, even if it seems that you have accepted the advice, it is advisable to perform after-sales follow-up. For that purpose, it is advisable to collect databases and evidence due to emotional changes. Assist the user who is satisfied.

In addition, when presenting advice, if the difficulty level is lowered after the advice with a high difficulty level is intentionally presented, the user can easily accept the lowered advice. It can be one of the assisted advice. In this case, it is good to advise, "Other people are also receiving it."

Further, when presenting advice, there is also a method of presenting not only one piece of advice but a plurality of pieces of advice and letting the user select. The user becomes active by selecting one of the plurality, even if it is passive until then. By becoming active, you will be proactive in giving advice. Also, by leaving room for negotiations, you will work on your own.

Next, it will be explained that the strength of persuasion and the content (method) are changed in consideration of the magnitude of "utility" and the strength of "resistance" of the user. Various advices to the user in steps S39 of FIG. 2, steps S5, S15, S21 to S25 of FIG. 7, S39a, S41a of FIG. 7, S61, S63 of FIG. 8, S79, S81, S85, S87 of FIG. And present a guide. When presenting this advice, etc., how strong the expression should be given to the user, that is, how much the "persuasion" is possible (strength, method) P should be, the size U of the "utility" and the user. Persuasive power can be increased by changing based on the strength R of the "resistance" of.

That is, it is desirable to change the propriety (strength, method) P of "persuasion" depending on the magnitude U of "utility". The benefits are, for example, "it should be delicious if you eat it", "it will be a memorial for your life if you take a picture", "if you do not treat it now, it will be fatal" and so on. On the other hand, it is desirable that the persuasionability (strength, method) P is also changed by the strength R of the user's "resistance". If you force it, you may feel disgusted, it may not be possible due to financial and physical circumstances, and it may not suit the taste of the user.

To persuade the user, considering that the strength of "persuasion" changes depending on the magnitude of "utility", it is advisable to do the following. (1) I would like to positively "persuade" (recommend) anything that leads to the "utility" of the user. (2) On the other hand, the user may "resist" to follow the advice due to various circumstances such as mental or physical "pain" even though it has "utility". (3) Given that the user's choice is first, usually, if the user "resists", he does not "convince". (4) However, the user may not be able to accurately recognize the relationship between "pain" and "utility" because he / she is not aware of the magnitude of the "utility". (5) If "utility"> "pain", that is, the user recognizes "utility" <"pain" and "resists" even though the utility is greater than the pain. If so, it is better to do "convincing" that is not normally done. (6) The strength and content of "persuasion" in this case changes not only depending on the dissociation between "utility" and "pain" but also on other situations.

Further, since the strength R of the user's "resistance" may be a temporary situation or a permanent situation, these can be considered separately. For example, it is a temporary situation that I do not want to eat because I am full, while it is a permanent situation that I cannot eat because I am diabetic. In addition, since there are many cases where the side giving advice (persuading side) has more information about the magnitude U of "utility" than the side receiving advice (user side), this implementation is carried out. In the form, this is premised. On the other hand, information on the user's "resistance" is often lacking on the persuasive side, so some means for searching for this information is necessary. For example, it is possible to search for information on the user's resistance R from the interactive format, SNS and other information, the usage status of the device (for example, a past photographed subject, etc.), biological information (heartbeat, voice print, facial expression), and the like.

The success of persuasion depends on overcoming "resistance" and gaining great utility. In order to make effective persuasion, not only quantitative (strong, weak) but also qualitative (soft, persuasive, flexible) factors work strongly as a real problem. Therefore, the strength R of the resistance is accurately measured (for example, S32 in FIG. 2, S7, S17 in FIG. 6, S38 in FIG. 7, S73 in FIG. 9, S75, S83 in FIG. 9, etc.), and this “resistance”. It is preferable to appropriately change the propriety (strength, method) of "persistence" according to the strength R and the magnitude U of "utility". Further, it is desired to change not only the strength of "persuasion" but also the method of persuasion (for example, S21 to S25 in FIG. 6A, S39 and S41 in FIG. 7, S61 and S63 in FIG. 8, and S79 in FIG. See S81, S85, S87).

As described above, the advice system according to the embodiment of the present invention determines the reaction of the user when the first advice is presented to the user (for example, the user reaction determination unit 15a of FIG. 1A, FIG. 2). (See S32, S7, etc. in FIG. 6A) and, at the time of presenting the first advice, one of the second advice classified into a plurality of categories is selected according to the user's reaction determined by the user reaction determination unit (see S7, etc.). For example, see the advice information selection unit 15c in FIG. 1A, S39 and S41 in FIG. 2, S21 to S25 in FIG. 6A, S39a and 41a in FIG. 7). Therefore, it is possible to give advice that is reasonably accepted by the user.

Further, the advice system according to the embodiment of the present invention has a first database in which the relationship between the user's situation and the first advice is associated (for example, the advice information group 0 in FIG. 1A, FIGS. 4A and 5A, A second database (for example, the advice information group 1, the advice information group 2, and FIG. 4 (for example, FIG. 1A)) that stores the relationship between the user's reaction and the second advice in association with the advice 0 in FIG. 5B (a). It has a) and assist-type advice, vision-type advice, etc. in FIGS. 5A and 5B (a). In this advice system, the user's situation is determined (see, for example, S1 in FIG. 6A), and the first advice is selected from the first database based on the determined user's situation (for example, FIG. 6A). (See S3, S5), determine the user's reaction when presenting the selected first advice to the user (see, for example, S7 in FIG. 6A), and determine when presenting the first advice. According to the reaction, the second advice related to the first advice is selected (see S17 to S25 in FIG. 6A).

Further, the advice system according to the embodiment of the present invention further has a third database (for example, refer to the emotion control DB 16b in FIG. 1A) for storing the second advice in association with the type of the avatar, and the second one. At the time of advice selection, the type of avatar is changed according to the selected second advice (for example, FIG. 3A (a) (c), FIG. 3B emotion control, FIG. 4 (b), FIG. 5B (b)). reference). Since the type of avatar can be changed according to the second advice selected, it is easy to persuade the user and to build a relationship of trust with the user depending on the user's situation.

In one embodiment of the present invention, the operation unit 11, the display unit 12, the sensor cooperation unit 13, the image pickup unit 14, the recording unit 16, the net communication unit 17, and the device communication unit 18 are independent of the control unit 15. And placed it. However, the present invention is not limited to this, and the control unit 15 may be responsible for a part of the function, or any of the above-mentioned parts may be responsible for a part of the function of the control unit 15. Further, the control unit 15 may be configured by software mainly by a CPU and a program, or may be a hardware configuration such as a gate circuit generated based on a program language described by Verilog, or a DSP. (Digital Signal Processor) may be used for configuration. Of course, these may be combined as appropriate.

Further, each part may be a processor each configured as an electronic circuit, or may be each circuit part in a processor composed of an integrated circuit such as an FPGA (Field Programmable Gate Array). Alternatively, a processor composed of one or more CPUs may execute the functions of each unit by reading and executing the computer program recorded on the recording medium. Further, the control unit 15 is not limited to the CPU, and may realize the function by an element that functions as a controller.

Further, among the techniques described in the present specification, the controls mainly described in the flowchart can be set by a program in many cases, and may be stored in a recording medium or a recording unit. The recording method to the recording medium and the recording unit may be recorded at the time of product shipment, the distributed recording medium may be used, or may be downloaded via the Internet.

Further, in one embodiment of the present invention, the operation in the present embodiment has been described using a flowchart, but the order of the processing procedures may be changed, or any step may be omitted. Steps may be added, and specific processing contents in each step may be changed.

Further, even if the scope of claims, the specification, and the operation flow in the drawings are explained using words expressing the order such as "first" and "next" for convenience, in the parts not particularly explained, the explanation is made. It does not mean that it is essential to carry out in this order.

The present invention is not limited to the above embodiment as it is, and at the implementation stage, the components can be modified and embodied within a range that does not deviate from the gist thereof. In addition, various inventions can be formed by an appropriate combination of the plurality of components disclosed in the above-described embodiment. For example, some components of all the components shown in the embodiment may be deleted. In addition, components across different embodiments may be combined as appropriate.

10 ... Mobile terminal, 11 ... Operation unit, 12 ... Display unit, 13 ... Sensor cooperation unit, 14 ... Imaging unit, 15 ... Control unit, 15a ... User reaction judgment Unit, 15b ... Communication control unit, 15c ... Advice information selection unit, 15d ... Advice information output unit, 16 ... Recording unit, 16a ... User action history recording unit, 16b ... Advice Information recording unit, 16ba ... Advice information group 0, 16bb ... Advice information group 1, 16bc ... Advice information group 2, 17 ... Net communication unit, 18 ... Equipment communication unit, 20 ... 2nd information device, 21 ... Imaging unit, 22 ... Operation unit, 23 ... Display unit, 24 ... Control unit, 24a ... Advice information selection unit, 24b ... Display control unit , 24c ... Communication control unit, 24d ... Recording control unit, 25 ... Communication unit, 26 ... Recording unit, 26a ... Image recording unit, 26b ... Advice information recording unit, 30.・ ・ Advice takeover information, 30a ・ ・ ・ User status reaction history, 30b ・ ・ ・ Advice information

Claims (37)

When the first advice is presented to the user, the user reaction determination unit that determines the reaction of the user and the user reaction determination unit
An advice selection unit that selects one of the second advice classified into a plurality of categories according to the reaction of the user determined by the user reaction determination unit when the first advice is presented.
Advice system equipped with. The advice system according to claim 1, wherein the user reaction determination unit determines whether or not the user can be persuaded based on the determined reaction of the user. The user reaction determination unit determines emotions and others separately, and determines them.
The advice system according to claim 1, further comprising an emotion control unit that controls the user's emotions based on the emotions determined by the user reaction determination unit. The second advice classified into the above-mentioned plurality is divided into the content that appeals to the logical profit and the content that appeals to the sensory temptation, and when presenting the above-mentioned second advice to the above-mentioned user, the above-mentioned content is distributed. The advice system according to claim 1, wherein the two are different. The second advice classified into the plurality of cases is divided into the content for forcing the user and the content for guiding the user, and the distribution of the content is different when presenting the second advice to the user. The advice system according to claim 1, characterized in that. The second advice classified into the plurality of cases is divided into the content with assistance and the content without assistance, and is characterized in that the distribution of the above content is different when presenting the second advice to the user. The advice system according to claim 1. The advice system according to claim 1, wherein the second advice classified into the plurality of cases includes a plurality of proposals having different additional information and can be selected by the user. Claim 1 is characterized in that, when the second advice is presented to the user, the user reaction determination unit determines information regarding the user's reaction when the second advice is accepted or rejected. The advice system described in. The advice system outputs information regarding the second advice from the device presented to the user to the takeover device.
The advice system can output the third advice related to the selected second advice from the advice output unit of the second device related to the action performed by the user based on the second advice. do,
The advice system according to claim 1, wherein the advice system is characterized in that. The second advice selection unit determines whether or not the result determined by the user reaction determination unit is likely to persuade the rejection reaction, and if the result of this determination is likely to be persuasive, the second advice selection unit is described. The advice system according to claim 1, wherein the advice is selected. The advice system according to claim 10, wherein the second advice selection unit selects alternative advice when the result determined by the user reaction determination unit is unlikely to be convincing. The user reaction determination unit determines whether or not the user mentally or physically rejects the advice, and determines whether or not the user physically rejects the advice.
The second database stores the second advice depending on whether it is a mental refusal or a physical refusal.
The second advice selection unit selects the second advice corresponding to the mental refusal or the second advice corresponding to the physical refusal based on the determination result by the user reaction determination unit.
The advice system according to claim 1, wherein the advice system is characterized in that. The advice system according to claim 1, wherein the advice presenting unit presents the first advice to a user by an avatar. The advice system according to claim 13, wherein the avatar is changed according to the preference or operation history of the user. The advice system according to claim 1, wherein the second database is divided into a plurality of types and stored according to the situation of the user. The second aspect of claim 15 is characterized in that the second advice selection unit determines one of the plurality of types based on the determination result by the user reaction determination unit, and selects the second advice. Described advice system. The first advice presentation unit that presents advice belonging to the first category to the user,
A second advice presentation unit that presents advice belonging to the second category to the above users,
The first advice presentation unit, the stress determination unit for determining the stress of the user when the advice is presented to the user by the second advice presentation unit, and the stress determination unit.
Among the stresses determined by the stress determination unit, the advice selection unit that selects the advice belonging to the one with the smaller stress as the advice to the user, and the advice selection unit.
An advice system characterized by having. The first advice presentation section mentioned above presents advice corresponding to mental refusal as the first category, and
The second advice presenting unit presents advice corresponding to physical refusal as the second category.
17. The advice system according to claim 17. The user status determination unit that determines the user status and
The advice presentation section that presents advice to the above users,
A first database that correlates the relationship between the user's situation and the first advice,
A first advice selection unit that selects the first advice from the first database based on the user's situation determined by the user status determination unit.
A user reaction determination unit that determines the reaction of the user when the first advice selected by the first advice selection unit is presented to the user.
A second database that stores the relationship between the user's reaction and the second advice in association with each other,
When the first advice is presented, the second advice selection unit that selects the second advice related to the first advice according to the reaction of the user determined by the user reaction determination unit, and the second advice selection unit.
Advice system equipped with. In an advice method in an advice system having a first database that associates the relationship between the user's situation and the first advice, and a second database that associates and stores the relationship between the user's reaction and the second advice.
Judging the situation of the above user,
Based on the determined situation of the user, the first advice is selected from the first database.
When the selected first advice is presented to the user, the reaction of the user is determined.
At the time of presenting the first advice, the second advice related to the first advice is selected according to the determined reaction of the user.
An advice method that is characterized by that. The advice system further has a third database that associates and stores the second advice with the avatar type.
The advice method further changes the type of the avatar according to the selected second advice when the second advice is selected.
20. The advice method according to claim 20. Judge the user's situation and
Based on the determined situation of the above user, select the first advice from the first database,
When the selected first advice is presented to the user, the reaction of the user is determined.
At the time of presenting the first advice, one of avatars and items having different modes classified into a plurality of types can be selected and displayed according to the determined reaction of the user.
An advice method that is characterized by that. The advice method according to claim 22, wherein the avatar and the item can be combined independently. 22 or 23. The advice method of claim 22 or 23, wherein the item comprises at least one of clothing, hairstyle, background, facial expression, voice, and gender. It is provided in an advice system having a first database that associates the relationship between the user's situation and the first advice, and a second database that associates and stores the relationship between the user's reaction and the second advice. It is a program to make the computer give advice to the above users.
Judging the situation of the above user,
Based on the determined situation of the user, the first advice is selected from the first database.
When the selected first advice is presented to the user, the reaction of the user is determined.
At the time of presenting the first advice, the second advice related to the first advice is selected according to the determined reaction of the user.
A program characterized by having the above-mentioned computer execute such a thing. When the first advice is presented to the user, the user reaction determination unit that determines the reaction of the user and the user reaction determination unit
A transmission unit that transmits the reaction of the user and / or information related to the reaction determined by the user reaction determination unit to the outside.
The advice presentation unit that presents the second advice according to the information obtained from the outside,
An advice terminal characterized by being equipped with. When the first advice is presented to the user, the reaction of the above user is determined.
The above-determined reaction of the user and / or the information related to the above reaction is transmitted to the outside.
Presenting a second piece of advice according to the information obtained from the outside,
An advice method on an advice terminal that is characterized by that. When the first advice is presented to the user, the reaction of the above user is determined.
The above-determined reaction of the user and / or the information related to the above reaction is transmitted to the outside.
Presenting a second piece of advice according to the information obtained from the outside,
A program characterized by having a computer do things. A user reaction acquisition unit that acquires the user's reaction obtained when the first advice is presented to the user from the outside,
An advice selection unit that selects a second advice based on the reaction of the user acquired by the user reaction acquisition unit, and
An information transmission unit that sends information to the outside so that the above-selected advice can be presented to the above users.
A server for advice characterized by being equipped with. Obtaining the user's reaction obtained when presenting the first advice to the user from the outside,
Select the second advice from the above obtained reaction of the above user,
Send information to the outside so that the above selected advice can be presented to the above users,
The advice method in the advice server, which is characterized by that. Obtaining the user's reaction obtained when presenting the first advice to the user from the outside,
Select the second advice from the above obtained reaction of the above user,
Send information to the outside so that the above selected advice can be presented to the above users,
A program characterized by having a computer in the server for advice execute this. In the advice method that can be displayed by selecting from different avatars classified into multiple types and items that modify the avatar.
Judgment step to judge user status and
The step of selecting the above item based on the above-determined user situation, and
An advice method characterized by having. The determination step includes an emotion determination step for determining an emotional reaction based on the user situation.
Further, the advice method according to claim 32, wherein when the emotional reaction deviates from a predetermined level, a step for changing the item is provided prior to the verbal advice. The advice method according to claim 33, wherein the predetermined level is determined from the history of the user situation according to an emotional reaction when the user refuses the advice. The advice method according to claim 32, wherein the user situation is determined from coordinate values of a graph having a plurality of axes or items in a tabular database. The advice method according to claim 32, wherein the avatar and the item can be independently classified and recorded, and the item can be emphasized. 32. The item is characterized in that it includes at least one of clothes, a hairstyle, a background, a style, a posture, a facial expression, a voice, a tone, a background, a make-up, an aura, and a halo that can modify the avatar. The advice method described.

Images