JP2023032649A - Information processing system and information processing method - Google Patents

Abstract

To provide an information processing system and an information processing method capable of performing appropriate communication while using an expression for a user to easily understand.SOLUTION: An information processing system detects an expression mode of a user, determines an expression mode of an agent to be a simulation object, sets a simulation degree of the expression mode of the agent to the detected expression mode of the user about the expression mode, and generates output data corresponding to the simulation degree. In such a case, the simulation degree of the expression mode of the agent is set so as to come close to the expression mode of the user with time as the generated output data is outputted to the user.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system and an information processing method.

When outputting response information using a voice response system, the user's utterance characteristics such as abbreviations, nicknames, and common names used in the user's utterance are used instead of always responding using the formal name or typical phrase. There is known an information processing apparatus that recognizes a user's name as a user and outputs response information using a name according to the user's utterance characteristics (see Patent Document 1).

International Publication No. 2017/199486 pamphlet

In the above-described conventional information processing apparatus, response information of the voice response system is generated using the name used in the user's utterance. In a communication device using an agent, it is easier for a user to understand information presented in a manner similar to the expression used by the user than information presented in a manner different from the expression used by the user. However, if the agent excessively imitates the expression used by the user, the user may feel uncomfortable with the agent's expression or feel discomfort, which may hinder appropriate communication. Therefore, it is desired to perform appropriate communication while using expressions that are easy for users to understand in devices using agents.

The problem to be solved by the present invention is to provide an information processing system and an information processing method that enable appropriate communication while using expressions that are easy for the user to understand.

The present invention detects the expression mode of a user, determines the expression mode of an agent to be imitated, sets the degree of imitation of the agent's expression mode with respect to the detected expression mode of the user, and sets the degree of imitation. Generates output data according to In this case, the above problem is solved by setting the degree of imitation of the expression mode of the agent so as to approach the expression mode of the user over time as the generated output data is output to the user.

According to the present invention, it is possible to provide an information processing system and an information processing method that enable appropriate communication while using expressions that are easy for users to understand. In particular, when information is provided using an agent, the degree of imitation of the user's expression mode is appropriately changed, so an effect that the user can easily feel attachment to the agent can be expected.

1 is a block diagram showing an embodiment of an information processing system according to the present invention; FIG. 2(a) and 2(b) are diagrams of the interior of a vehicle showing an example of an installation location of the agent device of FIG. 1. FIG. Each of (a) to (c) is a diagram showing an example of an agent device including an anthropomorphic agent according to the present invention. 2 is a diagram showing a configuration example of an expression mode database subjected to information processing by the information processing apparatus of FIG. 1; FIG. 2 is a diagram showing a configuration example of a familiarity database processed by the information processing apparatus of FIG. 1; FIG. 2A and 2B are diagrams (part 1) for explaining intimacy information processed by the information processing apparatus in FIG. 1; FIG. FIG. 7 is a flow chart showing an example of an information processing procedure for intimacy, which is processed based on conversation content input by the user in FIG. 6 ; FIG. 2A to 2C are diagrams (part 2) for explaining intimacy information processed by the information processing apparatus of FIG. 1; FIG. FIG. 9 is a flow chart showing an example of a familiarity information processing procedure that is processed based on a reaction from a user in FIG. 8. FIG. 2 is a diagram for explaining information processing for determining an expression mode of an agent to be imitated, which is executed by the information processing apparatus of FIG. 1; FIG. Each of (a) to (c) is a diagram for explaining the information processing for determining the imitation target of the agent based on the expression mode of FIG. 10 and setting the imitation degree of the expression mode to be the imitation target. 1). Each of (a) to (c) is a diagram for explaining the information processing for determining the imitation target of the agent based on the expression mode of FIG. 10 and setting the imitation degree of the expression mode to be the imitation target. 2). Each of (a) to (c) is a diagram for explaining the information processing for determining the imitation target of the agent based on the expression mode of FIG. 10 and setting the imitation degree of the expression mode to be the imitation target. 3). 4 is a diagram for explaining another example of information processing for setting the degree of imitation of an agent's expression mode, which is executed by the information processing apparatus of FIG. 1; FIG. FIG. 11 is a diagram (part 1) for explaining still another example of information processing for setting the degree of imitation of an agent's expression mode, which is executed by the information processing apparatus of FIG. 1; FIG. 11 is a diagram (part 2) for explaining still another example of information processing for setting the degree of imitation of an agent's expression mode, which is executed by the information processing apparatus of FIG. 1; 2 is a flow chart showing an example of an information processing procedure executed in the information processing system of FIG. 1; FIG. 18 is a flow chart showing an example of a subroutine executed in step S8 of FIG. 17; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of an information processing system 1 according to the present invention, and FIGS. It is a diagram of the interior of the vehicle showing the. The information processing system 1 according to the present invention provides the user with an agent function by an anthropomorphic agent 52 (hereinafter simply referred to as the agent 52), specifically voice, image, motion of a character robot, and combinations thereof. The agent device 5 is used to provide information or interact with the user through the medium of .

The agent device 5 includes a speaker and other audio output units for outputting voice and sound effects, and a display and other display units for displaying images including characters. image is provided to the user to output communication information. In this embodiment, the agent 52 is a three-dimensional object such as a character robot, but is not limited to this, and may be a two-dimensional image displayed on the display.

As shown in FIGS. 1, 2(a) and 2(b), the agent device 5 of the present embodiment has a human-like character robot as an agent 52 that can appear and disappear on a base 51 by an actuator (not shown). is provided in When the agent device 5 receives a control command from the output unit 28 and communicates with the user using the agent function, the agent device 5 emerges from the base 51 as shown in FIG. A character robot is housed in a base 51 as shown in FIG.

The character robots imitating humans shown in FIGS. 1, 2(a) and (b) are examples of anthropomorphic agents 52, and even if they do not imitate humans, they can be animals, plants, predetermined characters, avatars, and so on. It may be the agent 52 displaying an icon. The agent 52 may be provided as a physical individual, or may be displayed on the display as an image in the form of a person, an animal, a plant, or a predetermined character as the agent 52 .

Also, although an example in which the agent device 5 is provided in a vehicle will be described below, the form and installation location of the agent device 5 are not limited to this. The agent device 5 may be an electronic device having a personified agent and an agent function. For example, it may be a portable speaker-type electronic device or an electronic device with a display as shown in FIG. 3(b). may Furthermore, the functions related to audio output and video output of the agent device 5, which will be described below, may be installed in a mobile phone such as a smart phone as shown in FIG. 3(c).

The information processing system 1 of the present embodiment detects expressions included in the user's actions and conversations, and an anthropomorphic agent 52, as shown in FIGS. Communication is performed by imitating the mode of expression. By using so-called mirroring, which imitates expressions, information that is easy for the user to understand can be provided, and good communication can be performed. On the other hand, if the agent 52 excessively imitates the user's mode of expression, the user feels uncomfortable with the expression of the agent 52, or feels uncomfortable. Therefore, the information processing system 1 sets the degree of imitation so that the expression mode of the agent 52 gradually approaches the expression mode of the user as communication is repeated. In the present embodiment, a user refers to a person who uses the information processing system 1, and an example of application to a driver of a vehicle will be described below. , these drivers and fellow passengers are also simply referred to as passengers.).

Returning to FIG. 1, the information processing system 1 of the present embodiment includes a ROM (Read Only Memory) storing programming for executing various processes, and executing the program stored in the ROM to perform information processing. An information processing device 2 having a CPU (Central Processing Unit) as an operating circuit functioning as a device and a RAM (Random Access Memory) functioning as an accessible storage device, vehicle sensors 3, and an input device 4 , and an agent device 5 as an output device. These devices are connected, for example, by a CAN (Controller Area Network) or other in-vehicle LAN, and can mutually transmit and receive information. As shown in FIG. 1, the information processing apparatus 2 has an occupant identification unit 21, an expression mode detection unit 22, an expression mode database 23, and a familiarity A determination unit 24 , a familiarity database 25 , an expression mode setting unit 26 , a data generation unit 27 , and an output unit 28 are provided.

The occupant identification unit 21 identifies the driver D based on information acquired from the vehicle sensors 3 and the input device 4, and at least temporarily stores the information. The vehicle sensors 3 are, for example, an in-vehicle camera that is provided near the rear-view mirror to capture an image of the interior of the vehicle. The occupant identification unit 21 identifies the driver D based on the image information captured by the in-vehicle camera and the voice information input to the in-vehicle microphone. The occupant identification unit 21 detects the expression mode of the identified attribute information (identification information) of the driver D, the image information of the driver D obtained from the vehicle sensors 3, and the voice information of the driver D obtained from the input device 4. It outputs to the unit 22 and the familiarity determination unit 24 .

In order to specify the driver D, an input signal from a seat sensor provided inside the seat portion of the seat or a seat belt sensor may be used. Further, for example, the identification data of the driver is stored in the wireless key of the vehicle, and the identification data of the driver D is automatically or semi-automatically read when the vehicle is unlocked or started using the wireless key. You may

The expression detection unit 22 detects the expression of the driver D based on the attribute information (identification information), the image information, and the audio information of the driver D received from the occupant identification unit 21, and accumulates the expression in the expression database 23. do. The expression mode of the driver D is an expression used when the driver D expresses his/her will, emotion, thought, etc. by actions or language. The expression mode includes an action mode having visual features used by the driver D and a conversation mode having auditory features used by the driver D. The behavior mode and conversation mode of the driver D detected by the expression mode detection unit 22 are used to determine the expression mode of the agent 52 to be imitated when the agent 52 outputs communication information. This is because the agent 52 imitates the expression used by the driver D, so that the driver D can communicate in a manner that is easy for the driver D to understand. In addition, the agent 52 imitates the expressions actually used by the driver D, so that the expression of the agent 52 can be expected to be favorably received by the driver D.

The expression mode detection unit 22 detects, but is not limited to, the driver D's appearance including clothes and hairstyle, appearance including face and body, posture when driving the vehicle, joy, and so on, from image information obtained by imaging the driver D. Visual features such as facial expressions such as anger and sorrow, and actions such as gestures may be detected as the action mode. Since the driver D can easily notice the imitation in the behavior detected from these visual features, the gradual imitation by the agent 52 can prevent the impression of excessive imitation.

In addition, although not particularly limited, the expression mode detection unit 22 detects, from the voice information input by the driver D, the voice quality of the driver D, the intonation, the phrasing including the use of the tone, honorifics, and buzzwords. Features relating to tone such as language and dialect, content of conversation including topics and preferences, auditory features such as estimated age and gender may be detected as the mode of conversation. It is difficult for the driver D to notice the imitation of the conversation mode detected from these auditory features compared to the action mode, so that the agent 52 gradually imitates the imitation, thereby enabling good communication.

The expression mode database 23 stores the attribute information (identification information) of the driver D and the expression mode of the driver D detected by the expression mode detection unit 22 in association with each other. FIG. 4 is a diagram showing a configuration example of the representation mode database 23. As shown in FIG. The expression mode database 23 stores the attribute information of the driver D, the detected expression mode, information on the date and time when the expression mode was detected, and the detection history such as the total number of times of detection. Although the representation mode database 23 is included in the information processing apparatus 2, it may be configured to store or acquire various types of information by communicating with an external server.

As shown in FIG. 4, the attribute information of the occupant stores, for example, the father as user 1 and the detection history of the expressions used by the father of user 1 . At 13:00 on March 18, 2020, from the voice information input by User 1's father, the conversation mode of "Okay" and from the image information of User 1's father, <Thumb up > is detected, and the total number of detections of these expression modes is 108 times. Similarly, at 11:11 on January 11, 2020, the conversation mode of “understood” and the action mode of <making a circle with the thumb and index finger> (accumulated detection times 502 times) were recorded on December 8, 2019. At 6 o'clock, the conversation mode of "understand" and the action mode of <salute> (cumulative number of detections: 49 times), and at 7:59 on November 28, 2019, the conversation mode of "raja" and <one hand It is stored that the operation mode of "raise up and turn the other hand to the side" was detected (accumulated number of detections: 9 times). As for the expression mode, either the conversation mode or the motion mode may be detected and stored. As shown in FIG. 4, both the conversation mode and the motion mode may be detected and stored in association with each other. . Further, the expression mode detected by the expression mode detection unit 22 may be classified into items such as "acceptance" and stored.

The degree of familiarity determination unit 24 estimates the degree of familiarity between the driver D and the agent 52 and stores it in the degree of familiarity database 25 . A higher degree of familiarity indicates that the driver D accepts the agent 52 and is accustomed to communicating with the agent 52, and a lower degree of familiarity indicates that the driver D is not accustomed to communicating with the agent 52. show. The familiarity of the driver D estimated by the familiarity determination unit 24 is used to determine the degree of imitation of the expression mode of the agent 52 when the agent 52 outputs communication information. Note that the familiarity determination unit 24 and the familiarity database 25 are not essential components of the present invention, and may be omitted as necessary.

The familiarity database 25 stores the attribute information (identification information) of the driver D and the familiarity of the driver D estimated by the familiarity determination unit 24 in association with each other. FIG. 5 is a diagram showing a configuration example of the familiarity database 25. As shown in FIG. The familiarity database 25 stores passenger attribute information and familiarity, as well as the riding history of the driver D acquired from the vehicle sensors 3, and information on various in-vehicle devices such as the driving support device, the navigation device, and the agent device 5. Usage history may be stored. As shown in FIG. 5, in the attribute information of the occupant, for example, the father is recorded as user 1, and for the father of user 1, the familiarity with the agent 52, the vehicle boarding history, and the usage history of various in-vehicle devices are stored. be.

The degree of familiarity determination unit 24 estimates the degree of familiarity between the driver D and the agent 52 based on the attribute information (identification information) and voice information of the driver D received from the occupant identification unit 21, for example. Specifically, the content of the conversation included in the voice information input by the driver D is classified according to the field and depth of the topic, and digitized to calculate the degree of intimacy. The deeper the topic, the more specific and private events are included, and the shallower the topic, the more abstract and general. Therefore, as the conversation content included in the voice information of the driver D increases and becomes deeper, it can be estimated that the driver D accepts the agent 52, and it can be determined that the degree of intimacy is high.

Classification of topic field and depth is not particularly limited, but as shown in FIG. Personality/physical appearance, [4] friendships/heterosexual relationships, and [5] financial relationships, and the depth is classified into (1) general topics such as news, and (2) personal topics including the positive information of the speaker. , (3) categorize personal topics that contain negative information about the speaker. Personal topics have more depth than general topics such as news, and personal topics including negative information have more depth than positive information. A score of 1 for general topics, a score of 3 for personal topics containing positive information about the speaker, and a score of 5 for personal topics containing negative information about the speaker. Digitize.

As shown in FIG. 6B, the intimacy determination unit 24 quantifies the depth of the fields [1] to [5] based on the content of the conversation included in the voice information input by the driver D, Calculate the average value. In the example of the father of user 1, based on the father's conversation content, [1] hobbies and views of society (opinions and attitudes) include (3) personal topics including the speaker's negative information. For [2] Life/Study, (1) general topics such as news are included, so 1 point is given, and so on. Let the average value 3.8 of the points up to and including the user 1's father's intimacy. In this way, by calculating the degree of intimacy using the content of the conversation included in the voice information input by the driver D, the actual behavior of the driver D with respect to the agent 52 can be reflected in the degree of intimacy. Appropriate communication between person D and agent 52 can be facilitated.

In the present embodiment, the field of topic is classified into five items and the depth of topic is classified into three items and quantified, but the number of items is not limited to this. Also, if the depth of the topic is the same, the same score (score) was assigned to each field. You may set so that a score may become high so that it may go. In addition, although the average value of the fields [1] to [5] is used to calculate the degree of intimacy, the median value may be used, or the degree of intimacy may be calculated by an addition method or a subtraction method. In addition, instead of numerical values, levels such as "high", "medium", and "low" may be used as the degree of intimacy. Note that the familiarity determination unit 24 may appropriately update the once-calculated familiarity based on the content of the conversation included in the newly acquired voice information of the driver D.

FIG. 7 is a flow chart showing an example of a familiarity information processing procedure based on the content of the conversation input by the driver D. As shown in FIG. 4 is a flow chart showing information processing for calculating a degree of familiarity from conversation content included in voice information of a driver D, which is executed by the degree of familiarity determination unit 24. FIG. In the following description, it is assumed that the familiarity of the driver D is already stored in the familiarity database 25 .

First, at step S101 in FIG. 7, when the ignition switch of the vehicle is turned on, the following information processing is executed. In step S<b>102 , the occupant identification unit 21 identifies the driver D based on the image information acquired from the vehicle sensors 3 . Next, in step S<b>103 , the familiarity determination unit 24 acquires voice information of the driver D input via the input device 4 .

In subsequent step S104, the intimacy determination unit 24 classifies the conversation content included in the voice information of the driver D according to the field and depth of the topic as shown in FIG. ) to calculate the degree of intimacy. In step S<b>105 , it is determined whether or not the calculated familiarity score is higher than the familiarity of the driver D recorded in the familiarity database 25 .

As a result of the determination in step S105, if it is determined that the calculated familiarity is higher than the familiarity of driver D stored in the familiarity database 25, the process proceeds to step S106 to increase the familiarity of driver D. update. On the other hand, if it is determined in step S105 that the calculated degree of familiarity is not higher than the degree of familiarity of driver D recorded in the familiarity database 25, the driving Maintain the closeness of person D as it is.

In step S108, the information processing from step S103 to step S107 is repeatedly executed until the ignition switch of the vehicle is turned off. Note that the calculation of the degree of intimacy may be repeated at predetermined intervals while the driver D is in the vehicle, or the information processing may be performed once for each time the driver D gets in the vehicle.

Further, the familiarity determination unit 24 estimates the degree of familiarity between the driver D and the agent 52 based on the attribute information (identification information), image information, and audio information of the driver D received from the occupant identification unit 21. good. Specifically, after the agent 52 outputs an expression mode imitating the driver D, the image information and voice information of the driver D are obtained. Then, the reaction of the driver D included in the image information and the voice information is detected, and the expression mode output by the agent 52 and the expression mode responded by the driver D are compared. It is estimated that the higher the degree of similarity between the expression form output by the agent 52 and the expression form responded by the driver D, the higher the degree of intimacy, and the lower the degree of similarity, the lower the degree of intimacy. In this way, by estimating the degree of intimacy based on the reaction of the driver D to the expression mode output by the agent 52, appropriate communication between the driver D and the agent 52 can be further promoted.

For example, as shown in FIG. 8A, it is assumed that the agent 52 outputs a conversation mode of "understood" and an action mode of <make a circle with the thumb and forefinger>. As shown in FIG. 8(b), when the driver D responds to this expression mode of the agent 52 with the action mode of <pointing the thumb up>, the agent 52 outputs <with the thumb and forefinger Since the degree of similarity between the action mode of "making a circle" and the action mode of "pointing the thumb up" responded by the driver D is low, it is determined that the degree of intimacy is low. On the other hand, as shown in FIG. 8(c), when the driver D responds with the action mode of <making a circle with the thumb and forefinger>, the action mode output by the agent 52 and the driver D Since the similarity of the motion modes responded by is high, it is determined that the degree of intimacy is high. As for the conversation mode, similarly, the conversation mode output by the agent 52 is compared with the conversation mode responded by the driver D, and the degree of intimacy is determined based on the degree of similarity. Note that the familiarity determination unit 24 may appropriately update the once-estimated familiarity based on the newly acquired image information and voice information of the driver D.

FIG. 9 is a flow chart showing an example of a familiarity information processing procedure based on the reaction of the driver D. As shown in FIG. 4 is a flow chart showing information processing for estimating familiarity based on a response of driver D to an expression mode of agent 52, which is executed by familiarity determination unit 24. FIG. In the following description, it is assumed that the familiarity of the driver D is already stored in the familiarity database 25 . It should be noted that the information processing of familiarity based on the content of conversation of driver D in FIG. 7 and the information processing of familiarity based on reaction of driver D in FIG. 9 may be combined as appropriate.

First, in step S111 of FIG. 9, when the ignition switch of the vehicle is turned on, the following information processing is executed. In step S<b>112 , the occupant identification unit 21 identifies the driver D based on the image information acquired from the vehicle sensors 3 . Next, in step S<b>113 , the degree of intimacy determination unit 24 determines whether or not the agent 52 has output an expression that imitates the driver D. If the agent 52 outputs, for example, a conversation mode of "understood" and an action mode of <making a circle with the thumb and forefinger> as shown in FIG. 8A, the process proceeds to step S114. On the other hand, when the agent 52 does not output the representation mode imitating the driver D, step S113 is repeated for a predetermined time.

As a result of the determination in step S113, when it is determined that the agent 52 has output the representation mode imitating the driver D, in step S114, the degree of familiarity determination unit 24 outputs the representation mode after the agent 52 outputs the representation mode. In order to detect the reaction of driver D, image information and/or audio information of driver D is acquired. In subsequent step S<b>115 , the familiarity determination unit 24 determines whether or not the driver D has responded to the expression mode of the agent 52 based on the image information and/or voice information of the driver D. If it is determined that the driver D has responded to the expression mode of the agent 52, the process proceeds to step S116. On the other hand, when it is determined that the driver D has not responded to the expression mode of the agent 52, the process proceeds to step S118.

As a result of the determination in step S115, if it is determined that the driver D has responded to the expression form of the agent 52, then in step S116, the expression form output by the agent 52 is compared with the expression form responded by the driver D. to determine the degree of similarity. For example, as shown in FIG. 8(c), the agent 52 outputs the conversation mode of "understood" and the action mode of <make a circle with the thumb and forefinger>. If D responds in the action mode of <making a circle with the thumb and forefinger>, it is determined that the degree of similarity is high, and the process proceeds to step S117 to increase and update the intimacy of driver D. On the other hand, as shown in FIG. 8(b), when the driver D responds with the action mode of <thumbs up>, the action mode output by the agent 52 and the response of the driver D are It is determined that the degree of similarity between the motion modes obtained is low, and the process proceeds to step S118.

As a result of the determination in step S115, if it is determined that the driver D has not responded to the expression mode of the agent 52, and as a result of the determination in step S116, the expression mode output by the agent 52 and the expression that the driver D responded to If it is determined that the degree of similarity between the modes is low, the degree of intimacy of driver D is maintained at step S118.

The information processing from step S113 to step S118 is repeatedly executed until the ignition switch of the vehicle is turned off at step S119. Note that the estimation of familiarity may be repeated at predetermined intervals while the driver D is in the vehicle, or may be configured to execute information processing once for each time the driver D gets in the vehicle.

Furthermore, the degree of familiarity determination unit 24 may estimate the degree of familiarity between the driver D and the agent 52 based on the driver D's use history of various in-vehicle devices acquired from the vehicle sensors 3 . For example, when the frequency of use of an in-vehicle device such as a driving assistance device, a navigation device, or the agent device 5 is equal to or higher than a predetermined value, it can be determined that the receptivity to the in-vehicle device is high. Assume high density. On the other hand, if the usage frequency of the in-vehicle device is equal to or less than the predetermined value, it can be determined that the receptivity to the in-vehicle device is low, so it is estimated that the degree of intimacy between the driver D and the agent 52 is low. Although the predetermined value is not particularly limited, it is a value suitable for judging the acceptability of the in-vehicle device, such as the frequency of use of various in-vehicle devices being 3 times with respect to the history of driving the vehicle 5 times. By estimating the degree of intimacy based on the usage status of various in-vehicle devices, it is possible to reflect the receptivity of the driver D to the in-vehicle devices in the communication between the driver D and the agent 52 .

Returning to FIG. 1, the expression mode setting unit 26 determines the expression mode of the agent 52 to be output to the driver D. FIG. Specifically, when a trigger expression included in the image information of the driver D acquired from the vehicle sensors 3 or the voice information of the driver D acquired from the input device 4 is detected, the driving behavior stored in the expression mode database 23 is detected. The representation mode of the person D is referred to, and the representation mode of the agent 52 to be imitated is determined.

Furthermore, the expression mode setting unit 26 sets the degree of imitation for the determined expression mode of the agent 52 . The degree of imitation is the degree of similarity of the agent's 52 expression mode to the driver's D expression mode. The higher the degree of imitation, the higher the degree of similarity between the two, and the closer the expression mode of the agent 52 is to the expression mode of driver D. move away from the mode of expression of

As described above, when the agent 52 communicates in an expression mode similar to that used by the driver D, information that is easy for the driver D to understand can be provided. However, if the agent 52 imitates the expression of the driver D too much, for example, the agent 52 suddenly imitates the expression of the driver D, the driver D feels uncomfortable with the expression of the agent 52 or feels uncomfortable. and solicit Therefore, the expression mode setting unit 26 sets the degree of imitation so that the expression mode of the agent 52 gradually approaches the user's expression mode over time, that is, as communication is repeated.

In this manner, the expression mode used by the driver D is set as an imitation target, and the expression mode of the agent 52 is determined, and the expression mode of the agent 52 is gradually brought closer to the expression mode of the driver D as the communication is repeated. Set the degree of imitation to As a result, the driver D can be prevented from feeling uncomfortable with the expressions of the agent 52 or feeling uncomfortable, and appropriate communication can be performed while using expressions that are easy for the driver D to understand.

After setting the degree of imitation, the expression mode setting unit 26 determines the degree of imitation used when the agent 52 outputs communication information. Then, along with the expression mode of the agent 52 determined to be the target of imitation, information on the degree of imitation output by the agent 52 is output to the data generation unit 27 . The processing for determining the expression mode of the agent 52 and the processing for setting the degree of imitation by the expression mode setting unit 26 will be described later.

After setting the degree of imitation, the expression mode setting unit 26 may use the degree of intimacy between the driver D and the agent 52 to determine the degree of imitation used when the agent 52 outputs communication information. . Specifically, the degree of familiarity between the driver D and the agent 52 stored in the degree of familiarity database 25 is acquired via the degree of familiarity determination unit 24, and if the degree of familiarity is relatively high, the degree of familiarity is relatively high. It is determined to output an expression mode with a higher degree of imitation than when the degree of imitation is relatively low.

The higher the degree of imitation, the closer the representation of the agent 52 is to the representation of the driver D, so the driver D is more likely to notice the imitation. On the other hand, the lower the degree of imitation, the farther away from the expression mode of the driver D, so it is difficult for the driver D to notice the imitation. When the degree of intimacy between the driver D and the agent 52 is low, the driver D's receptivity to the agent 52 is low, and it is likely that the imitation is excessive. Therefore, an expression mode with a low degree of imitation is output to make it difficult for the driver D to notice imitation.

On the other hand, when the degree of intimacy between the driver D and the agent 52 is relatively high, the receptivity of the driver D to the agent 52 is high, and it is difficult to feel that the imitation is excessive. may be output. This is because, as the degree of intimacy increases, the driver D becomes more likely to feel attachment to the agent 52 when he notices that the mode of expression of the agent 52 resembles his own mode of expression. In addition, when the degree of intimacy is particularly high, the peculiar expressions used by the driver D (for example, the conversational mode of "rajah" shown in FIG. > operation mode), etc. may be output. In this way, by appropriately changing the degree to which the agent 52 imitates the expression mode of the driver D, smooth communication can be performed.

The data generation unit 27 generates output data to be output to the driver D based on the expression mode of the agent 52 to be imitated and the information on the degree of imitation of the expression mode received from the expression mode setting unit 26. do. When generating output data in a conversation mode, text data (character string) is first generated, the text data is converted into voice data by speech synthesis processing, and this is output as output data in a conversation mode to be output by the agent 52. 28. When generating the output data of the action mode, digital data is generated using a preset motion path and motion capture, and the digital data is transmitted to the output unit 28 as the output data of the action mode output by the agent 52 . Known techniques can be applied for voice synthesis processing, motion capture, and digital data generation.

Furthermore, the data generator 27 may set the output frequency of the output data by the agent device 5 . The higher the output data output frequency, the higher the frequency of communication between the driver D and the agent 52, thus promoting communication. Therefore, the data generator 27 may refer to the degree of familiarity between the driver D and the agent 52, for example, and set the output frequency of the expression mode of the agent 52 to be high when the degree of familiarity is low. By increasing the output frequency, it can be expected that the agent 52 gives the impression that the driver D is interested. When the degree of familiarity is low, the mode of expression with a low degree of imitation is output with high frequency, and when the degree of familiarity is medium, the mode of expression with a medium degree of imitation is output with a medium frequency, The output frequency may be set in consideration of both the degree of familiarity and the degree of imitation, such as outputting an expression mode with a high degree of imitation at a low frequency when the degree of familiarity is high. Moreover, in addition to changing the output frequency of one representation mode output by the agent 52, when using a plurality of representation modes, the output frequency may be set for each representation mode.

When the output unit 28 receives the output data from the data generation unit 27, the output unit 28 outputs a control signal to the operation unit of the agent device 5, the speaker or other audio output unit, the display or other display unit, and the agent function of the agent device 5 outputs: Print the generated output data.

Next, the processing of determining the expression mode of the agent 52 to be imitated and the setting process of the degree of imitation for the expression mode by the expression mode setting unit 26 will be described. As described above, when the expression mode setting unit 26 detects a trigger expression included in the image information of the driver D acquired from the vehicle sensors 3 or the voice information of the driver D acquired from the input device 4, the expression mode setting unit 26 The representation mode of the driver D stored in the database 23 is referred to, and the representation mode of the agent 52 to be imitated is determined. Then, the degree of imitation is set for the determined expression mode.

FIG. 10 is a diagram for explaining information processing for determining the expression mode of the agent 52 to be imitated. As shown in FIG. 10, the expression mode setting unit 26 detects, for example, the trigger expression of "acceptance" of "that's okay" included in the voice information of the driver D acquired via the input device 4, and the expression An expression mode related to “acceptance” is extracted from the expression modes of the driver D stored in the mode database 23 . Extracted expressions related to "acceptance", for example, the conversational mode of "okay" and the action mode of <pointing the thumb up>, the conversational mode of "understood" and the conversational mode of <making a circle with the thumb and forefinger> From the action mode, the conversation mode of "understanding" and the action mode of <salute>, the conversation mode of "rajah" and the action mode of <raise one hand up and turn the other hand to the side>, it is possible to imitate determine the representation mode of the agent 52. After determining the expression mode of the agent 52 to be imitated, the expression mode setting unit 26 specifies a change target for bringing the expression mode of the agent 52 closer to the expression mode of the driver D, and sets the degree of imitation.

11 to 13 are diagrams showing examples of scenes for setting the degree of imitation of the expression mode of the agent 52. FIG. Each of FIGS. 11A to 11C is a scene in which the degree of imitation is set using different modes of expression. For example, when the action mode and conversation mode of "approval" are determined as the imitation target of the agent 52, the expression mode setting unit 26 specifies the number of detections of each expression mode stored in the expression mode database 23 as a change target. An expression mode with a larger number of detections can be closer to the expression mode used by the driver D. Therefore, the expression mode setting unit 26 sets the degree of imitation to be higher as the action mode and conversation mode used are detected more frequently. If the number of times of detection is changed, it is possible to suppress an unnatural phenomenon in which the agent 52 frequently imitates an expression mode with a small number of detection times, that is, an expression mode that the driver D accidentally takes.

In the scenes shown in FIGS. 11(a) to 11(c), as shown in FIG. 11(a), the degree of imitation is set low for the “understand” conversation mode and <salute> action mode, which are detected less frequently. do. In addition, as shown in FIG. 11(b), the degree of imitation is set to a medium level for the conversation mode of "Okay" and the action mode of <pointing thumb up>, which have a medium number of detections. Similarly, as shown in FIG. 11(c), the degree of imitation is set high for the conversation mode of "understood" and the action mode of <making a circle with the thumb and forefinger>, which are detected a large number of times. Then, the degree of imitation to be reflected in the output data generated this time is determined so that the agent 52 outputs the expressions of FIG. 11(a)→FIG. 11(b)→FIG. A command signal is transmitted to the generator 27 . In the present embodiment, the degree of imitation is set in three stages of "low", "medium", and "high", but the degree of imitation is not limited to this, and the degree of imitation may be two stages or three stages or more. .

Each of FIGS. 12(a) to 12(c) is a scene in which the degree of imitation is set using the same expression mode. For example, when the expression mode setting unit 26 determines the action mode of <making a circle with the thumb and forefinger> among the expression modes of "approval" as the imitation target of the agent 52, the expression mode setting unit 26 sets the action mode of the agent 52 to the motion of the driver D. In order to approximate the mode, the shape of the hand is specified as a change target. The degree of imitation is set higher as the shape of the hand of the agent 52 is more similar to the motion mode of the driver D <a circle is formed with the thumb and forefinger>.

In the scenes shown in FIGS. 12A to 12C, as shown in FIG. Set the intensity low. In addition, as shown in FIG. 12B, the degree of imitation is set to be moderate for the <circle with one hand> shape, which has a medium degree of similarity with the <circle with thumb and forefinger> shape. As shown in FIG. 12(c), the degree of imitation is set high for a shape <circle with thumb and forefinger> that is highly similar to the shape <circle with thumb and forefinger>. Then, the degree of imitation to be reflected in the output data generated this time is determined so that the agent 52 outputs the action modes of FIG. 12(a)→FIG. 12(b)→FIG. A command signal is output to the generator 27 .

Each of FIGS. 13(a) to 13(c) is a scene in which the degree of imitation is set using a plurality of modes of expression. The expression mode setting unit 26 may set the degree of imitation using a plurality of behavior modes and conversation modes. By using a plurality of modes of expression, the expression used by the agent 52 can be varied step by step. The expression mode setting unit 26 determines, for example, the "approval" conversation mode as one imitation target of the agent 52, and specifies the number of detections of the "approval" conversation mode as a change target. Furthermore, another imitation target is determined to be <making a circle with the thumb and forefinger> action mode, and the shape of the hand is specified as a change target. The higher the degree of imitation is set, the more conversation mode is used, and the more similar the shape of agent 52's hand is to the driver's motion mode <a circle is formed with a thumb and forefinger>.

In the scenes shown in FIGS. 13(a) to 13(c), as shown in FIG. 13(a), the conversation mode of “understood” with a low number of detections and the shape of <raise one hand> with a low similarity are imitations. Set the intensity low. Also, as shown in FIG. 13(b), the degree of imitation is set to a medium level for the conversation mode of "Okay" with a medium number of detections and the shape <making a circle with one hand> with a medium degree of similarity. do. Furthermore, as shown in FIG. 13(c), the degree of imitation is set high for the shape of <making a circle with a thumb and forefinger>, which has a high degree of similarity with the conversation mode of “understood”, which is detected a large number of times. Then, the degree of imitation to be reflected in the output data generated this time is determined so that the agent 52 outputs the expressions of FIG. 13(a)→FIG. 13(b)→FIG. A command signal is output to the generator 27 .

Note that the expression mode setting unit 26 may use a plurality of behavior modes and conversation modes to increase the number of expression modes to be imitated over time. For example, the degree of imitation may be set low when using one mode of expression, the degree of imitation may be set medium when using two modes of expression, and the degree of imitation may be set high when using three modes of expression. In this case, the agent 52 reflects in the output data so as to imitate one representation → two representations → three representations over time.

In addition, with respect to the conversation mode, the expression mode setting unit 26 determines voice features, expression features, and conversation content features included in the voice information of the driver D as objects to be imitated, so as to approximate the voice information of the driver D. , the degree of imitation may be set by specifying an item related to each feature as a change target. It is difficult for the driver D to notice the imitation of the features of the voice, and it is rather easy for the driver D to notice the imitation of the features of the expression. In addition, the feature of the conversation content is that the driver D can easily notice the imitation. The expression mode setting unit 26 sets the degree of imitation to a higher level, assuming that the more easily noticeable features are targeted for imitation, the closer they are to the voice information of the driver D.

As shown in FIG. 14, when speech characteristics such as speech speed, tone, voice quality, and volume are targeted for imitation, the degree of imitation is set low. In addition, when the characteristics of expression such as intonation, dialect, phrasing (use of tone, honorifics, buzzwords), and accent are to be imitated, the degree of imitation is set to a medium level. Furthermore, when the features of conversation content such as topics, tastes, and negative information about a specific person are targeted for imitation, the degree of imitation is set high. However, with respect to negative information about a specific person, a predetermined condition is set so that, for example, only when the voice information input by the driver D contains negative information about a specific person, it is targeted for imitation. is preferred. The expression mode setting unit 26 reflects this in the output data so that the agent 52 imitates the characteristics of the speech, the characteristics of the expression, and the characteristics of the content of the conversation over time.

Furthermore, the expression mode setting unit 26 may specify conditions related to time, place, and situation included in the behavior mode and conversation mode of the driver D, and set the degree of imitation by changing these conditions. good. By using these conditions, the imitation can be set step by step from the imitation that the driver D can easily notice to the imitation that is difficult to notice, and appropriate communication can be performed using various expressions.

FIG. 15 is a diagram showing an example in which conditions regarding time, place, and situation are subject to change regarding the operation mode of the driver D. In FIG. The higher the degree of imitation, the closer to the action mode of the driver D, and the setting is made to reproduce the state when the action mode is detected from the driver D.

When the condition regarding the time of the action mode is to be changed, for example, the timing at which the agent 52 outputs the action mode or the length of time for which the action mode is output is changed. The scene shown in FIG. 15 is an imitation target of the motion mode <yawning> of the driver D, and consists of three motions: <close the eyes>, <place one hand on the mouth>, and <stretch out>. This is a scene in which the time conditions are subject to change when the agent 52 imitates the mode. In the scene shown in FIG. 15, when delaying the timing of outputting the motion mode by one motion and outputting two motions of <close your eyes> and <place one hand on your mouth>, the degree of imitation is set low. do. In addition, the timing of outputting the action mode is delayed by one action, and when outputting the three actions of <close your eyes>, <place one hand on your mouth>, and <stretch>, the degree of imitation is set to medium. set. On the other hand, when outputting the three motions of <close your eyes>, <place one hand on your mouth>, and <stretch> while aligning the timings of outputting the motion modes, the degree of imitation is set high. .

Similarly, when the conditions relating to the location of the action mode are to be changed, for example, the locations and positions to be imitated by the agent 52 are changed. The scene shown in FIG. 15 is a scene in which the motion mode of the driver D <stretching the left arm> is targeted for imitation, and the condition of the location is targeted for change. In the case of the scene shown in FIG. 15, when outputting the action mode of <extending the left leg> away from the left arm, the degree of imitation is set low. In addition, when outputting the action mode <stretching the right arm> in the opposite position to the left arm, the degree of imitation is set to medium, and when outputting the motion mode <stretching the left arm> in the same position, the degree of imitation is set high. set.

In the case of changing the conditions regarding the situation of the action mode, for example, the situation that the agent 52 imitates is changed. The scene shown in FIG. 15 is a scene in which the <yawning> action mode of the driver D is to be imitated, and the condition regarding the situation is to be changed. Since the <yawning> motion mode of the driver D is detected in the morning, the condition regarding the situation is morning. In the case of the scene shown in FIG. 15, when outputting the <yawn> action mode at night in a situation different from that in the morning, the degree of imitation is set low. If the <yawning> motion mode is to be output in the afternoon in a situation slightly closer to morning, the degree of imitation is set to medium, and if the <yawning> motion mode is to be output in the morning of the same situation, the degree of imitation is set to be higher. set.

Note that the expression mode setting unit 26 may set the degree of imitation with any one of the conditions related to time, place, and situation as the change target for the action mode, or set the degree of imitation with a plurality of conditions as the change targets. may be set. In addition, the expression mode setting unit 26 may set the degree of imitation by associating the conditions regarding the time, place, and situation to be changed with the degree of intimacy of the driver D. For example, if the degree of intimacy with the driver D is low, all of the time, place, and situation conditions are subject to change so that the driver D is less likely to notice the imitation. If the degree of familiarity with the driver D is medium, two of the conditions regarding time, place, and situation are subject to change. If the degree of intimacy with the driver D is high, the condition regarding the situation is a condition with high sympathy. The expression mode setting unit 26 reflects in the output data so that the agent 52 uses an expression mode with a low degree of imitation over time to an expression mode with a high degree of imitation.

FIG. 16 is a diagram showing an example in which conditions relating to time, place, and situation are subject to change regarding the manner of conversation of driver D. In FIG. The higher the degree of imitation, the higher the responsiveness to the conversation mode of the driver D, and the agent 52 is set to output an sympathetic response. As shown in FIG. 16, for example, when detecting the conversation content of "National highway No. 129 is congested today as well" included in the conversation mode of driver D, the time condition is "today" and the location condition is "National highway No. 129" is extracted as a condition, and the word "congested" is extracted as a condition of the situation, and these words are changed. Based on the detected values obtained through the vehicle sensors 3, time conditions and current time information, location conditions and vehicle position information, situation conditions and vehicle surrounding information, etc. are associated and changed. may

When the conditions related to the time of the conversation mode are to be changed, for example, the word related to the time is changed, and the current time information is changed in association with it. In the scene shown in FIG. 16, the word "today" is changed as the time-related condition, and "commuting time zone" is changed as the current time information. When using a response sentence such as "National highway No. 129 will be congested during the tourist season," which is a concept that is separate from the time "today", the degree of imitation is set low. Also, when using a response sentence such as "National Highway No. 129 is congested this week as well," which is a concept slightly different from the time "today", the degree of imitation is set to a medium level. On the other hand, when using a response sentence such as "It's commuting time, so National Route 129 is congested." set.

Similarly, when the condition regarding the location of the conversation mode is to be changed, for example, the word regarding the location is changed, and the positional information of the vehicle is changed in correspondence. In the scene shown in FIG. 16, the word "National highway No. 129" is changed as the location condition, and the position information of the vehicle "near Atsugi bypass" is changed in association with it. When using a response sentence such as "Main roads are often congested", which is a concept distant from the place "National highway No. 129", the degree of imitation is set low. In addition, when using a response sentence such as "National highway is often congested", which is a concept slightly different from the place "National highway No. 129", the degree of imitation is set to a medium level. On the other hand, when using a response sentence such as "National Route 129 near the Atsugi Bypass is often congested, isn't it?" set high.

When the condition regarding the situation of the conversation mode is to be changed, for example, the words related to the situation are changed, and the surrounding information of the vehicle is also changed. In the scene shown in FIG. 16, the word "congested" is changed as the condition of the situation, and the surrounding information of the vehicle "near lane regulation" is changed in association with it. When using a response sentence such as "There are times when there is no congestion on Route 129," which is a concept apart from the situation of "congested", the degree of imitation is set low. Also, when using a response sentence such as "National Highway No. 129 is congested", which is the same concept as the situation of "congested", the degree of imitation is set to a medium level. On the other hand, in addition to the situation of "congested", when using a response sentence such as "It seems that National Highway 129 is congested due to lane restrictions ahead", which is associated with information about the surroundings of the vehicle. sets the degree of imitation high.

Note that the expression mode setting unit 26 may set the degree of imitation with any one of the conditions related to time, place, and situation as the change target for the conversation mode, or set the degree of imitation with a plurality of conditions as the change targets. may be set. In addition, the expression mode setting unit 26 may set the degree of imitation by associating the conditions regarding the time, place, and situation to be changed with the degree of intimacy of the driver D. For example, if the degree of intimacy with the driver D is low, all of the time, place, and situation conditions are subject to change so that the driver D is less likely to notice the imitation. If the degree of intimacy with the driver D is medium, two of the conditions regarding time, place, and situation are subject to change. If the degree of intimacy with the driver D is high, the condition regarding the situation is a condition with high sympathy. The expression mode setting unit 26 reflects in the output data so that the agent 52 uses an expression mode with a low degree of imitation over time to an expression mode with a high degree of imitation.

Next, the information processing procedure of the information processing system 1 of the present embodiment will be described with reference to FIGS. 17 and 18. FIG. FIG. 17 is a flowchart showing an example of information processing for determining the expression mode of the agent 52 and setting the degree of imitation, which is executed by the information processing system 1 of the present embodiment. FIG. 18 shows an example of the subroutine of step S8 shown in FIG. 5 is a flowchart showing an example of information processing for determining the degree of imitation output by an agent 52; The following description will be made with reference to the scenes shown in FIGS. 10 and 11(a) to (c).

First, at step S1 in FIG. 17, when the ignition switch of the vehicle is turned on, the following information processing is executed. In step S<b>2 , the occupant identification unit 21 identifies the driver D based on image information obtained from an in-vehicle camera as the vehicle sensors 3 . Next, in step S3, the expression mode setting unit 26 refers to the expression mode database 23 to determine the expression mode stored for the driver D, the number of times the expression mode is detected, and the intimacy between the driver D and the agent 52. to get

In step S4, the expression mode setting unit 26 detects whether or not the trigger expression is detected based on the image information of the driver D obtained from the vehicle sensors 3 and the voice information of the driver D obtained from the input device 4. determine whether For example, when the trigger expression "That's all right (acceptance)" shown in FIG. 10 is detected, the process proceeds to step S5. On the other hand, step S4 is repeated for a predetermined time until the trigger expression is detected.

When it is determined in step S4 that the trigger expression has been detected, in step S5 the expression mode setting unit 26 determines the expression mode of the agent 52 to be imitated. As shown in FIG. 10 , when the trigger expression “accept” of the driver D is detected, the expression mode setting unit 26 refers to the expression mode database 23 and selects the action mode and conversation of “accept” from the expression mode of the driver D. A mode is determined as an imitation target.

In subsequent step S6, the expression mode setting unit 26 specifies change targets for bringing the expression mode of the agent 52 closer to the expression mode of the driver D with respect to the action mode and conversation mode of "acceptance" determined to be imitated. . For example, the number of detections of the motion mode and the conversation mode shown in FIG. 10 are subject to change.

In subsequent step S7, the expression mode setting unit 26 sets the degree of imitation of the agent 52's expression mode. At this time, the expression mode setting unit 26 sets the expression mode of the agent 52 so as to gradually approach the expression mode of the driver D as the communication is repeated. When the number of times of detection is changed, an expression mode with a higher number of detection times can be closer to the expression mode used by the driver D. Therefore, for example, as shown in FIG. 11A, the degree of imitation is set to be low for the "understand" conversation mode and the <salute> action mode, which are detected less frequently. In addition, as shown in FIG. 11(b), the degree of imitation is set to a medium level for the conversation mode of "Okay" and the action mode of <pointing thumb up>, which have a medium number of detections. On the other hand, as shown in FIG. 11(c), the degree of imitation is set high for the conversation mode of "I understand" and the action mode of <making a circle with the thumb and forefinger>, which are detected a large number of times.

In step S8, the expression mode setting unit 26 determines the degree of imitation of the expression mode that the agent 52 outputs this time, and proceeds to step S9. In subsequent step S9, the data generation unit 27 generates output data for the driver D based on the expression mode of the agent 52 to be imitated and the degree of imitation of the expression mode received from the expression mode setting unit 26. to generate

In step S<b>10 , the output unit 28 outputs control signals to the action unit, speaker or other audio output unit, display or other display unit of the agent device 5 , and outputs output data by the agent function of the agent device 5 .

In step S11, the information processing from step S4 to step S10 is repeatedly executed until the ignition switch of the vehicle is turned off.

In step S8 of FIG. 17, as shown in FIG. 18, information processing is executed to determine the degree of imitation of the expression mode output by the agent 52 this time. When the degree of imitation of the expression mode is set in step S7, in step S81 of FIG. Obtain the degree of intimacy between D and agent 52.

If the degree of familiarity with the driver D obtained in step S81 is low, the process proceeds to step S82, and the degree of imitation to be output this time is determined to be "low". In this case, as shown in FIG. 11(a), the "understand" conversation mode and the <salute> action mode with a low degree of imitation are output. On the other hand, if the degree of familiarity with the driver D acquired in step S81 is medium, the process proceeds to step S83, and the degree of imitation to be output this time is determined to be "medium". In this case, as shown in FIG. 11(b), a conversation mode of "Okay" and a motion mode <pointing thumb up> with a medium degree of imitation are output. If the intimacy level of driver D obtained in step S81 is high, the process proceeds to step S84, where the degree of imitation output this time is determined to be "high", and as shown in FIG. 11(c), the degree of imitation is high. The conversation mode of "understood" and the action mode <make a circle with the thumb and forefinger> are output.

As described above, according to the information processing system 1 and the information processing method of the present embodiment, the expression mode of the driver D (user) is detected, the expression mode of the agent 52 to be imitated is determined, and the expression mode is determined. , sets the degree of imitation of the expression mode of the agent 52 with respect to the detected expression mode of the driver D (user), and generates output data according to the degree of imitation. At this time, the degree of imitation of the representation mode of the agent 52 is set so as to approach the representation mode of the driver D (user) over time as the generated output data is output to the driver D (user). As a result, appropriate communication can be performed using expressions that are easy for the driver D (user) to understand. In addition, it is possible to prevent the driver D (user) from feeling uncomfortable with the expression of the agent 52 or feeling uncomfortable.

Further, according to the information processing system 1 and the information processing method of the present embodiment, the familiarity determination unit 24 that determines the degree of familiarity between the driver D (user) and the agent 52 is further provided, and the expression mode setting unit 26 When the degree of intimacy between the driver D (user) and the agent 52 is relatively high, compared to the case where the degree of intimacy between the driver D (user) and the agent 52 is relatively low, the degree of imitation is determined by the driver. It is set so as to approximate the expression mode of D (user). As a result, the agent 52 can appropriately change the degree of imitation of the expression mode of the driver D (user), and smooth communication can be performed. In addition, an effect that the driver D (user) can easily feel attachment to the agent 52 can be expected.

Further, according to the information processing system 1 and the information processing method of the present embodiment, the intimacy determination unit 24 determines whether the driver D (user) and the agent 52 are connected based on the content of the conversation input by the driver D (user). Estimate the intimacy of As a result, the actual behavior of the driver D (user) with respect to the agent 52 can be reflected in the intimacy level, and appropriate communication between the driver D (user) and the agent 52 can be promoted.

Further, according to the information processing system 1 and the information processing method of the present embodiment, the familiarity determination unit 24 acquires the reaction of the driver D (user) to the output expression mode of the agent 52, Based on the reaction of the person D (user), the familiarity between the driver D (user) and the agent 52 is estimated. This further promotes appropriate communication between the driver D (user) and the agent 52 .

Further, according to the information processing system 1 and the information processing method of the present embodiment, the agent device 5 is an in-vehicle device mounted on a vehicle, the user is an occupant of the vehicle, and the intimacy determination unit 24 uses the driving support device. The degree of intimacy between the driver D (user) and the agent 52 is estimated based on the driver D (user) usage status of the in-vehicle device installed in the vehicle. Thereby, the receptivity to the in-vehicle device can be reflected in the communication between the driver D (user) and the agent 52 .

Further, according to the information processing system 1 and the information processing method of the present embodiment, the expression mode of the driver D (user) is the behavior mode detected from the visual characteristics of the driver D (user) and the and a conversation mode detected from the auditory features of the (user), and the expression mode setting unit 26 uses the behavior mode of the driver D (user) and/or the conversation mode of the user to be the imitation target. Determining the mode of representation of the agent 52 . As a result, the agent 52 imitates the expression actually used by the driver D (user), so communication can be performed in a manner that is easy for the driver D (user) to understand. In addition, since the agent 52 imitates the expression actually used by the driver D (user), an effect can be expected that the expression of the agent 52 will be favorably received by the driver D (user).

Further, according to the information processing system 1 and the information processing method of the present embodiment, the visual features of the driver D (user) are obtained from the image information of the driver D (user). Appearance, Appearance, Posture, Facial Expression, and/or Action. Since the driver D (user) easily notices the imitation in the behavior mode detected from the visual characteristics, the gradual imitation by the agent 52 can prevent the impression of excessive imitation.

Further, according to the information processing system 1 and the information processing method of the present embodiment, the auditory characteristics of the driver D (user) are obtained from the voice quality of the driver D (user) obtained from the voice information of the driver D (user). , intonation, phrasing, language, dialect, content of conversation, estimated age, and/or gender. It is difficult for the driver D (user) to notice the imitation of the conversation mode detected from the auditory features compared to the action mode.

Further, according to the information processing system 1 and the information processing method of the present embodiment, the expression mode setting unit 26 uses a plurality of behavior modes of the driver D (user) and/or conversation modes of the driver D (user). to set the degree of imitation so as to approach the expressive mode of the driver D (user). As a result, the expression used by the agent 52 can be varied stepwise.

Further, according to the information processing system 1 and the information processing method of the present embodiment, the expression mode setting unit 26 is included in the operation mode of the driver D (user) and/or the conversation mode of the driver D (user). A condition including at least one of time, place, and situation is specified, and using the condition, the degree of imitation is set so as to approximate the expressive mode of the driver D (user). By using these conditions, it is possible to set the imitation in steps from imitation that is easy for the driver D (user) to notice to imitation that is difficult for the driver D (user) to perform appropriate communication using various expressions.

Further, according to the information processing system 1 and the information processing method of the present embodiment, when the degree of intimacy between the driver D (user) and the agent 52 is relatively low, the driver D (user) and the agent 52 The output frequency of outputting the representation mode of the agent 52 is set higher than when the intimacy of the agent 52 is relatively high. Thereby, communication between the driver D (user) and the agent 52 can be promoted.

It should be noted that the embodiments described above are described to facilitate understanding of the present invention, and are not described to limit the present invention. Therefore, each element disclosed in the above embodiments is meant to include all design changes and equivalents that fall within the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1... Information processing system 2... Information processing apparatus 21... Passenger identification part 22... Expression mode detection part 23... Expression mode database 24... Familiarity determination part 25... Familiarity level database 26... Expression mode setting part 27... Data generation part 28... Output unit 3 Vehicle sensors 4 Input device 5 Agent device

Claims (12)

an agent device including an anthropomorphic agent that imitates an expression mode of a user; and an information processing device that sets the expression mode of the agent and generates output data for the user. An information processing system that outputs generated output data to the user,
The information processing device is
a detection unit that detects the expression mode of the user;
a setting unit that determines an expression mode of the agent to be imitated, and sets, for the determined expression mode, a degree of imitation of the agent's expression mode with respect to the detected expression mode of the user;
a data generation unit that generates output data according to the set degree of imitation,
The information processing system, wherein the setting unit sets the degree of imitation of the expression mode of the agent so as to approach the expression mode of the user over time as the generated output data is output to the user. further comprising a determination unit that determines a degree of intimacy between the user and the agent;
When the degree of intimacy between the user and the agent is relatively high, the setting unit expresses the degree of imitation to the user more than when the degree of intimacy between the user and the agent is relatively low. 2. The information processing system according to claim 1, wherein the information processing system is set so as to approximate the mode. 3. The information processing system according to claim 2, wherein the determination unit estimates the degree of intimacy between the user and the agent based on conversation content input by the user. 4. The determination unit acquires the user's reaction to the output expression mode of the agent, and estimates a degree of intimacy between the user and the agent based on the acquired user's reaction. The information processing system according to . The agent device is an in-vehicle device mounted in a vehicle, the user is an occupant of the vehicle, and the determination unit is configured based on the user's usage of the in-vehicle device including a driving support device. 5. The information processing system according to any one of claims 2 to 4, wherein the degree of intimacy between the user and said agent is estimated. The user's expression mode includes an action mode detected from the user's visual features and a conversation mode detected from the user's auditory features,
The information processing system according to any one of claims 1 to 5, wherein the setting unit determines the expression mode of the agent to be imitated by using the user's behavior mode and/or the user's conversation mode. . 7. The information processing system according to claim 6, wherein the user's visual features include at least one of the user's appearance, appearance, posture, facial expression, and motion obtained from image information of the user. 8. The auditory features of the user according to claim 6 or 7, wherein the user's auditory features include at least one of the user's voice quality, intonation, phrase, language, dialect, conversation content, estimated age, and gender obtained from the user's voice information. Information processing system. 9. The setting unit according to any one of claims 6 to 8, wherein the setting unit sets the degree of imitation to be close to the expression mode of the user by using a plurality of behavior modes and/or conversation modes of the user. information processing system. The setting unit specifies conditions including at least one of time, place, and situation included in the user's operation mode and/or the user's conversation mode, and uses the conditions to determine the degree of imitation of the user. 10. The information processing system according to any one of claims 6 to 9, wherein the information processing system is set so as to approximate the expression mode. When the degree of familiarity between the user and the agent is relatively low, the output frequency of outputting the expression mode of the agent is made higher than when the degree of familiarity between the user and the agent is relatively high. The information processing system according to any one of claims 2-10. A processor for setting an expression mode of an anthropomorphic agent that imitates an expression mode of a user, generating output data for the user, and outputting the generated output data to the user using the agent An information processing method executed by
The processor
detecting an expression mode of the user;
determining the expression mode of the agent to be imitated, setting the degree of imitation of the agent's expression mode with respect to the detected expression mode of the user for the determined expression mode;
generating output data according to the set degree of imitation;
An information processing method for setting a degree of imitation of the expression mode of the agent so as to approach the expression mode of the user over time as the generated output data is output to the user.

Images