JP2023085752A - Information provision support method, information provision support device, information provision support program and information provision system - Google Patents

Abstract

To provide an information provision support method, information provision support device, information provision support program and information provision support system which can induce a prescribed operation that should be performed by a driver in order to obtain information provided from a vehicle or on the basis of the information provided from the vehicle.SOLUTION: An information provision support method provides information to a driver 10 by using an information provision support device 101 that supports information provision from a vehicle 100 to the driver 10. In the information provision support method, information to be provided to the driver 10 is acquired from the vehicle 100, the operation of the driver 10 is recognized, and a response operation that has been executed by the information provision support device 101 according to a prescribed operation when the driver 10 performs the prescribed operation in order to obtain the information or on the basis of the information is executed by the information provision support device 101 before the driver 10 performs the prescribed operation after the information is acquired.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information provision support method, an information provision support device, an information provision support program, and an information provision system for supporting information provision from a vehicle to a driver.

Patent Literature 1 describes a toy that recognizes a player's motion and performs a motion that reflects the player's motion.

JP 2003-080484 A

Vehicles in recent years assist the driver in providing a comfortable experience by providing various information to the driver according to the conditions inside and outside the vehicle. The content of the information provided by the vehicle to the driver is diverse, and some of them are for the purpose of ensuring safety, for example, in order to obtain the information, or for the driver to perform a predetermined action based on the information. It may be desirable to In this case, it is desirable for the vehicle to induce the predetermined action of the driver by providing information in such a manner that the driver can intuitively or naturally recognize the predetermined action to be performed.

On the other hand, even if the vehicle or the device mounted on the vehicle can recognize the motion of the driver and can perform flexible motions according to the motion of the driver, the toy described in Patent Literature 1 does not have a passive function. It is not possible to induce the predetermined action that the driver should perform only by performing a sensible action.

The present invention provides an information provision support method and information provision support device capable of inducing a predetermined action to be performed by a driver in order to obtain information provided by a vehicle or based on information provided by the vehicle. , an information provision support program, and an information provision system.

One aspect of the present invention is an information provision support method for providing information to a driver using an information provision support device that supports information provision from a vehicle to the driver. In this information provision support method, information to be provided to the driver is acquired from the vehicle. Also, the motion of the driver is recognized. Then, when the driver performs a predetermined action that should be performed in order to obtain the information or based on the information, the response action that the information provision support device has performed in response to the predetermined action is the information is obtained, and before the driver performs a predetermined action, the information provision support device executes.

ADVANTAGE OF THE INVENTION According to this invention, the information provision assistance method, information provision assistance apparatus, information provision assistance program, and information provision system which can induce a predetermined operation|movement which a driver|operator should perform can be provided.

FIG. 1 is a block diagram showing a schematic configuration of a vehicle and an information provision support device. FIG. 2 is a block diagram showing the configuration of the agent controller. FIG. 3 is a flow chart of user learning for allowing the driver to recognize the response action of the agent. FIG. 4 is an explanatory diagram showing an example of user learning. FIG. 5 is an explanatory diagram showing another example of user learning. FIG. 6 is a flow chart of action induction. FIG. 7 is an explanatory diagram showing an action inducing action and an example of action after the action inducing is successful. FIG. 8 is an explanatory diagram showing an operation example after failure of action induction. FIG. 9 is an explanatory diagram showing another operation example after failure of behavior induction. FIG. 10 is an explanatory diagram showing another action inducing action and its action.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 is a block diagram showing a schematic configuration of a vehicle 100 and an information provision support device 101. As shown in FIG. The vehicle 100 can assist the driver 10 in driving operations that should be performed by the driver 10, such as automatically following other vehicles traveling ahead. In addition, the vehicle 100 supports driving by providing information to the driver 10 using the information provision support device 101 .

As shown in FIG. 1, the vehicle 100 includes a driving operation unit 11, a car navigation system 12, an agent 13, an agent controller 14, a vehicle controller 15, sensors 16, and the like.

The driving operation unit 11 is an operation unit operated by the driver 10 to operate the vehicle 100 . The driving operation unit 11 is, for example, a steering wheel 42 (see FIG. 5, etc.), an accelerator pedal, a brake pedal, and the like.

The car navigation system 12 is a guidance device that guides the driver 10 on a route to a destination and the like. The car navigation system 12 has its own display and operation keys. The driver 10 uses the operation keys of the car navigation system 12 to perform operations necessary for route guidance, such as setting a destination. In addition, the driver 10 receives information including route guidance and the like through the display of the car navigation system 12 and/or the sound or voice output from the speaker mounted in the car navigation system 12 or the vehicle 100. be able to.

The agent 13 is a user interface that assists in providing information from the vehicle 100 to the driver 10 . In other words, the agent 13 is an information providing assistant that provides the driver 10 with information that the vehicle 100 should provide to the driver 10 together with the vehicle 100 or on behalf of the vehicle 100 .

The agent 13 is physically or virtually represented in the form of a mascot (also called a mascot character or simply a character), such as a person, animal, plant, object, or other symbol. Therefore, the agent 13 is anthropomorphic and can express emotions and emotions, which are changes in emotions, by means of its "facial expression" and the like. In addition, the agent 13 can express feelings and emotions by actions such as colors, sounds or voices, or gestures. That is, the agent 13 expresses its pseudo-feelings and emotions with facial expressions, colors, sounds or voices, or actions.

The agent 13 performs, for example, a response action or an action-inducing action by means of facial expressions or the like that express emotions and emotions. A response action is an action performed in response to an action of the driver 10 . In the present embodiment, "induction of action" means to induce the action that the driver 10 should perform by teaching the action (behavior) that the driver 10 should perform in an indirect expression form or an expression form that appeals to intuition. Say. For example, the agent 13 suggests, suggests, or implies the action that the driver 10 should take by using the behavior-inducing action. That is, the behavior-inducing action executed by the agent 13 is an action performed to induce an action of the driver 10, and is executed before the action that the driver 10 should perform. In particular, in this embodiment, the agent 13 executes the "response action" that has been executed in response to the action of the driver 10 before the action that the driver 10 should perform, so that the agent 13 provoke In other words, the response action and the behavior-inducing action are actions of the same form, but are executed at different timings. Responsive actions are performed after the associated driver 10 actions, and behavior inducing actions are performed before the associated driver 10 actions.

In this embodiment, the agent 13 is composed of a robot having a substantial form, and is placed within the driver's field of vision during driving, such as near the driver's seat. However, when the vehicle 100 employs a head-up display, the agent 13 can be virtually configured by the display of the head-up display. Further, when a device such as a smartphone possessed by the driver 10 cooperates with the vehicle 100, the agent 13 can be virtually displayed by the device such as the smartphone.

The agent controller 14 is a control unit that controls operations of the agent 13 and the like. More specifically, the agent controller 14 is a computer programmed with the expression, color, sound or voice of the agent 13 according to the information provided by the vehicle 100 to the driver 10, the action of the driver 10, or the action. is. The agent 13 and the agent controller 14 constitute an information provision support device 101 that supports the provision of information from the vehicle 100 to the driver 10 . Further, the information provision support device 101 constitutes an information provision system 102 together with the vehicle 100 as a whole or components of the vehicle 100 related to information provision. A specific configuration of the agent controller 14 will be described in detail later.

In this embodiment, the agent controller 14 is configured separately from the vehicle controller 15 , but some or all of the components that make up the agent controller 14 can be configured integrally with the vehicle controller 15 . . Further, in this embodiment, the agent 13 is formed as one of the devices mounted on the vehicle 100 , so the agent controller 14 for controlling this is also provided in the vehicle 100 . However, when the agent 13 is a cooperative device independent of the vehicle 100 , such as a smart phone, the device can constitute the agent controller 14 .

The vehicle controller 15 is a control unit that controls the operation of the vehicle 100 in an integrated manner. The vehicle controller 15 controls an electric motor and/or an engine that drives the vehicle 100, a brake, and the like, based on an input from the driving operation unit 11. FIG. The vehicle controller 15 is configured by, for example, one or a plurality of computers in which the operation of each unit that configures the vehicle 100 is programmed. In order to control the vehicle 100, the vehicle controller 15 may refer to the signal output by the sensor 16 and the like. The vehicle controller 15 acquires operations on the vehicle 100 and operations on various devices mounted on the vehicle 100 (hereinafter referred to as device operation information 33; see FIG. 2). In addition, the vehicle controller 15 obtains information to be provided to the driver 10 from the vehicle 100 by calculation using signals obtained from the sensor 16 and the like.

The information provided by the vehicle 100 to the driver 10 includes information that calls attention to objects to be noticed such as traffic signs and passers-by, information that provides route guidance, information that the vehicle 100 or the vehicle 100 carries. There are various types of information, such as information requesting operation of equipment. In any case, the information provided by the vehicle 100 to the driver 10 is provided according to situations inside and/or outside the vehicle. Therefore, information provided by the vehicle 100 to the driver 10 is called situation information 32 (see FIG. 2).

In the present embodiment, the situation information 32 provided by the vehicle 100 to the driver 10 includes information that requires the action of the driver 10 and other information (that is, information that does not require the action of the driver 10). are classified into two types. Whether or not the various situation information 32 is information that requires an action by the driver 10 is determined in advance according to the type, nature, and the like.

If there is a predetermined action that the driver 10 should perform in order to know the information provided by the vehicle 100 or based on the information provided by the vehicle 100, the information is information that requires the action of the driver 10. is. Further, an action of the driver 10 that should be performed in order to know the information provided by the vehicle 100 or based on the information provided by the vehicle 100 is hereinafter simply referred to as a predetermined action. The predetermined actions to be performed by the driver 10 include movement of the limbs of the driver 10 and simply movement of the line of sight of the driver 10 .

The information that requires the action of the driver 10 includes, for example, a call for attention to an object of interest outside or inside the vehicle (hereinafter simply referred to as an object of interest 41; see FIG. 4, etc.) and an instruction to grasp the steering wheel 42. Alternatively, it is a request and a suggestion that various devices mounted on the vehicle 100 such as the car navigation system 12 should be operated. Movement of the line of sight to the object of interest 41, gripping the steering wheel 42, operating various devices, and the like are predetermined actions that the driver 10 should perform.

The sensor 16 is one or a plurality of sensors that measure the condition of each part constituting the vehicle 100 and the condition inside or outside the vehicle 100 . The sensor 16 includes, for example, an acceleration sensor, an in-vehicle camera 17 for photographing the interior of the vehicle, a camera for photographing the exterior of the vehicle, a microphone for collecting sounds inside or outside the vehicle, a GNN (Global Navigation Satellite System) sensor, and a LiDAR (Light Detection and Ranging) sensor. In this embodiment, the sensor 16 includes at least the in-vehicle camera 17 .

The in-vehicle camera 17 continuously or intermittently captures the interior of the vehicle 100 , especially the range including the driver 10 . An image or video captured by the in-vehicle camera 17 (hereinafter referred to as an in-vehicle image 31 , see FIG. 2) is used by the agent controller 14 to recognize the actions of the driver 10 .

FIG. 2 is a block diagram showing the configuration of the agent controller 14. As shown in FIG. As shown in FIG. 2, an information acquisition unit 21, a driver motion recognition unit 22, a behavior induction determination unit 23, an agent motion control unit 24, and a motion pattern storage unit 25 are provided. A program that causes the agent controller 14 to function as these units constitutes an information provision support program that causes the information provision support device 101 and the information provision system 102 to operate. The information provision support program is stored in advance in the information provision support device 101 or the information provision system 102 . Further, the information provision support program may be provided, for example, in the form of the information provision support device 101, the information provision system 102, or other computer-readable recording media. Furthermore, after being installed in the information providing support device 101 or the information providing system 102 using a recording medium or by so-called OTA (Over The Air) technology, the information providing support program is partly or wholly updated. obtain.

The information acquisition unit 21 acquires an in-vehicle image 31 , situation information 32 , and device operation information 33 . The information acquisition unit 21 acquires the in-vehicle image 31 directly from the in-vehicle camera 17 or indirectly via the vehicle controller 15 . The information acquisition unit 21 also acquires situation information 32 and device operation information 33 from the vehicle controller 15 . Note that the information acquisition unit 21 can acquire a signal or the like output from the sensor 16 and generate the situation information 32 by itself instead of the vehicle controller 15 .

The driver motion recognition unit 22 recognizes the motion of the driver 10 . Further, the driver's motion recognition unit 22 functions as a driver's motion monitoring unit that monitors the motion of the driver 10 by continuously or intermittently recognizing the motion of the driver 10 . In the present embodiment, the driver's motion recognition unit 22 recognizes the motion of the driver 10 using the in-vehicle image 31 and/or based on the operation status of equipment mounted on the vehicle 100 . Specifically, the driver's motion recognition unit 22 includes a motion detection unit 34 , a line-of-sight detection unit 35 , and a device operation detection unit 36 .

The motion detection unit 34 detects global motions of the driver's 10 such as hands and face. For example, the motion detection unit 34 recognizes the driver 10 from the in-vehicle image 31 and detects the physical motion of the driver 10 based on temporal changes in the shape thereof.

The line-of-sight detection unit 35 detects the line of sight of the driver 10 . For example, the line-of-sight detection unit 35 detects the driver's 10 line of sight and its transition (hereinafter referred to as line-of-sight movement) by recognizing the driver's 10 eyeball (pupil) and its movement from the in-vehicle image 31 .

Based on the device operation information 33, the device operation detection unit 36 detects an operation of the vehicle 100 or an operation of various devices mounted on the vehicle 100 (hereinafter referred to as device operation operation). For example, when the car navigation system 12 is operated, the device operation information 33 is used to detect the operation of the device.

Hereinafter, the physical movement, line-of-sight movement, and device operation movement of the driver 10 are collectively referred to simply as the movement of the driver 10 unless otherwise specified. Of the physical motions, line-of-sight motions, and/or device operation motions of the driver 10, those performed in order to obtain the situation information 32 or based on the situation information 32 are predetermined motions, as described above. is.

Based on the situation information 32, the action inducement determination unit 23 determines whether or not it is necessary to induce a predetermined action of the driver 10 when the situation information 32 is provided to the driver 10. FIG. The action inducement determination unit 23 determines in advance whether or not it is necessary to induce a predetermined action of the driver 10 according to the type and nature of the situation information 32 using, for example, a data table. Therefore, the action inducement determination unit 23 determines whether or not it is necessary to induce a predetermined action of the driver 10 according to the type and nature of the individual specific situation information 32 .

Further, the action inducement determination unit 23 sets parameters for defining a specific action mode of the agent 13 that induces the predetermined action of the driver 10 when it is necessary to induce the predetermined action of the driver 10 . For this purpose, the behavior induction determination unit 23 includes an importance determination unit 37 and a learning situation determination unit 38 .

The importance determination unit 37 determines the importance of the situation information 32 . The importance of the situation information 32 is set in advance based on the urgency of the situation information 32 and/or the degree of danger when the predetermined action based on the situation information 32 is not performed. That is, the importance of the situation information 32 is set in advance according to the type and nature of the situation information 32 and the like. In particular, when the situation information 32 is information related to the driving operation of the vehicle 100, it is determined that its importance is high. On the other hand, when the situation information 32 is information other than information related to the driving operation of the vehicle 100, it is determined that its importance is low. To give a more specific example, when the situation information 32 includes traffic signs, passers-by, and the like as attention objects 41, and has content that calls attention necessary for ensuring safety, the importance of the situation information 32 is " high”. On the other hand, when the situation information 32 specifically designates the device or the like mounted on the vehicle 100 as the object of attention 41 and has the contents of a warning for prompting the operation thereof, the importance of the situation information 32 is “low”. be judged. Also, for example, if the situation information 32 is information that requires the driver to grip the steering wheel 42, the importance of the situation information 32 is determined to be "high". The importance of the situation information 32 determined by the importance determination unit 37 is one of the parameters that determine the action mode of the agent 13 .

The learning state determination unit 38 determines the learning (cognition) state of the driver 10 with respect to the action of the agent 13 .

Specifically, the learning status determination unit 38 acquires the motion of the driver 10 from the driver motion recognition unit 22 . Further, when the driver 10 performs a predetermined action in accordance with the situation information 32, the agent 13 performs a response action in accordance with the situation information 32. receives a notification that has performed a response operation. Based on these pieces of information, the learning status determination unit 38 determines whether or not the driver 10 has recognized the response action performed by the agent 13 . For example, when the driver 10 visually recognizes the agent 13 after the agent 13 executes a response action corresponding to a predetermined action of the driver 10, the learning status determination unit 38 determines whether the agent 13 has responded to the predetermined action. It is determined that the driver 10 has recognized the response action.

In addition, the learning status determination unit 38 counts the number of times the driver 10 recognizes the response motion of the agent 13 corresponding to the predetermined motion (hereinafter referred to as the number of times of response motion recognition). Then, when the number of response motion recognition times is equal to or greater than a predetermined threshold value, the learning status determination unit 38 determines whether the driver 10 has learned (hereinafter referred to as user learning) about the response motion of the agent 13 corresponding to the predetermined motion. Determined as complete. The determination result as to whether or not user learning has been completed is one of the parameters that determine the behavior of the agent 13 .

Note that in the present embodiment, the learning status determination unit 38 counts the number of times the response action is recognized as the number of times the response action is recognized, but the present invention is not limited to this. The learning status determination unit 38 can count the number of trials of performing the response motion instead of the number of times of recognition of the response motion. In this case, the learning status determination unit 38 determines that the user learning has been completed when the number of times the response action has been tried is greater than or equal to a predetermined threshold.

The agent operation control unit 24 controls operations of the agent 13 . The agent action control unit 24 causes the agent 13 to perform a response action, an action inducing action, and other actions based on the situation information 32 and the like. Therefore, the agent action control section 24 includes at least a response action control section 39 and an action inducing action control section 40 .

The response operation control unit 39 controls the response operation of the agent 13 . Based on the situation information 32 and the motion of the driver 10 acquired from the driver motion recognition unit 22, the response motion control unit 39 performs a predetermined motion according to the situation information 32 by the driver 10. The agent 13 is made to perform a response operation corresponding to this. Response actions corresponding to predetermined actions and other action modes of the agent 13 are predetermined according to the type and nature of the situation information 32 and stored in the action pattern storage unit 25 . Therefore, when the driver 10 performs a predetermined action, the response action control section 39 refers to the action pattern storage section 25 to cause the agent 13 to execute a response action corresponding to the predetermined action in real time. can be done.

The behavior inducing motion control unit 40 controls the behavior inducing motion of the agent 13 when the behavior inducing determination unit 23 determines that it is necessary to induce the predetermined motion of the driver 10 . Also, the action inducing operation control unit 40 causes the agent 13 to execute an action inducing operation corresponding to the situation information 32 based on the situation information 32 and the determination result of the learning situation determination unit 38 . Specifically, when it is determined by the determination result of the learning status determination unit 38 that the user learning has been completed, the behavior inducing motion control unit 40 controls whether the driver 10 performs a predetermined motion corresponding to the situation information 32. Without waiting for , the action pattern storage unit 25 is referred to and the agent 13 is made to execute a response action corresponding to the situation information 32 as an action inducing action. Therefore, substantially before the driver 10 performs a predetermined action according to the situation information 32, the agent 13 executes the action inducing action.

When the agent 13 executes the action-inducing action, the driver 10 who has completed the user learning about the action-inducing action can intuitively recognize that the predetermined action should be performed. Further, when the agent 13 executes an action inducing action, the driver 10 who has completed the user learning can almost reflexively perform the predetermined action in response.

It should be noted that when the action inducing action control unit 40 causes the agent 13 to perform the action inducing action, the response action control unit 39 does not cause the agent 13 to perform the response action. In other words, the agent 13 alternatively executes the behavior-inducing action or the response action in providing the situation information 32 once. When the driver 10 has little experience in using the vehicle 100, the agent 13 generally performs a response action, and when the driver 10 has a certain amount of experience in using the vehicle 100, the agent 13 generally performs an action-inducing action.

Further, the action inducing motion control unit 40 determines whether or not the driver 10 has performed the predetermined motion after causing the agent 13 to execute the action inducing motion as described above. In other words, the action inducement operation control unit 40 functions as an action inducement success/failure determination unit that determines the success or failure of the action inducement by the agent 13 . The action inducing action control unit 40 determines whether the action inducing action is successful or not by monitoring the action of the driver 10 at least after the agent 13 executes the action inducing action. For example, the action-inducing motion control unit 40 measures the elapsed time (hereinafter simply referred to as elapsed time) after the agent 13 executes the action-inducing motion. Then, when the driver 10 performs a predetermined action before the elapsed time elapses a predetermined time (threshold value), the action inducing action control unit 40 determines that the action inducing has succeeded. On the other hand, when the elapsed time exceeds the predetermined time, the behavior induction operation control unit 40 determines that the behavior induction has failed.

After that, the action inducing operation control unit 40 causes the agent 13 to perform an additional action depending on whether the action inducing is successful or not. In the present embodiment, the additional action after the action-inducing action is the action expressing the feeling or emotion of the agent 13 (hereinafter referred to as emotion) or the emphasis of the action-inducing action.

The emotions that the action-inducing motion control unit 40 causes the agent 13 to additionally perform include, for example, a first emotion and a second emotion. The first emotion is an action in which the agent 13 expresses a positive feeling or emotion (hereinafter referred to as positive emotion) by means of facial expressions or the like of the agent 13 . For example, it is generally assumed that the agent 13 generally expresses joy, such as making a facial expression or gesture expressing joy, emitting a relatively high-pitched sound or voice, or exhibiting a warm color. The emotion that can be felt is the primary emotion. The second emotion is an action in which the agent 13 expresses a negative feeling or emotion (hereinafter referred to as negative emotion) by means of facial expressions or the like of the agent 13 . For example, it is generally assumed that the agent 13 generally expresses sadness, such as making a facial expression or gesture expressing sadness, emitting a relatively low tone or voice, or exhibiting a cool color. The emotion that can be felt is the secondary emotion.

After causing the agent 13 to perform the action inducing action, the action inducing action control unit 40 causes the agent 13 to perform a positive first emotion when the driver 10 performs a predetermined action. That is, when the action induction is successful, the agent 13 executes the first emotion. By perceiving the positive first emotion of the agent 13, the driver 10 recognizes that the action inducing action of the agent 13 suggests or teaches the predetermined action that the driver 10 should perform. , gratitude and other positive emotions. For this reason, the next and subsequent action inducements are more likely to succeed.

On the other hand, when the driver 10 does not perform the predetermined action after causing the agent 13 to perform the action inducing action, the action inducing action control unit 40 causes the agent 13 to perform an additional action according to the importance of the situation information 32. Change behavior. That is, when the action induction fails, the agent 13 performs an additional action according to the importance of the situation information 32. FIG.

Specifically, when the action induction fails and the importance of the situation information 32 is low, the action induction operation control unit 40 causes the agent 13 to perform a negative second emotion. By feeling the negative second emotion of the agent 13, the driver 10 can perceive that the driver 10 should perform the action. As a result, the driver 10 naturally comes to pay attention to the action of the agent 13, so that the next and subsequent actions are likely to succeed.

On the other hand, when the action inducement fails and the importance of the situation information 32 is high, the action inducement operation control unit 40 causes the agent 13 to execute the emphasized action inducement operation. Emphasizing the action-inducing motion means changing and/or adding the facial expression, color, sound or voice, or action of the agent 13 related to the action-inducing motion to make it a relatively exaggerated form of expression. For example, the behavior-inducing action is emphasized by increasing the volume of the sound or voice produced by the agent 13, or by expressing the change in the facial expression or gesture of the agent 13 in a large manner. Since it becomes easier for the driver 10 to perceive the behavior-inducing motion of the agent 13, the behavior-inducing motion should be particularly emphasized in the form of adding the types of the five senses (so-called modalities) that the driver 10 perceives in the behavior-inducing motion. is preferred. For example, an action-inducing action represented only by visual information such as a change in facial expression is emphasized in a form that appeals not only to the sense of sight but also to the sense of hearing, for example, by issuing a sound or voice.

The action pattern storage unit 25 stores action patterns of the agent 13 corresponding to the types and properties of the situation information 32 . The motion pattern storage unit 25 stores, for example, corresponding response motions and behavior inducing motions for each situation information 32 . Further, the motion pattern storage unit 25 stores in advance, for each situation information 32, the first emotion, the second emotion, and the emphasized mode of the behavior-inducing motion.

The operation of the vehicle 100 configured as described above, that is, the operation of the information providing support device 101 and the information providing system 102 regarding user learning and behavior induction will be described below.

<User learning>
FIG. 3 is a flow chart of user learning for making the driver 10 recognize the response action of the agent 13 . As shown in FIG. 3, in step S101, the driver's motion recognition unit 22 recognizes the motion of the driver 10. As shown in FIG. In step S102, the response motion control unit 39 refers to the motion pattern storage unit 25 and determines a response motion corresponding to the recognized motion of the driver 10. FIG. Further, in step S103, the response operation control unit 39 causes the agent 13 to perform the response operation.

After that, in step S<b>104 , the learning status determination unit 38 determines whether or not the driver 10 has recognized the response action of the agent 13 . When it is determined that the driver 10 has recognized the response motion of the agent 13, the learning status determination unit 38 increments the response motion recognition count in step S105.

FIG. 4 is an explanatory diagram showing an example of user learning. As shown in FIG. 4, the scene a1 at the time t1 is a scene in which the target 41 occurs, the driver 10 notices it, and shifts the line of sight. When the driver 10 shifts the line of sight to the attention target 41 in this way, in the stage of user learning, the agent 13 follows the line of sight transition of the driver 10 as shown in scene a2 at time t2. Turn to the direction of the target 41 and perform a response action of looking at the target 41 . Then, as in scene a3 at time t3, when the driver 10 recognizes the response action of the agent 13 to "look at the object of interest 41", it is assumed that the driver 10 has learned this response action. Incremented. Then, the user learning related to the response motion is repeated until it is determined that the number of times of recognition of the response motion reaches or exceeds the threshold value and the user learning is completed.

FIG. 5 is an explanatory diagram showing another example of user learning. As shown in FIG. 5, a scene b1 at time t1 is a scene in which the driver 10 takes his or her hand off the steering wheel 42 . In scene b1, agent 13 expresses a sleeping expression. Then, in scene b2 at time t2, when the driver 10 grips the steering wheel 42, the agent 13, for example, widens its eyes in response to this and performs a response action expressing a facial expression indicating activation or awakening. After that, in scene b3 at time t3, when the driver 10 recognizes the response motion of "holding the steering wheel 42, the agent 13 wakes up." is incremented. Then, the user learning related to the response motion is repeated until it is determined that the number of times of recognition of the response motion reaches or exceeds the threshold value and the user learning is completed.

In the following, user learning has been completed for the response action of the agent 13 looking at the object of interest 41 when it occurs and the response action of the agent 13 waking up when the steering wheel 42 is gripped. shall be

<Action inducement>
FIG. 6 is a flow chart of action induction. As shown in FIG. 6 , in step S<b>201 , the information acquisition unit 21 acquires situation information 32 that the vehicle 100 should provide to the driver 10 . Further, in step S202, the action inducement determination unit 23 determines whether or not the action inducement for inducing a predetermined action of the driver 10 is necessary based on the specific type, nature, etc. of the acquired situation information 32. judge.

If it is determined that behavior induction is necessary, the learning situation determination unit 38 determines whether or not user learning of the response action corresponding to the situation information 32 has been completed in step S203. When user learning has not been completed, the response actions described above are performed. On the other hand, when the user learning is completed, the action inducing motion control unit 40 refers to the action pattern storage unit 25 and determines the action inducing motion corresponding to the situation information 32 in step S204. Then, in step S205, the action inducing action control unit 40 causes the agent 13 to execute the determined action inducing action.

After that, in step S206, the action inducing action control unit 40 determines whether or not the action inducing action of the agent 13 has induced the predetermined action of the driver 10 or not. Then, when it is determined that the predetermined action of the driver 10 has been induced by the action inducing action of the agent 13, the action inducing action control unit 40 causes the agent 13 to execute a positive first emotion in step S107.

On the other hand, when it is determined that the predetermined action of the driver 10 could not be induced by the action inducing action of the agent 13, the action inducing action control unit 40 refers to the importance of the situation information 32 in step S208. When the importance of the situation information 32 is high, in step S209, the action inducing motion control unit 40 causes the agent 13 to execute the emphasized action inducing motion. On the other hand, when the importance of the situation information 32 is low, the action inducing motion control unit 40 causes the agent 13 to perform a negative second emotion in step S210.

FIG. 7 is an explanatory diagram showing an action inducing action and an example of action after the action inducing is successful. As shown in FIG. 7, a scene A1 at time T1 is a scene at the moment when an object of interest 41 for which the vehicle 100 should provide information to the driver 10 occurs. In scene A1, the driver 10 has not yet noticed this and is thinking of something else. Scene A2 at time T2 is a scene at substantially the same time as time T1. In scene A2, before the driver 10 notices the target 41, the agent 13 performs a behavior-inducing action of looking at the target 41. FIG. Since the agent 13 is within the field of view of the driver 10, the driver 10 perceives the behavior-inducing motion of the agent 13 almost unwillingly. In addition, since the user learning related to this action-inducing action has been completed, the driver 10 intuitively and substantially reflects that the action-inducing action of the agent 13 suggests a predetermined action that the driver 10 should perform. to understand Therefore, in scene A3 at time T3 immediately thereafter, the driver 10 reflexively shifts his/her line of sight to the attention target 41 as if being lured by the action-inducing action of the agent 13 under the influence of the action-inducing action of the agent 13 . .

Thus, when the driver 10 performs the predetermined action of "looking at the object of interest 41" by the action inducing action of the agent 13, the agent 13 shows a first emotion such as a smile as shown in scene A4 shown at time T4. to run.

FIG. 8 is an explanatory diagram showing an operation example after failure of action induction. As shown in FIG. 8, the scene A1 at time T1 is the scene at the instant when the object of interest 41 occurs, as in the case of FIG. In the scene A2' at the time T2, which is a scene substantially at the same time as the time T1, the agent 13 performs an action-inducing action of looking at the target 41 of interest. However, in the scene A2', the driver 10 is not aware of the action inducing action of the agent 13 and does not perform the predetermined action.

Here, for example, when the importance of the situation information 32 is high, for example, when the attention target 41 must be visually recognized and a flexible driving operation must be performed as soon as possible in order to ensure safety, etc., in scene A3' at time T3, the agent 13 emphasizes action-induced actions. Here, the agent 13 changes its facial expression by changing a part of the face and the color of the eyes to emphasize the action-inducing action, while maintaining the essential mode of the action-inducing action of "looking at the object of interest 41". there is As a result, the driver 10 notices the action-inducing action emphasized by the agent 13 and intuitively and almost reflexively understands the predetermined action suggested by the action-inducing action. Therefore, the driver 10 reflexively shifts the line of sight to the target 41 as if being influenced by the agent 13's action-inducing action and lured by the agent 13's action-inducing action.

FIG. 9 is an explanatory diagram showing another operation example after failure of behavior induction. As shown in FIG. 9, the scene A1 at time T1 is the scene at the instant when the object of interest 41 occurs, as in the case of FIGS. In scene A2' at time T2, as in the case of FIG. 8, the agent 13 performs an action-inducing action of looking at the object of interest 41, but the driver 10 does not notice the action-inducing action of the agent 13. , the specified action is not performed.

Here, it is assumed that the importance of the situation information 32 is low, for example, the object of interest 41 is an in-vehicle device such as the car navigation system 12 and the situation information 32 recommends operating the in-vehicle device. In this case, in scene A3″ at time T3, the agent 13 performs a second emotion such as showing a crying face or showing a sad expression. , it can be inferred that some kind of predetermined action was requested, and after that, it becomes easier for the driver to notice the target of interest 41. Further, when the driver 10 recognizes the second emotion of the agent 13, the potential for the anthropomorphic agent 13 is detected. guilt, etc., after that, it becomes easier to notice at least the action of the agent 13. As a result, it becomes easier to succeed in inducing behavior from the next time onward.

FIG. 10 is an explanatory diagram showing another action inducing action and its action. In case 1 and case 2 of FIG. 10 , it is assumed that the vehicle 100 automatically follows the vehicle ahead and the driver 10 releases the steering wheel 42 .

Case 1 in FIG. 10 is a successful example of behavior induction. First, in scene B1 at time T1, it is assumed that the follow-up running function for the preceding vehicle is about to run out, and the vehicle 100 needs to provide information to the driver 10 that the driver 10 should grip the steering wheel 42. . At this time, the driver 10 has completed the user learning that "the agent 13 wakes up when the driver 10 grips the steering wheel 42." Execute the provoked action. Then, when the driver 10 recognizes the agent 13 with a suddenly aroused expression within the field of vision, the driver 10 intuitively and almost reflexively understands that the driver 10 should grip the steering wheel 42 . Therefore, in scene B3 at time T3, the driver 10 grips the steering wheel 42 almost reflexively. When the driver 10 performs the predetermined action of gripping the steering wheel 42 in this manner, the agent 13 performs a first emotion such as smiling, as in scene B4 at time T4.

On the other hand, case 2 in FIG. 10 is an operation example when action induction fails. First, in scene B1′ at time T2, similar to scene B1, the vehicle 100 provides information to the driver 10 that the follow-up function for the preceding vehicle is about to run out and that the driver 10 should grip the steering wheel 42. This is the scene where it was necessary to Then, in scene B2' at time T2, the agent 13 performs an action-inducing action showing an aroused expression. However, here, it is assumed that the driver 10 did not notice the action inducing action of the agent 13 even though the driver 10 grips the steering wheel 42, which is a highly important situation related to the driving operation. In this case, in the subsequent scene B3' at time T3, the agent 13 emphasizes the action-inducing action by, for example, changing the complexion while maintaining the essential aspect of the action-inducing action, which is an aroused facial expression. As a result, the driver 10 notices the behavior-inducing action of the agent 13, and can perform the predetermined action of gripping the steering wheel 42 in scene B4' at time T4.

As described above, the information provision support method according to the present embodiment is an information provision support method for providing information to the driver 10 using the information provision support device 101 that supports information provision from the vehicle 100 to the driver 10. be. In this information provision support method, information to be provided to the driver 10 is acquired from the vehicle 100 as the situation information 32 . Also, the motion of the driver 10 is recognized. Then, in order to know the situation information 32, or when the driver 10 performs a predetermined action that should be performed based on the situation information 32, information provided by the information provision support device 101 in accordance with the predetermined action. A certain response action is executed by the information provision support device 101 as an action inducing action after the situation information 32 is acquired and before the driver 10 performs a predetermined action.

In this way, when the response action is executed as the action inducing action before the driver 10 performs the predetermined action, the driver 10 intuitively and almost reflexively performs the predetermined action associated with the situation information 32. You can feel it. Therefore, according to the information provision support method according to the present embodiment, the predetermined action of the driver 10 is induced. That is, according to the information provision support method of the present embodiment, the information provision support device 101 can encourage the driver 10 to naturally want to perform the predetermined action. As a result, according to the information provision support method of the present embodiment, the information provision support device 101 can quite naturally induce the driver 10 to take necessary actions in accordance with individual specific situations.

For example, when the driver 10 receives a large amount of information provided by the vehicle 100, the direct provision of information may feel cumbersome, and the response of the driver 10 to the provision of important information may be delayed. However, according to the provision of information by means of behavior induction as described above, it is possible to elicit the predetermined action that the driver 10 should perform more quickly and reliably than in the case of direct provision of information.

In the information provision support method according to the present embodiment, it is assumed that the information provision support device 101 performs a response action when the driver 10 performs a predetermined action. Further, it is determined whether or not the driver 10 has recognized the response action executed by the information provision support device 101, and the number of times the driver 10 has recognized the response action executed by the information provision support device 101 is counted. be. Then, when the number of response motion recognition times is equal to or greater than the threshold, the response motion is executed by the information provision support device 101 as an action inducing motion before the driver 10 performs the predetermined motion.

In this manner, when the learning (user learning) of the driver 10 who is the user about the response motion is completed, the response motion is executed as the action-inducing motion, so that the predetermined motion of the driver 10 is particularly reliably performed. induced. This is because the user learning has been completed, so that the driver 10 can intuitively perceive the predetermined action suggested by the action-inducing action without being confused by the action-inducing action of the information provision support device 101. Because you can do this reflexively.

In the information provision support method according to the present embodiment, when the driver 10 performs the predetermined action after the information provision support apparatus 101 executes the response action as the behavior inducing action, the information provision support apparatus 101 responds to the positive emotion Executes the first emotion, which is an action that expresses In this way, when the positive first emotion is executed after the action inducement is successful, the information providing support device 101 recognizes that the action inducement action suggests a predetermined action to be performed by the driver 10, and the affirmative action is given to the information provision support device 101. It becomes easier to hold emotional feelings. As a result, the next and subsequent action inducements are more likely to succeed.

In the above-described embodiment, the information provision support device 101 particularly executes the response action and the action inducing action by the agent 13, which is the mascot, and the first emotion is expressed by facial expression, color, sound or voice, action, or the like. be done. For this reason, the first emotion tends to make the agent 13 feel positive, and the success rate of action induction is particularly likely to improve.

In the information providing method according to this embodiment, the importance of the situation information 32 to be provided to the driver 10 is determined. When the driver 10 does not perform the predetermined action after the information providing support device 101 executes the response action as the action inducing action, the operation of the information providing support device 101 is changed according to the importance. As a result, when the action inducement fails, it is possible to perform an appropriate action according to the importance of the situation information 32, such as making the action inducement successful or increasing the success rate of the next action inducement.

In the above-described embodiment, the information provision support device 101 executes the response action emphasized as the behavior inducing action, particularly when the importance of the situation information 32 is determined to be high. Therefore, when the situation information 32 is highly important, such as when the situation information 32 is related to the driving operation, the information provision support device 101 can actively work on the driver 10 to successfully induce the predetermined action.

Further, in the above-described embodiment, the information provision support device 101 performs the second emotion, which is the action expressing the splenectomy emotion, particularly when the importance of the situation information 32 is determined to be low. By sensing the negative second emotion by the information provision support device 101, the driver 10 naturally pays attention to the action of the agent 13, so that the next and subsequent action inducements are more likely to succeed. In addition, when the importance of the situation information 32 is low, the information provision support device 101 modestly encourages the driver 10 by executing the second emotion, so that the troublesomeness of providing information is reduced.

Furthermore, in the above-described embodiment, the information provision support device 101 particularly executes the response action and the action inducing action by the agent 13, which is the mascot, and the second emotion is the facial expression, color, sound or voice, action, or the like. is represented by For this reason, by recognizing the second emotion, the driver 10 becomes more concerned about the action of the agent 13, so that the success rate of the next action inducement is especially likely to be improved.

In the above-described embodiment, when the situation information 32 is information related to the driving operation of the vehicle 100, the importance is determined to be high, and when the situation information 32 is other information, the importance is determined to be low. In this way, when the situation information 32 related to the driving operation is determined to be highly important, it is possible to more reliably induce the predetermined action of the driver 10 in relation to the provision of the situation information 32 related to the driving operation. As a result, it is particularly easy to ensure safety.

Although the embodiments of the present invention have been described above, the configurations described in the above embodiments merely show a part of application examples of the present invention, and are not intended to limit the technical scope of the present invention.

For example, the information provided by the vehicle 100 to the driver 10 is not limited to the example of the above embodiment. In addition, the actions that the driver 10 should perform in order to know the information provided by the vehicle 100 or based on the information provided by the vehicle 100 are not limited to the example of the above embodiment. Therefore, the present invention can be applied to combinations of various information provided by the vehicle 100 to the driver 10 and various actions to be performed by the driver 10 .

In addition, the in-vehicle camera 17 is provided in the vehicle 100 as one of the sensors 16, but the present invention is not limited to this. The information provision support device 101 has its own camera, and the camera may capture an image of the inside of the vehicle 100 to the extent that the motion of the driver 10 can be recognized. In this case, the information provision support device 101 can acquire the in-vehicle image 31 using a camera uniquely provided in the information provision support device 101 instead of the in-vehicle camera 17 provided in the vehicle 100 .

The situation information 32 of the above-described embodiment and the corresponding response action and action inducing action are merely examples. Therefore, the information provision support method and the like of the present invention can be applied to various scenes other than the scenes described in the above embodiments.

For example, when the driver 10 is operating the air conditioner and the agent 13 responds by looking at the air conditioner, the driver 10 or the passenger in the front passenger seat or the rear seat looks cold or hot. When this happens, the agent 13 can perform a behavior-inducing action of looking at the air conditioner. When the driver 10 operates the car navigation system 12 and the agent 13 looks at the car navigation system 12 in response, the agent 13 asks the driver to look at the car navigation system 12 or operates the car navigation system 12. An action-inducing action to look at the car navigation system 12 can be performed when desired. When the door of the vehicle 100 is locked, the agent 13 can respond by barking, for example, when the door of the vehicle 100 is to be locked. In addition, regardless of whether the vehicle 100 is automatically following the vehicle 100 or not, the agent 13 can react such as waking up in a scene where the steering wheel 42 should be gripped.

The information provision support device 101 of the above embodiment can be implemented by a so-called AI (Artificial Intelligence) program. AI programs are sometimes referred to as trained models. Further, although the information provision support device 101 of the above embodiment is clearly distinguished from the vehicle 100 , the information provision support device 101 can be mounted integrally with the vehicle 100 as part of the vehicle 100 . That is, some or all of the functions of agent controller 14 can be implemented by vehicle controller 15 .

10: driver, 11: driving operation unit, 12: car navigation system, 13: agent, 14: agent controller, 15: vehicle controller, 16: sensor, 17: in-vehicle camera, 21: information acquisition unit, 22: driver Action recognition unit 23: Action induction determination unit 24: Agent action control unit 25: Action pattern storage unit 31: In-vehicle image 32: Situation information 33: Device operation information 34: Action detection unit 35: Line of sight detector, 36: device operation detector, 37: importance determination unit, 38: learning situation determination unit, 39: response operation control unit, 40: behavior inducing operation control unit, 41: attention target, 42: steering wheel, 100 : vehicle, 101: information provision support device, 102: information provision system

Claims (12)

An information provision support method for providing information to a driver using an information provision support device for supporting information provision from a vehicle to the driver,
obtaining from the vehicle information to be provided to the driver;
recognizing the movement of the driver;
In order to know the information, or when the driver performs a predetermined action to be performed based on the information, a response action that the information provision support device has performed in response to the predetermined action is performed. , performed by the information provision support device after the information is acquired and before the driver performs the predetermined action;
Information provision support method. The information provision support method according to claim 1,
when the driver performs the predetermined action, the information provision support device executes the response action;
determining whether the driver has recognized the response action executed by the information provision support device;
counting the number of times the driver recognizes the response action executed by the information provision support device;
When the number of times is equal to or greater than a predetermined threshold, the response action is performed by the information provision support device before the driver performs the predetermined action.
Information provision support method. The information provision support method according to claim 1 or 2,
After execution of the response action by the information provision support device, when the driver performs the predetermined action, the information provision support device executes a first emotion that is a motion expressing a positive emotion;
Information provision support method. The information provision support method according to claim 3,
The information provision support device executes the response operation by means of a mascot,
The first emotion is expressed by the facial expression, color, sound or voice, or action of the mascot,
Information provision support method. The information provision support method according to any one of claims 1 to 3,
determining the importance of the information;
changing the operation of the information provision support device according to the degree of importance when the driver does not perform the predetermined action after execution of the response action by the information provision support device;
Information provision support method. The information provision support method according to claim 5,
When the importance is determined to be high, the information provision support device executes the emphasized response action;
Information provision support method. The information provision support method according to claim 5,
When the importance is determined to be low, the information provision support device executes a second emotion, which is an action expressing a negative emotion.
Information provision support method. The information provision support method according to claim 7,
The information provision support device executes the response operation by means of a mascot,
The second emotion is expressed by the facial expression, color, sound or voice, or action of the mascot,
Information provision support method. The information provision support method according to any one of claims 5 to 8,
The importance is determined to be high when the information is information related to the driving operation of the vehicle, and the importance is determined to be low when the information is information other than the information related to the driving operation of the vehicle. Ru
Information provision support method. An information provision support device for supporting information provision from a vehicle to a driver,
an information acquisition unit that acquires information to be provided to the driver from the vehicle;
a driver action recognition unit that recognizes the action of the driver;
In order to know the information, or when the driver performs a predetermined action to be performed based on the information, the response action that has been performed in response to the predetermined action is performed after obtaining the information. , executed before the driver performs the predetermined action;
Information provision support device. An information provision support program for operating an information provision support device for supporting information provision from a vehicle to a driver,
causing the information provision support device to acquire information to be provided to the driver from the vehicle;
causing the information provision support device to recognize the operation of the driver;
A response action that has been executed by the information providing support device in response to the predetermined action performed by the driver in order to obtain the information or based on the information. is executed by the information provision support device after the information is acquired and before the driver performs the predetermined action.
Informational support program. An information providing system that includes a vehicle and an information provision support device that supports information provision from the vehicle to a driver, and provides information to the driver using the information provision support device,
The information provision support device acquires information to be provided to the driver from the vehicle,
The information provision support device recognizes the operation of the driver,
In order to acquire the information, or when the driver performs a predetermined action to be performed based on the information, a response action that the information provision support device has performed in response to the predetermined action is performed. , executed before the driver performs the predetermined action;
information system.

Images