JP7181744B2 - Vehicle agent device - Google Patents

Description

The present invention relates to a vehicle agent device that enables communication between an occupant and an anthropomorphic character.

In vehicles such as automobiles, it is an ongoing challenge to further improve safety by improving the driving environment for passengers (especially drivers) (for example, to prevent drowsiness). Various methods have been proposed as methods.
For example, a vehicular agent device has been proposed as one of the above-described techniques, which enables communication between a passenger and an anthropomorphic character (a so-called agent). Document 1 proposes detecting the direction of the face or line of sight of an occupant to control the display of an agent image.

Further, in the vehicle agent device disclosed in Patent Document 1, when the line of sight of the occupant is not directed forward, the agent image issues a warning notification to call the occupant's attention, thereby improving safety. It is

JP 2006-189394 A

According to Patent Literature 1, although there is an advantage that comfortable communication can be realized between the passenger and the agent, excessive communication is attempted, and the agent issues a warning notification. There is a possibility that the warning information may be overlooked or missed.

SUMMARY OF THE INVENTION The present invention has been made to solve such conventional problems. An object of the present invention is to provide an agent device for

A vehicle agent device according to the present invention is a vehicle agent device capable of two-way communication between a character and an occupant, comprising: agent control means for controlling the character; a situation acquiring means for acquiring at least one of the occupant's physical or thinking situation; and an identification means for identifying whether the situation acquired by the situation acquiring means corresponds to a specific situation; The agent control means controls two-way communication between a visually or aurally anthropomorphic normal personified character and a passenger, and controls the normal personified character by means of the identification means. The method is characterized in that when a specific situation is identified, a specific personified character is controlled, and the specific personified character issues a warning regarding the specific situation.

Further, the agent control means may disable communication from the character when the specific personified character is giving warning information regarding the specific situation.

Further, the agent control means is heteronomous conversation control means for controlling two-way communication between the character and the passenger, and responding to the passenger with the contents of the input conversation when the passenger inputs the conversation. and autonomous conversation control means for conversing with the occupant based on predetermined occupant information irrespective of the presence or absence of conversation input from the occupant, and when controlling the specific anthropomorphic character. may regulate the conversation by the autonomous conversation control means or the reply by the heteronomous conversation control means.

Further, the agent control means controls the normal personified character capable of communication after causing the specific personified character to issue a warning regarding the specific situation, and controls the normal personified character capable of communication. You may make it perform the support alerting|reporting of warning alerting|reporting.

Further, when the agent control means is identified as corresponding to the specific situation and the specific situation is at the first level, the agent control means only issues a warning to the specific personified character concerning the specific situation. and if the specific situation is at a second level with a higher degree of danger than the first level, the specific personified character is given a warning about the specific situation and a warning about the specific situation. You may make it perform the support notification of notification.

According to the present invention, it is possible to provide a vehicle agent device that enables a passenger to accurately recognize a warning when an anthropomorphic character issues a warning.

1 is a perspective view showing the interior of a vehicle according to one embodiment of the present invention; FIG. FIG. 4 is a front view showing a state in which the first display panel and the second display panel are viewed from the driver's seat side; 1 is a block diagram for explaining a vehicle agent device; FIG. FIG. 4 is a main flow chart diagram performed by a control unit of the vehicle agent device; FIG. 4 is a flow chart diagram showing notification control processing performed by a control unit of the vehicle agent device; FIG. 11 is an animation diagram showing a notification example 1 of the vehicle agent device; FIG. 11 is an animation diagram showing a notification example 2 of the vehicle agent device;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

1 is a perspective view showing the interior of a vehicle according to one embodiment of the present invention, and FIG. FIG. 3 is a block diagram for explaining the vehicle agent device, FIG. 4 is a main flow chart performed by the controller of the vehicle agent device, and FIG. 5 is a control of the vehicle agent device. FIG. 6 is an animation diagram showing notification example 1 of the vehicle agent device, and FIG. 7 is an animation diagram showing notification example 2 of the vehicle agent device. In the drawing, FR indicates the front of the vehicle, and UP indicates the upper side of the vehicle. Further, the left-right direction in the following description means the left-right direction when an occupant seated in the vehicle faces the front of the vehicle.

(Each configuration in the passenger compartment)
As shown in FIGS. 1 and 2, a vehicle 1 includes an instrument panel 3 provided in front of a driver's seat 2 and a steering wheel 4 disposed between the driver's seat 2 and the instrument panel 3. have. The steering wheel 4 is rotatably attached to a steering column (not shown) via a steering shaft (not shown). An airbag 8 that deploys toward the driver H when the vehicle 1 collides is stored inside the steering wheel 4 .

As shown in FIG. 1 , the interior of the vehicle 1 is provided with three display panels, a first display panel 30 , a second display panel 40 and a third display panel 50 . The first display panel 30 and the second display panel 40 are arranged separately on the instrument panel 3 in front of the driver's seat 2 , and the third display panel 50 is located on the left of the driver's seat 2 . It is arranged on the instrument panel 3 on the front side.

(First display panel 30)
As shown in FIGS. 1 and 2, the first display panel 30 includes a pointer-type meter configured like an analog clock, and a so-called liquid crystal display device in which a liquid crystal panel and a backlight are integrally provided. configured with. A driver H (hereafter referred to as driver H) can view various information displayed in the first display area 30a of the first display panel 30 through the upper space 4a of the steering wheel 4. It has become.

As shown in FIG. 2, the first display panel 30 is provided with a first display area 30a. Two pointer type meters that display information such as the number of revolutions per unit time (tachometer), and general vehicle information in the central part of the first display panel 30 between the two pointer type meters and a small liquid crystal display device for displaying an image showing It should be noted that the first display panel 30 may be entirely composed of one liquid crystal display device without including a needle-type meter.

(Second display panel 40)
As shown in FIGS. 1 and 2, the second display panel 40 is composed of, for example, a so-called liquid crystal display device in which a liquid crystal panel and a backlight are integrally provided. A pair of rotating shafts 41 protruding in opposite directions are attached to both ends of each. The rotating shaft 41 is pivotally supported by a support member (not shown) so that it can be freely changed between an upright position in which it stands up and a down position in which it folds down toward the front of the vehicle. . The driver H can see various images displayed in the second display area 40 a of the second display panel 40 through the upper space 4 a of the steering wheel 4 . Although the second display panel 40 is configured to be freely variable between the standing position and the laid-down position, the second display panel 40 can be moved between the protruded position where it protrudes from the support member while being erected and the retracted position where it is stored inside the support member. may be configured to be vertically variable, or may be fixed so as not to be variable.

As shown in FIG. 2, the second display panel 40 is provided with a second display area 40a. In this second display area 40a, a normal anthropomorphic character image I1 and a normal anthropomorphic character image I1 to be described later. A dialogue image I2 emitted from the character image I1, a specific anthropomorphic character image I3, which will be described later, and a warning notification image I4 (see FIGS. 6 and 7) emitted from the specific anthropomorphic character image I3 are displayed. Since the second display area 40a is located between the speedometer and the tachometer, the driver H can check the speedometer, the tachometer, and the normal anthropomorphic character image I1 without moving the line of sight significantly. and dialogue image I2, and a specific personified character image I3 and warning notification image I4. Characteristic portions using the normal anthropomorphic character image I1 and the specific anthropomorphic character image I3 will be described later with reference to FIGS.

(Third display panel 50)
As shown in FIG. 1, the third display panel 50 is composed of, for example, a so-called liquid crystal display device in which a liquid crystal panel and a backlight are integrally provided. Note that the third display panel 50 displays map information and the like, and serves as a so-called car navigation system.

The liquid crystal display device constituted by the first display panel 30, the second display panel 40, and the third display panel 50 may be replaced by a self-luminous display device such as a plasma display or organic EL, or a projection type projector. or the like.

(Operation button 60)
As shown in FIG. 2, the operation button 60 is an operation device provided with keys for four directions of up, down, left, and right. A signal can be input. As a result, for example, it is possible to change the volume value of the voice corresponding to the dialogue image I2 emitted from the normal anthropomorphic character image I1, and to change the display position of the normal anthropomorphic character image I1.

Next, the configuration of the vehicle agent device 100 according to the present invention will be described with reference to the block diagram of FIG.

As shown in FIG. 3, the vehicle agent device 100 includes a control unit 110, a surrounding environment recognition unit 120, an occupant state recognition unit 130, a vehicle state recognition unit 140, a second display panel 40, and operation buttons. 60, a speaker 150, a microphone 160, a storage unit 170, and a transceiver 180.

The vehicle agent device 100 shown in FIG. 3 is merely an example, and the constituent elements of the vehicle agent device 100 can be changed as appropriate. For example, if at least one of the surrounding environment recognition unit 120, the occupant state recognition unit 130, and the vehicle state recognition unit 140 is provided, the vehicle agent device 100 of the present invention can be realized. 60, storage unit 170, and transmitter/receiver 180, the vehicle agent device 100 of the present invention can be realized. Further, if at least the speaker 150 and the microphone 160 are provided, the vehicle agent device 100 of the present invention can be realized without the second display panel 40 .

In addition, for example, when a voice input (for example, a question) is made by the driver H through the microphone 160, the vehicle agent device 100 according to the present invention responds to the input by the speaker 150 or the second display panel 40 ( Dialogue in a passive style performed via a normal anthropomorphic character image I1) and, for example, even if the voice input from the driver H is not performed via a microphone 160 described later, the tastes and preferences of the driver H It is possible to have both an active style of dialogue in which the vehicle agent device 100 talks about topics, etc., through the speaker 150 and the second display panel 40 (normal anthropomorphic character image I1). . In particular, by enabling dialogue in an active style, active talking can contribute to preventing drowsiness of the driver H and alleviating driving stress.

(control unit 110)
The control unit 110 includes a CPU, a ROM, a RAM (for example, a ring buffer), and an input/output port (not shown). Various devices (the second display panel 40 and the speaker 150) are controlled via the output port.

It should be noted that the ROM of the control unit 110 stores a data table (not shown) of lines emitted from a normal anthropomorphic character image I1 and a specific anthropomorphic character image I3. The CPU of the control unit 110, based on information obtained from each recognition unit (to be described later) and information obtained from the microphone 160, selects the lines emitted from the normal anthropomorphic character image I1 and the lines emitted from the specific anthropomorphic character image I3. is determined from the data table. For example, when information requesting that BGM be played is obtained from the microphone 160, a normal line of the personified character image I1 such as "I understand" is determined from the data table.

(Surrounding environment recognition unit 120)
The surrounding environment recognition unit 120 is provided to recognize the surrounding environment of the vehicle 1 (own vehicle). In addition, the surrounding environment recognition unit 120 includes an exterior camera 120a and a radar 120b, and can recognize the surrounding environment of the vehicle 1 by means of these devices.

(External photography camera 120a)
The exterior photographing camera 120 a is attached to, for example, a rearview mirror (not shown), and is capable of photographing the front of the vehicle 1 and the rear of the vehicle 1 . Then, the captured image information is input to the control unit 110, and the control unit 110 stores the image information in the RAM. Thereby, the control unit 110 can recognize the situation in front of the vehicle 1 and in the rear of the vehicle 1 both in real time and after the fact.

(Radar 120b)
The radar 120b uses, for example, a millimeter-wave radar that emits radio waves to detect obstacles and the like. Front side monitoring and rear side monitoring of the vehicle 1 can be performed. Then, the monitoring information is input to the control section 110, and the control section 110 stores the monitoring information in the RAM. Thereby, the control unit 110 can recognize the situation in front of the vehicle 1, the situation in the front side of the vehicle 1, and the situation in the rear side of the vehicle 1 both in real time and after the fact. In addition, although the millimeter wave radar is used in this embodiment, other radars may be used. For example, infrared radar may be used.

As described above, the control unit 110 can recognize the surrounding environment of the vehicle 1 both in real time and after the fact. Accordingly, the control unit 110 controls the normal anthropomorphic character image I1 on the second display panel 40 and the speaker 150 based on the recognition of the surrounding environment of the vehicle 1 to generate information on the surrounding environment of the vehicle 1. can be notified to the driver H. For example, if there is a falling object in front of the vehicle 1, it is possible to notify the vehicle 1 by means of an image and sound such as "There is a falling object ahead". Thereby, safety can be improved.

In this embodiment, the camera 120a for photographing outside the vehicle and the radar 120b are used as the surrounding environment recognition unit 120, but these are merely examples, and other devices may of course be used.

(Occupant state recognition unit 130)
The occupant state recognition unit 130 is provided for recognizing the state of the driver H. As shown in FIG. The occupant state recognizing unit 130 includes an occupant photographing camera 130a and a vital sensor 130b, and can recognize the state of the driver H by means of these devices.

(Passenger photographing camera 130a)
The occupant imaging camera 130a is attached to the instrument panel 3, for example, so that the driver H can be photographed. Then, the captured image information is input to the control unit 110, and the control unit 110 stores the image information in the RAM. Thereby, the control part 110 can recognize the state of the driver H in real time and after the fact. The state of the driver H mentioned here specifically includes the state of the eyelids of the driver H, the number of blinks, the direction of the line of sight, the direction of the face, and the like.

(Vital sensor 130b)
The vital sensor 130b is attached, for example, to a portion of the steering wheel 4 that is gripped by the driver H, so that vital information such as heart rate and blood pressure of the driver H can be obtained. Then, the acquired vital information is input to the control unit 110, and the control unit 110 stores the vital information in the RAM. Thereby, the control part 110 can recognize the state of the driver H in real time and after the fact.

As described above, the control unit 110 can recognize the state of the driver H in real time and after the fact. Accordingly, the control unit 110 controls the normal anthropomorphic character image I1 on the second display panel 40 and the speaker 150 based on the recognition of the state of the driver H to transmit predetermined information to the driver H. can be notified. For example, it is possible to provide an image and voice notification such as ``Your eyelids are drooping, would you like to take a break?'' and an image and voice notification such as ``Your heart rate is faster than usual. . This leads to an improvement in safety.

The occupant state recognition unit 130 can recognize the thoughts and feelings of the driver H to some extent based on the information acquired from the occupant photographing camera 130a and the vital sensor 130b and the information input from the microphone 160. FIG. For example, the driver H's facial expression is acquired from the passenger camera 130a, the heart rate and blood pressure of the driver H are acquired from the vital sensor 130b, the voice volume and input content are acquired from the microphone 160, and from these acquired information It can be recognized whether the driver H is in a normal thinking feeling or a different thinking feeling (for example, surprised, angry, etc.).

In the present embodiment, the occupant photographing camera 130a and the vital sensor 130b are used as the occupant state recognition unit 130, but this is merely an example, and other devices may of course be used.

(Vehicle state recognition unit 140)
Vehicle state recognition unit 140 is provided to recognize the state of vehicle 1 . The vehicle state recognition unit 140 includes a vehicle speed sensor 140a, a steering wheel angle sensor 140b, an accelerator pedal sensor 140c, a brake pedal sensor 140d, and a G sensor 140e. can be recognized.

(Vehicle speed sensor 140a)
The vehicle speed sensor 140a is a sensor for detecting the vehicle speed of the vehicle 1. The detected vehicle speed is input to the control unit 110 as a vehicle speed signal, and the control unit 110 stores the vehicle speed information in RAM. Thereby, the control part 110 can recognize the vehicle speed of the vehicle 1 in real time and after the fact.

(Handle angle sensor 140b)
The steering wheel angle sensor 140b is a sensor for detecting the steering wheel angle of the vehicle 1 (the angle of the steering wheel 4). to store the angle information. Thereby, the control unit 110 can recognize the steering wheel angle (the angle of the steering wheel 4) of the vehicle 1 both in real time and after the fact.

(Accelerator pedal sensor 140c)
The accelerator pedal sensor 140c is a sensor for detecting the amount of depression of an accelerator pedal (not shown). memorize Thereby, the control part 110 can recognize the depression amount of the accelerator pedal of the vehicle 1 in real time and after the fact.

(Brake pedal sensor 140d)
The brake pedal sensor 140d is a sensor for detecting the amount of depression of a brake pedal (not shown). memorize Thereby, the control part 110 can recognize the depression amount of the brake pedal of the vehicle 1 in real time and after the fact.

(G sensor 140e)
The G sensor 140e is a sensor for detecting the acceleration, deceleration and tilt of the vehicle 1, and detects the acceleration amount when acceleration is detected, the deceleration amount when deceleration is detected, and the tilt. In this case, the tilt angle amount is input to the control unit 110 as an acceleration amount signal, a deceleration amount signal, and a tilt angle signal, respectively, and the control unit 110 stores the acceleration information, deceleration information, and tilt information in the RAM. Thereby, the control part 110 can recognize the acceleration, deceleration, and inclination of the vehicle 1 in real time and after the fact.

As described above, the control unit 110 can recognize the state of the vehicle 1 both in real time and after the fact. As a result, the control unit 110 controls the normal personified character image I1 on the second display panel 40 and the speaker 150 based on the recognition of the state of the vehicle 1 to transmit the information on the state of the vehicle 1 to the driver. H can be notified. For example, if the vehicle is traveling at an appropriate speed, it can be notified by an image and sound that "You are traveling at an appropriate speed." This leads to an improvement in safety.

In this embodiment, the vehicle speed sensor 140a, the steering wheel angle sensor 140b, the accelerator pedal sensor 140c, the brake pedal sensor 140d, and the G sensor 140e are mentioned as the vehicle state recognition unit 140, but this is only an example. Of course, other devices may be used instead.

(Speaker 150)
The speaker 150 is attached to the instrument panel 3, for example, and outputs a sound corresponding to the dialogue image I2 emitted from the normal anthropomorphic character image I1. An audio speaker built in the vehicle 1 may be used without providing the speaker 150 .

(mic 160)
The microphone 160 is attached to the instrument panel 3, for example, and receives voices from the driver H and other passengers.

(storage unit 170)
The storage unit 170 can store information acquired from each of the recognition units described above, dialogue between the driver H and the control unit 110, and the like. By accumulating such information in the storage unit 170, the control unit 110 can determine the driving tendency of the driver H (for example, how the driver drives) and the hobbies and preferences of the driver H (for example, which Do you like such BGM?), etc., can be recognized. In addition, by recognizing these, it is possible to actively engage in conversations that match the driving tendencies of the driver H and the tastes and preferences of the driver H from the side of the vehicle agent device 100 (normal anthropomorphic character image I1).

(Transceiver 180)
The transmitter/receiver 180 can, for example, acquire information using a vehicle-mounted wireless LAN, acquire position information using a satellite positioning system, and the like. Based on the acquired information and the information accumulated in the storage unit 170, the control unit 110 communicates with the vehicle agent device 100 side (normal This can also be done actively from the anthropomorphic character image I1).

Next, control processing performed by the control unit 110 in realizing the above-described passive-style dialogue, active-style dialogue, and warning notification when a risk factor occurs, which will be described later. is performed using FIGS. 4 and 5. FIG. The control flows shown in FIGS. 4 and 5 are stored in the ROM of the control unit 110, and the CPU of the control unit 110 reads them from the ROM and performs various processes.

First, FIG. 4 shows main processing performed by the control unit 110 . The main process is a process that is periodically performed by the control unit 110 every predetermined cycle.

(Step S10)
In step S10, the CPU of the control unit 110 performs vehicle state recognition processing. Specifically, the state of the vehicle 1 is recognized based on the information input from the vehicle state recognition section 140 . After completing the vehicle state recognition process, the process proceeds to step S20.

(Step S20)
In step S20, the CPU of the control unit 110 performs surrounding environment recognition processing. Specifically, the surrounding environment of the vehicle 1 is recognized based on the information input from the surrounding environment recognition section 120 . Then, when the peripheral environment recognition processing is completed, the processing proceeds to step S30.

(Step S30)
At step S30, the CPU of the control unit 110 performs passenger state recognition processing. Specifically, the state of the driver H is recognized based on the information input from the passenger state recognition section 130 . When the occupant state recognition process is completed, the process proceeds to step S40.

(Step S40)
In step S40, the CPU of control unit 110 performs risk factor extraction processing. Specifically, as a result of recognizing various kinds of information in steps S10 to S30, a process of extracting whether risk factors are included is performed.

Here, the "risk factor" in this embodiment means a risk factor that may cause a vehicle accident, such as the close proximity distance from one's own vehicle to another vehicle/obstacle, etc. Excessive driving, sudden start, sudden braking, reverse driving, etc.), drowsiness estimate value and driving stress value of driver H (condition of eyelids, number of blinks, direction of gaze, direction of face, heart rate, blood pressure, etc.) is mentioned.

Each piece of information acquired from each recognition unit is provided with a predetermined threshold value, and when each piece of information acquired exceeds the predetermined threshold value, it is determined that a risk factor is included. In the case of recognition by the surrounding environment recognition unit 120, for example, when the inter-vehicle distance to the vehicle in front is less than 10 m, it exceeds a predetermined threshold value and is determined to include a risk factor. Further, in the case of recognition by the occupant state recognition unit 130, for example, if the number of times the driver H blinks per hour does not meet a predetermined number of times, it exceeds a predetermined threshold value and is determined to include a risk factor. . In the case of recognition by the vehicle state recognition unit 140, for example, when the acceleration amount exceeds a certain value, it exceeds a predetermined threshold value, and it is determined that a risk factor is included. When the risk factor extraction process is completed, the process proceeds to step S50.

(Step S50)
In step S50, the CPU of control unit 110 determines whether a specific situation exists. Note that the specific situation means a situation in which the information obtained from each recognition unit exceeds a predetermined threshold and contains risk factors. That is, as a result of performing the risk factor extraction process in step S40, it is determined whether a risk factor is included. Then, when it is determined that the specific situation exists, the process proceeds to step S60, and when it is determined that the specific situation does not exist, the process proceeds to step S70.

(Step S60)
In step S60, the CPU of the control unit 110 performs a change process to a specific personified character (personified character image I3 shown in FIG. 6B). Specifically, the CPU of the control unit 110 replaces the identification flag "0" corresponding to the normal anthropomorphic character (the anthropomorphic character image I1 shown in FIG. 6A) in the RAM with the specific anthropomorphic character set the identification flag "1" corresponding to Then, after finishing the process of changing to a specific personified character, the process proceeds to step S61.

It should be noted that a plurality of types of risk factors are assumed to be extracted in step S40 described above, and the degree of risk differs depending on the type of risk factor. The specific anthropomorphic character to be used is preferably a single character (see FIG. 6(b-1), etc.). As a result, the character that gives the warning does not change depending on the content of the risk factor or the degree of danger, but by using a single character, when the character that gives the warning is displayed, some kind of risk factor is displayed. It is possible to make the driver H strongly recognize that it is occurring.

(Step S61)
In step S61, the CPU of the control unit 110 performs support notification mode creation processing for determining the character and notification content for which the support notification is to be given, depending on the difference in the degree of risk of the risk factors. Specifically, for those with a low degree of risk factor, an ordinary anthropomorphic character is determined, and output data for notification content for eliminating the risk factor is created, and for those with a high degree of risk factor. determines a specific anthropomorphic character and creates output data of notification contents for eliminating the risk factor. Then, when the support notification mode creating process is completed, the process proceeds to step S70. In this manner, when the support notification (see FIGS. 6(c) and 7(c)) is performed, the character to which the support notification is performed is changed according to the difference in the degree of danger.

(Step S70)
In step S70, the CPU of control unit 110 determines whether there is a specific situation. Specifically, as a result of extracting the risk factor in step S40, it is determined whether the risk factor was not extracted, that is, whether the vehicle was in a specific situation. If it is determined that the specific situation has not occurred, the process proceeds to step S80, and if it is determined that the specific situation has occurred, the process proceeds to step S100.

(Step S80)
In step S80, the CPU of control unit 110 determines whether or not a specific personified character is being set. Specifically, the identification flag of the RAM is referred to, and if "0" is set, it is recognized that a normal anthropomorphic character is being set, and if "1" is set, a specific Recognize that an anthropomorphic character is being set. That is, in this process, it is determined whether or not the identification flag of the RAM is set to "1". When it is determined that the specific personified character is being set, the process proceeds to step S90, and when it is determined that the specific personified character is not being set, the process is moved to step S100.

(Step S90)
In step S90, the CPU of the control unit 110 performs normal change processing to an anthropomorphic character. Specifically, "0" is set instead of the RAM identification flag "1". Thus, if no risk factor is extracted, the specific personified character is changed to a normal personified character. As a result, when a conversation input is made by the passenger, it is possible to respond from the vehicle agent device 100 side. Then, when the change process to the normal personified character is completed, the process proceeds to step S100.

(Step S100)
In step S100, the CPU of the control unit 110 performs notification control processing. Note that the notification control process will be described in detail later with reference to FIG. After finishing the notification control process, the main process ends.

Next, the notification control process performed by the CPU of the control unit 110 will be described with reference to FIG. FIG. 5 is a subroutine of step S100 in FIG.

(Step S101)
In step S101, the CPU of the control unit 110 determines whether or not a specific personified character is being set. Specifically, the identification flag in the RAM is referenced to determine whether "1" is set. When it is determined that the specific personified character is being set, the process proceeds to step S105, and when it is determined that the specific personified character is not being set, the process proceeds to step S102.

(Step S102)
In step S102, the CPU of the control unit 110 performs autonomous dialogue control processing. Specifically, a process for performing a dialogue in an active style, which is performed from the side of the vehicle agent device 100 described above, is performed. For example, when the driver H gets into the car and is driving on a straight road, the normal anthropomorphic character image I1 may be displayed as follows: I'm sorry..." is reported (spoken) by voice and image. If the driver H responds to the speech, the topic is continued (steps S103 to S106, which will be described later). For example, if the driver H responds through the microphone 160, such as "I will win today because xx will be playing", the normal anthropomorphic character image I1 will respond "Yes! xx is a reliable player!" Two-way dialogue is possible. On the other hand, if the driver H does not respond, another subject is brought up after a predetermined period of time, and the normal anthropomorphic character image I1 informs (talks to) the driver again. If there is no response from the driver H after speaking to the driver a plurality of times, it can be recognized that the driver H does not want to have a conversation, and the frequency of speaking can be reduced. After completing the autonomous dialogue control process, the process proceeds to step S103.

(Step S103)
In step S103, the CPU of control unit 110 determines whether or not there has been a passenger's conversation input. Specifically, it is determined whether voice input is performed from the input port of the microphone 160 . The control unit 110 is provided with a known voice recognition unit (not shown), and the voice recognition unit determines what the input voice (conversation content) is. ing. Further, the passenger here is not limited to the driver H, and may be another passenger. In addition, it is possible to input a conversation by operating the operation button 60 instead of the voice input by the microphone 160 . However, in view of safety, it is preferable to be able to input a conversation with a one-touch operation. In addition, the passenger's conversational input in this process also includes the above-described passive-style interaction. Then, when it is determined that there is an input of conversation by the passenger, the process proceeds to step S104, and when it is determined that there is no input of conversation by the passenger, the process proceeds to step S105.

(Step S104)
In step S104, the CPU of the control unit 110 performs response control processing for the corresponding conversation content. Specifically, the above-mentioned speech recognition unit judges what kind of content the input voice (conversation content) is, determines the content of the response, and determines the display data of the lines and the voice data of the lines. to generate When the response control process for the corresponding conversation content is completed, the process proceeds to step S105.

Here, when it is determined that the specific personified character is not being set, the autonomous dialogue control process of step S102 and the response control process of step S104 are performed, but the specific personified character is being set. If so, the autonomous dialogue control process in step S102 and the response control process in step S104 are not performed (restricted). For this reason, when a normal anthropomorphic character image I1 shown in FIG. When the specific anthropomorphic character image I3 shown in b) is displayed, neither active-style conversations nor responses to conversations by passengers are performed. That is, the passenger is configured to be unable to communicate with a specific personified character. As a result, the passenger recognizes the specific anthropomorphic character image I3 as a special image different from the normal anthropomorphic character image I1.

(Step S105)
In step S105, the CPU of the control unit 110 performs image display processing. Specifically, the display data of the words determined in step S102 and the display data of the words generated in step S104 are output, and the process of displaying the words on the second display panel 40 is performed. Then, when the image display process is completed, the process proceeds to step S106.

(Step S106)
In step S106, the CPU of the control unit 110 performs sound output processing. Specifically, the voice data of the lines determined in step S102 and the voice data of the lines generated in step S104 are output, and the lines are output from the speaker 150. FIG. Then, when the sound output processing ends, the main processing ends. Through the above-described processing, active-style dialogue, passive-style dialogue, and warning notification when a risk factor occurs are realized from the side of the vehicle agent device 100 described above. .

Next, an example of notification performed by the control process described above will be described with reference to FIGS. 6 and 7. FIG. 6 and 7 show the display screen of the second display panel 40, and the speaker 150 is not shown. explain.

First, the normal anthropomorphic character image I1 is a character image simulating a woman, and from the normal anthropomorphic character image I1, the words and voice of a woman are output. On the other hand, the specific anthropomorphic character image I3 is a character image simulating a man, and from the specific anthropomorphic character image I3, the lines and voice of the man are output. When the risk factor is not extracted, the normal personified character image I1 is displayed on the second display panel 40, and when the risk factor is extracted, the normal image is displayed on the second display panel 40. A specific personified character image I3 is displayed in place of the personified character image I1, and the specific personified character image I3 notifies the user of warning information related to the risk factors by means of images and sounds. A specific example will be described below.

FIG. 6(a-1) shows, for example, a dialogue in an active style, in which the vehicular agent device 100 side (normal anthropomorphic character image I1) talks about the topic of weather (word image I2). shows what is being done. Although not shown, it is assumed that the driver H and the vehicle agent device 100 have had several conversations after FIG. 6(a-1).

Also, at this time, as shown in FIG. 6(a-2), it is assumed that the vehicle 1 is traveling in the left lane L of a two-lane road and is traveling in an appropriate position in the left lane L. there is

In FIG. 6(b-2), while the driver H and the vehicle agent device 100 are having a conversation several times, the following vehicle B, which is traveling in the right lane R at a speed higher than a predetermined speed, appears. exceeds a predetermined threshold. In this case, the control unit 110 uses the surrounding environment recognition unit 120 (existence, speed, etc. of the following vehicle B) and the vehicle state recognition unit 140 (vehicle 1 speed, etc.) to recognize the rapid approach of the following vehicle B (there is a risk factor). and recognize).

When the control unit 110 recognizes the rapid approach of the following vehicle B, the control unit 110 erases the normal personified character image I1 and displays the specific personified character image I3 as shown in FIG. Then, from the specific personified character image I3, a warning notification image I4 for warning of the rapid approach of the following vehicle B and a sound corresponding to the warning notification image I4 are output. As a result, even if the driver H is preoccupied with the dialogue with the vehicle agent device 100, a different character image appears and the warning is issued from the character image, so that the warning can be easily noticed. .

In the event of FIG. 6, it is assumed that the vehicle is traveling in the proper position of the left lane L, so the control unit 110 determines that the risk factor (degree of risk) is at the first level, As shown in FIG. 6(c), the specific anthropomorphic character image I3 is erased, the normal anthropomorphic character image I1 is displayed, and the current driving lane is maintained by the normal anthropomorphic character image I1. A support notification image I5 and a sound corresponding to the support notification image I5 are output (support notification is performed). As a result, even if a risk factor occurs, the driver H can be made aware of how to deal with the risk factor afterward, which contributes to an improvement in safety.

Next, FIG. 7 shows an example of notification when an event different from that in FIG. 6 occurs. It is assumed that the 100 side (normal anthropomorphic character image I1) speaks about the topic of the weather (words image I2), and then the driver H and the vehicle agent device 100 have a dialogue several times. ing.

In FIG. 7(b-2), the driver H is so absorbed in the dialogue with the vehicle agent device 100 that the vehicle 1 travels on the separation line C (inappropriate position). It is assumed that a following vehicle B, traveling on R at a predetermined speed or higher, appears and the mutual distance exceeds a predetermined threshold. In this case, the control unit 110 uses the surrounding environment recognition unit 120 (driving position of the vehicle 1, presence of the following vehicle B, speed, etc.) and the vehicle state recognition unit 140 (speed of the vehicle 1, etc.) to It recognizes that the vehicle 1 is approaching and that the running position of the vehicle 1 is not appropriate (recognizes that there is a risk factor).

Then, when the control unit 110 recognizes that the following vehicle B is approaching rapidly and that the vehicle 1 is not in a proper running position, the control unit 110 erases the normal personified character image I1 as shown in FIG. 7(b-1). Then, a specific personified character image I3 is displayed, and from the specific personified character image I3, a warning notification image I4 for notifying a rapid approach of the following vehicle B and a sound corresponding to the warning notification image I4 are output. . As a result, an effect similar to that described with reference to FIG. 6(b-1) can be obtained.

In the event of FIG. 7, it is assumed that the vehicle 1 is traveling on the separation line C (inappropriate position). Then, as shown in FIG. 7(c), the display of the specific personified character image I3 is maintained, and from the specific personified character image I3, A support notification image I6 notifying that it is better to move to the left lane L side and a sound corresponding to the support notification image I6 are output (support notification is performed). Thus, when the risk factor (degree of risk) is high, the display of the specific personified character image I3 is maintained, and how to deal with the risk factor from the specific personified character image I3. By providing this support notification, it is possible to make the driver H strongly recognize how to deal with the risk factor. As a result, it is possible to promptly respond to risk factors, which contributes to the improvement of safety.

(Modification)
The assumed modified examples are added below.

In the above embodiment, the second display panel 40 is provided separately from the first display panel 30, and the normal anthropomorphic character image I1 and the like are displayed on the second display panel 40. The panel 40 may not be provided, and the normal anthropomorphic character image I1 or the like may be displayed on the first display panel 30. FIG.

Alternatively, the normal anthropomorphic character image I1 or the like may be displayed on the third display panel (the display device serving as the car navigation system) without providing the second display panel 40 .

Further, instead of the second display panel 40, for example, a head-up display that projects information onto the windshield may be used to display a normal anthropomorphic character image I1 or the like on the windshield.

Alternatively, the speaker 150 and the microphone 160 may be used to perform a dialogue using only sound without displaying the normal anthropomorphic character image I1 or the like.

In addition, although the specific personified character that issues the warning does not change depending on the content of the risk factor and the degree of danger, and a single character is used, this is not the only option. The particular anthropomorphic character performed may vary.

In addition, although the character to which the support notification is given is differentiated according to the degree of danger, the specific personified character that gave the warning notification may directly give the support notification, or the support notification may not be performed.

Further, the output volume of the dialogue emitted from the specific anthropomorphic character image I3 may be the same output value as the output volume of the dialogue emitted from the normal anthropomorphic character image I1. It may be louder than the output volume of the dialogue emitted from. In particular, when the output volume of the words emitted from the normal anthropomorphic character image I1 is increased, it is possible to easily notice the warning notification.

Further, the adjustment of the volume value by the operation button 60 may reflect only the lines emitted from the normal anthropomorphic character image I1. For example, the dialogue emitted from the normal anthropomorphic character image I1 can be adjusted with a volume value in the range of "0 to 9", but the dialogue emitted from the specific anthropomorphic character image I3 cannot be adjusted. , may always be output with a volume value of "10". Even with this configuration, it is possible to make it easier to notice the warning information.

Further, the size of the normal personified character image I1 displayed on the second display panel 40 and the size of the specific personified character image I3 may be substantially the same, or the size of the specific personified character image I3 may be the same. I3 may be displayed larger. Even with this configuration, it is possible to make it easier to notice the warning notification.

Also, if BGM is being output when a risk factor occurs and a warning is issued, the BGM may be stopped and the warning may be issued. Even with this configuration, it is possible to make it easier to notice the warning information.

Also, when a specific anthropomorphic character is being set, the autonomous dialogue control processing and the response control processing are configured not to be performed, but it may be configured such that only the autonomous dialogue control processing is not performed. However, the configuration may be such that only the response control process is not performed. Furthermore, both an autonomous dialogue control process and a response control process may be performed to enable two-way communication with a specific anthropomorphic character.

Also, the surrounding environment recognition unit 120, the passenger state recognition unit 130, and the vehicle state recognition unit 140 are provided, and information is acquired and recognized from all of them. The present invention can also be realized by providing any two recognition units.

For example, when only the surrounding environment recognition unit 120 is provided, when an emergency vehicle is recognized within a radius of 100 m of the vehicle 1 using the camera 120a for photographing outside the vehicle or the radar 120b, it is determined that the vehicle exceeds a predetermined threshold value, and the vehicle is identified. The personified character image I3 may be made to issue a warning indicating the presence of an emergency vehicle.

Further, for example, when only the occupant state recognition unit 130 is provided, the occupant photographing camera 130a is used to measure the number of times the occupant blinks, etc., and it is determined whether the measured value exceeds a predetermined threshold value. If it exceeds, the specific personified character image I3 may be made to issue a warning to prevent drowsiness.

Further, for example, in the case where only the vehicle state recognition unit 140 is provided, the vehicle speed sensor 140a and the brake pedal sensor 140d are used to determine whether the side brake sensor (not shown) is ON (the side brake is applied). First, when the vehicle speed sensor 140a detects the vehicle speed, it is determined that the vehicle speed has exceeded a predetermined threshold value, and a warning notification indicating that the handbrake is being applied to the specific personified character image I3 is performed.

(Regarding correspondence)
In this embodiment, the control unit 110 constitutes agent control means for controlling a character of the present invention.

Further, in the present embodiment, the surrounding environment recognition unit 120, the occupant state recognition unit 130, and the vehicle state recognition unit 140 recognize at least one of the driving conditions of the vehicle or the conditions of the surrounding environment, or the condition of the occupant's body or thoughts. A status acquisition means for acquiring the status is configured.

Further, in the present embodiment, the control unit 110 constitutes identification means for identifying whether the situation acquired by the situation acquisition means corresponds to a specific situation.

In addition, in this embodiment, the ordinary anthropomorphic character image I1 constitutes an ordinary anthropomorphic character visually or aurally anthropomorphized according to the present invention.

Also, in this embodiment, the specific personified character image I3 constitutes the specific personified character of the present invention.

Also, in this embodiment, the alert annunciation image I4 and the corresponding sound constitute the particular situational alert annunciation of the present invention.

In addition, in this embodiment, the control unit 110 that performs response control processing of the content of the conversation corresponding to the input of conversation by the passenger is used as control means for two-way communication between the character and the passenger of the present invention. It constitutes a rule-type conversation control means.

Further, in the present embodiment, the control unit 110, which performs autonomous dialogue control processing regardless of the presence or absence of conversational input from the passenger, is an autonomous type control unit as control means for two-way communication between the character and the passenger of the present invention. Configure conversation control means.

Further, in the present embodiment, the driving tendency of the driver H and the hobbies and preferences of the driver H constitute the predetermined occupant information of the present invention.

Also, in this embodiment, the support notification image I5 and corresponding sound and the support notification image I6 and corresponding sound constitute the support notification of the warning notification for the specific situation of the present invention.

1: vehicle, 40: second display panel, 110: control unit, 120: surrounding environment recognition unit, 130: occupant state recognition unit, 140: vehicle state recognition unit, 150: speaker, 160: microphone

Claims (4)

A vehicle agent device capable of two-way communication between a character and an occupant,
agent control means for controlling the character;
a situation acquisition means for acquiring at least one of the situation of the driving of the vehicle or the situation of the surrounding environment, or the situation of the body or mind of the occupant;
identification means for identifying whether the situation acquired by the situation acquisition means corresponds to a specific situation;
The agent control means is
Controls two-way communication between an ordinary anthropomorphic character that is visually or audibly anthropomorphized and a passenger,
controlling a specific personified character when the identification means identifies that the specific situation corresponds to the situation while controlling the normal personified character;
After causing the specific personified character to issue a warning regarding the specific situation, the normal personified character capable of communication is controlled to support the warning about the specific situation. ,
A vehicle agent device characterized by: A vehicle agent device capable of two-way communication between a character and an occupant,
agent control means for controlling the character;
a situation acquisition means for acquiring at least one of the situation of the driving of the vehicle or the situation of the surrounding environment, or the situation of the body or mind of the occupant;
identification means for identifying whether the situation acquired by the situation acquisition means corresponds to a specific situation;
The agent control means is
Controls two-way communication between an ordinary anthropomorphic character that is visually or audibly anthropomorphized and a passenger,
controlling a specific personified character when the identification means identifies that the specific situation corresponds to the situation while controlling the normal personified character;
causing the normal anthropomorphic character to make a support notification regarding the specific situation when the specific situation is at the first level;
When the specific situation is at a second level with a higher degree of danger than the first level, the specific personified character is notified of the warning regarding the specific situation and the warning about the specific situation is supported. to inform and
A vehicle agent device characterized by: The agent control means is
Disabling communication from the character when the specific personified character is performing a warning notification regarding the specific situation;
3. The vehicle agent device according to claim 1 or 2 , characterized in that: The agent control means is
As a means of controlling two-way communication between characters and passengers,
heteronomous conversation control means for responding to the passenger with the content of the input conversation when there is an input of conversation from the passenger;
autonomous conversation control means for conversing with the occupant based on predetermined occupant information, regardless of whether the occupant has input a conversation;
When controlling the specific anthropomorphic character, restricting the conversation by the autonomous conversation control means or the reply by the heteronomous conversation control means;
3. The vehicle agent device according to claim 1 or 2 , characterized in that:

Images