WO2019064650A1

WO2019064650A1 - Information transfer support system for vehicle

Info

Publication number: WO2019064650A1
Application number: PCT/JP2018/012651
Authority: WO
Inventors: 真吾入方; 宗義難波
Original assignee: 三菱自動車工業株式会社
Priority date: 2017-09-28
Filing date: 2018-03-28
Publication date: 2019-04-04

Abstract

An information transfer support system for a vehicle includes an internal conversation robot 100 that has an artificial intelligence (AI) to exchange information with a speaker 10 inside a vehicle via conversation, and an external conversation robot 200 that exchanges information with a speaker 20 outside the vehicle in external facilities or in another vehicle via conversation. The internal conversation robot 100 extracts information that should be transmitted to the speaker 20 outside the vehicle from the conversation with the speaker 10 inside the vehicle and transmits the extracted information to the external conversation robot. The external conversation robot 200 receives the extracted information transmitted from the internal conversation robot 100 and transmits the extracted information to the speaker 20 outside the vehicle via conversation. The external conversation robot 200 extracts information that should be transmitted to the speaker 10 inside the vehicle from the conversation with the speaker 20 outside the vehicle and transmits the extracted information to the internal conversation robot 100. The internal conversation robot 100 receives the extracted information transmitted from the external conversation robot 200 and transmits the extracted information to the speaker 10 inside the vehicle via conversation. Thereby, the present invention improves convenience for vehicle drivers using an AI.

Description

Information Transmission Support System for Vehicles

The present invention relates to a vehicle information transmission support system for supporting information transmission between a vehicle and the outside of the vehicle.

In recent years, a technology has been developed which attempts to improve the convenience of the driver of a vehicle or the like using artificial intelligence (AI). For example, Patent Literatures 1 and 2 propose a technique for providing a robot with an AI (hereinafter, referred to as an AI robot) to a vehicle to support the driver.

Patent Document 1 proposes a technology in which a vehicle is equipped with an AI robot, and a management system for managing the robot outside the vehicle is equipped to enable conversation between a vehicle-mounted robot and a driver or the like. The on-board robot responds to the voice of the driver etc. to talk, and at this time, information necessary for the reaction of the robot is sent from the management system by communication. By preparing a large amount of information on the management system side, it is supposed that various information exchange with the driver etc. becomes possible through the on-vehicle robot.

Further, according to Patent Document 2, the vehicle is equipped with an AI robot, and the degree of familiarity (numerical value) is calculated based on history contents of the driver's uttered content, facial expressions and driving conditions (rough driving, etc.). A technology has been proposed that changes the expression / motion of the robot according to the degree.

JP 2003-280688 A WO 2009/107185

However, in the technology of Patent Document 1 described above, the service that the on-vehicle robot can provide to the driver etc. is only based on the information prepared on the management system side.
Further, the technology of Patent Document 2 described above relates to information transmission only between the driver or the like in the vehicle and the on-vehicle robot.
For the driver, for example, if it is possible to contact while driving with a family at home, the convenience can be further improved. Therefore, there is room for further technical development regarding such information transmission.

The present invention was devised focusing on such problems, and provides an information communication support system for a vehicle, which can further improve the convenience of the driver of the vehicle and the like by using the AI. The purpose is to

(1) A vehicle information transmission support system according to the present invention is mounted on a vehicle, has an artificial intelligence, and has an on-vehicle conversation robot that exchanges information by exchanging a conversation with a vehicle-side speaker in the vehicle; A first communication means connected to a robot, equipped in a facility or other vehicle outside the vehicle, having artificial intelligence, and having a conversation with an outside speaker in the facility or other vehicle The on-vehicle conversation robot should transmit information from the conversation with the on-vehicle speaker to the on-vehicle speaker, the second on-vehicle conversation robot for exchanging information, and the second communication means connected to the on-vehicle conversation robot. The information is extracted and transmitted to the conversation robot outside the vehicle by the first communication means, and the conversation robot outside the vehicle receives the extraction information transmitted from the on-vehicle conversation robot side by the second communication device, and the extraction is performed Information is transmitted to the outside speaker by conversation, and the outside conversation robot extracts information to be transferred from the conversation with the outside speaker to the on-vehicle speaker, and the second communication means While transmitting to the conversation robot, the on-vehicle conversation robot receives the extraction information transmitted from the outside-of-vehicle conversation robot by the first communication means, and transmits the extraction information to the vehicle-side speaker by conversation. And

(2) A third communication unit for communicating with the outside, the extraction information from the on-vehicle conversation robot side obtained by the communication by the third communication unit, and the outside of the vehicle obtained by the communication by the third communication unit It is preferable to have an external server having a database for storing the extracted information from the conversation robot side, and information transfer between the on-vehicle conversation robot and the outside-of-vehicle conversation robot is performed via the external server.

(3) The on-vehicle conversation robot includes: a first database storing analysis information for analysis of voice recognition information and analysis of image recognition information; and first voice receiving means for receiving the voice of the vehicle speaker. A first image acquisition unit that acquires an ambient image; a first voice recognition unit that recognizes a voice received by the voice reception unit; a first image recognition unit that recognizes an image acquired by the first image acquisition unit; First speaker recognition means for recognizing the vehicle-side speaker from at least one of recognition information of the voice recognition information of the first speech recognition means and image recognition information of the image recognition means and analysis information of the first database A first conversation meaning understanding means for understanding the meaning of the conversation of the vehicle speaker from the recognition information of the first speech recognition means and the analysis information of the first database; and the conversation of the first conversation meaning understanding means Intention First conversation content generation means for generating a conversation content to be sent back to the vehicle-side speaker from the understanding information and the analysis information of the first database; and the first conversation content generation means corresponding to the conversation content A first voice synthesizing means for synthesizing a reply voice, a first voice transmitting means for transmitting a reply voice synthesized by the first voice synthesizing means, and the conversation meaning comprehension information by the first conversation meaning understanding means It is preferable to have a first information extraction means for extracting information to be transmitted to a person.
(4) The on-vehicle conversation robot further includes a first feeling estimation means for estimating the state of the emotion of the vehicle side speaker from the recognition information of at least one of the first speech recognition means and the first image recognition means. Preferably, the first conversation content generation means generates a conversation content to the vehicle-side speaker from information including emotion estimation information by the first emotion estimation means.
(5) The vehicle includes an on-vehicle device mounted on the vehicle and operated according to a command, and the on-vehicle conversation robot extracts vehicle operation related information from conversation meaning understanding information by the first conversation meaning understanding means It is preferable to output a command according to the extracted information to the on-vehicle device.

(6) The outside-car conversation robot includes a second database storing analysis information for analyzing voice recognition information and image recognition information, and second voice receiving means for receiving the voice of the vehicle speaker. A second image acquisition unit that acquires an ambient image; a second voice recognition unit that recognizes a voice received by the voice reception unit; and a second image recognition unit that recognizes an image acquired by the second image acquisition unit. Second speaker recognition means for recognizing the vehicle speaker from the recognition information of at least one of the speech recognition information of the second speech recognition means and the image recognition information of the image recognition means and the analysis information of the second database And second conversation meaning understanding means for understanding the meaning of the conversation of the vehicle side speaker from the recognition information of the second speech recognition means and the analysis information of the second database, and the conversation of the second speech meaning understanding means Intention Second conversation content generation means for generating a conversation content to be sent back to the vehicle-side speaker from the understanding information and the analysis information of the second database; and the second conversation content generation means corresponding to the conversation content The vehicle side talk from the second speech synthesis means for synthesizing the reply speech, the second speech transmission means for transmitting the reply speech synthesized by the second speech synthesis means, and the conversation meaning comprehension information by the second conversation meaning understanding means And a second information extraction unit that extracts information to be transmitted to a person.
(7) The out-vehicle conversation robot further includes second emotion estimation means for estimating the state of emotion of the outside speaker from the recognition information of at least one of the second speech recognition means and the second image recognition means. Preferably, the second conversation content generation means generates a conversation content for the outside speaker from information including emotion estimation information by the second emotion estimation means.
(8) The facility or other vehicle is equipped with a specific device that is equipped in the facility or other vehicle and operates according to a command, and the out-of-vehicle conversation robot is a conversation according to the second conversation meaning understanding means It is preferable to extract equipment operation related information from the meaning understanding information and output a command according to the extracted information to the specific device.

According to the present invention, the conversation between the vehicle side speaker and the in-vehicle conversation robot, the conversation between the outside speaker and the outside conversation robot, and the exchange of information to be transmitted between the in-vehicle conversation robot and the outside conversation robot Thus, while the vehicle-side speaker drives the vehicle, the outside-vehicle speaker performs some work in the facility (for example, the home of the vehicle-side speaker) or while driving the vehicle in another vehicle, By having conversations with the respective conversation robots, it is possible to exchange necessary information between the vehicle-side speaker and the outside speaker through both conversation robots. Therefore, necessary information can be transmitted without interfering with the operation of the vehicle and the work in the facility.

It is a schematic block diagram of the information transmission support system for vehicles concerning one embodiment. It is a block diagram which shows the structure of the vehicle-mounted conversation robot which concerns on one Embodiment. It is a block diagram showing composition of the conversational robot outside the car concerning one embodiment. It is a flowchart which shows the initial setting of the vehicle-mounted conversation robot which concerns on one Embodiment, and a conversation robot outside a vehicle. It is a flowchart which shows the process in the vehicle-mounted conversation robot which concerns on one Embodiment. It is a flowchart which shows the process in the conversation robot outside which concerns on one Embodiment.

Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below are merely illustrative, and there is no intention to exclude the application of various modifications and techniques that are not specified in the following embodiments. The configurations of the following embodiments can be variously modified and implemented without departing from the scope of the embodiments, and can be selected as needed or can be combined as appropriate.

[1. Schematic configuration of system]
As shown in FIG. 1, the vehicular information transfer support system according to the present embodiment includes an on-vehicle conversation robot 100 mounted on a vehicle 1 and an on-vehicle conversation robot 200 equipped in a home 2 as a facility outside the vehicle. , And a cloud server (hereinafter referred to as a cloud) 300 as an external server. The vehicle 1 is an electric car.

The on-vehicle conversation robot 100 is an AI robot having artificial intelligence (hereinafter referred to as AI) using a computer, includes the first database 101, and has a conversation with a vehicle-side speaker (for example, a driver) 10 in the vehicle. Exchange information. For this reason, the on-vehicle conversation robot 100 has a function equivalent to an "ear" for listening to the voice of the vehicle side speaker 10 and a function equivalent to a "mouth" for emitting a voice to the driver 10. It is not necessary to have a part corresponding to the body as in a mold robot. However, the on-vehicle conversation robot 100 according to the present embodiment also has a function equivalent to an “eye” for observing the state of the vehicle-side speaker 10. Hereinafter, the on-vehicle conversation robot 100 is also referred to as the on-vehicle AI robot 100.

The vehicle 1 is mounted with an on-vehicle device 11 that operates according to a command from an air conditioner or a navigation device, for example. The on-vehicle AI robot 100 is also connected to the on-vehicle device 11 and can output commands to these devices. It is supposed to be. Also, the vehicle 1 is equipped with a communication device (first communication means) 12 for communicating with the outside, and the on-vehicle AI robot 100 is also connected to the communication device 12 and information is exchanged with the communication device 12 It is possible to send and receive signals.

The out-of-vehicle conversation robot 200 is also an AI robot having an AI using a computer, includes a second database 201, and exchanges information with the in-vehicle outside speaker (for example, a driver's family) 20 by exchanging a conversation. For this reason, in the outside-of-vehicle conversation robot 200, a function equivalent to the "ear" for listening to the voice of the outside speaker 20 and a function equivalent to the "mouth" for emitting the outside speaker 20 are essential. It is not necessary to have a part corresponding to the body as in a so-called humanoid robot. However, the outside-of-vehicle conversation robot 200 of the present embodiment is configured as a humanoid robot that also has a function equivalent to an “eye” that observes the appearance of the outside speaker 20. Hereinafter, the outside conversation robot 200 is also referred to as a home AI robot 200.

The home 2 is equipped with a home electric device (specific device) 21 that operates according to an instruction of an AV device such as an air conditioner or a television, for example, and the home AI robot 200 is also connected to these home electric devices 21. And can output commands to these. In addition, the home 2 is equipped with a communication device (second communication means) 22 for communicating with the outside, and the home AI robot 200 is also connected to the communication device 22 and information is exchanged with the communication device 22. It is possible to send and receive signals.

The cloud 300 is configured by a cloud computer and includes a database 301. The input information is stored in the database 301, and this information is managed while securing security. The cloud 300 is equipped with a communication device (third communication means) 32 that communicates with the outside, and can exchange information signals with a terminal connected via the Internet. This terminal includes the in-vehicle AI robot 100 and the home AI robot 200, and can transmit and receive information signals between the in-vehicle AI robot 100 and the home AI robot 200 and the cloud 300 through the

communication devices

12, 22, and 32. ing. In particular, a huge amount of big data is stored in the database 301, and each terminal can extract necessary data from the big data in the database 301 to perform each process.

[2. Schematic configuration of in-vehicle AI robot]
As shown in FIG. 2, the in-vehicle AI robot 100 includes an in-vehicle HMI (Human Machine Interface) unit 110 that is an interface unit with the vehicle-side speaker 10 and input information from the in-vehicle HMI unit 110 and the in-vehicle HMI unit 110. The in-vehicle HMI processing unit 120 processes output information, and the in-vehicle AI control unit 130 recognizes and processes information processed by the in-vehicle HMI processing unit 120 and information from the home AI robot 200 obtained by communication from the cloud 300. And an on-board AI assist unit 140 that complements the processing of the on-vehicle AI control unit 130, and the database 101.

The on-vehicle HMI unit 110 includes a microphone (first voice receiving means) 111 for receiving the voice of the vehicle side speaker 10, and a speaker (first voice sending means) 112 for transmitting voice to the vehicle side speaker 10. An in-vehicle camera (first image acquisition means) 113 for acquiring an ambient image including the vehicle side speaker 10, and a sensor 114 for detecting a vital sign of a driving vehicle and a vehicle environment (air pressure of a wheel or remaining amount of battery) Have. Information detected or acquired by the microphone 111, the speaker 112, the in-vehicle camera 113, and the sensor 114 is sent to the on-vehicle HMI processing unit 120 and the on-vehicle AI assist unit 140.

The on-vehicle HMI processing unit 120 includes a voice recognition unit (first voice recognition unit) 121 that recognizes voice received by the microphone 111, and a voice synthesis unit (first voice synthesis unit) 122 that combines the voice transmitted from the speaker 112. And an image recognition unit (first image recognition means) 123 for recognizing an ambient image acquired by the in-vehicle camera 113. In addition, information from the in-vehicle AI assist unit 140 is also appropriately used for voice recognition and image recognition. Information from the in-vehicle AI control unit 130 is used for speech synthesis, and information from the in-vehicle AI assist unit 140 is also used as appropriate.

The in-vehicle AI control unit 130 identifies a speaker from the recognition result by the speech recognition unit 121 and the image recognition unit 123 and recognizes a speaker, and a speaker recognition unit (first speaker recognition means) 131, a speech recognition unit 121, A conversation meaning understanding unit (first conversation meaning understanding means) 132 that understands the meaning of the speaker's conversation by referring to a word map stored in advance from the recognition result by the image recognition unit 123, a speech recognition unit 121, and an image recognition unit A personal emotion estimation unit (first personal emotion estimation means) 133 that estimates the emotional state of the speaker with reference to the emotion map stored in advance from the recognition result by 123 and a conversation with the vehicle speaker 10 And an information extraction unit (first information extraction means) 134 for extracting information to be transmitted to the person 20. Also, information from the in-vehicle AI assist unit 140 is appropriately used for speaker recognition, conversation meaning understanding, individual emotion estimation, and information extraction.

The in-vehicle AI assist unit 140 appropriately accesses the database 101 and the cloud 300 to obtain information necessary for speech recognition, image recognition and speech synthesis from the database 101 and the database (big data) 301, and performs speech recognition and image recognition. And analysis and evaluation for speech synthesis. Further, the information exchange is also performed with the in-vehicle AI control unit 130 to assist speaker recognition, conversation meaning understanding, and personal emotion estimation by the in-vehicle AI control unit 130. Further, the in-vehicle AI assist unit 140 outputs a command signal to the in-vehicle device.

In the database 101, for example, personal attribute data such as data on the driver and his family, communication attribute data necessary for communication in the communication unit, vehicle attribute data on the vehicle 1, and registered specific events are managed. For example, event management data for scheduling, scheduling data such as a schedule of a driver or a family member thereof, vital data such as a health condition of the driver, and the like are stored.
The communication unit 12 includes a communication unit 12A, and the communication unit 12A performs authentication of received data and encryption of launch data.

[3. Schematic Configuration of Home AI Robot]
As shown in FIG. 3, the home AI robot 200 processes the home HMI unit 210, which is an interface unit with the outside speaker 20, and input information from the home HMI unit 210 and output information to the home HMI unit 210. Home AI control unit 230 that recognizes and understands information processed by the home HMI processing unit 220, information processed by the home HMI processing unit 220, and information obtained from the cloud 300 by communication from the on-vehicle AI robot 100, and home AI control A home AI assist unit 240 that complements the processing of the unit 230 and a database 201 are provided.

The home HMI unit 210 has a microphone (second voice receiving means) 211 for receiving the voice of the outside speaker 20 and a speaker (second voice transmitting means) 212 for sending out voice toward the outside speaker 20; A home camera (second image acquisition means) 213 for acquiring an ambient image including the vehicle outside speaker 20, and a sensor 214 for detecting the security state of the home, the air conditioning, and the illumination state are provided. Information detected or acquired by the microphone 211, the speaker 212, the in-vehicle camera 213, and the sensor 214 is sent to the home HMI processing unit 220 and the home AI assist unit 240.

The home HMI processing unit 220 includes a voice recognition unit (second voice recognition unit) 221 that recognizes voice received by the microphone 211, and a voice synthesis unit (second voice synthesis unit) 222 that combines the voice transmitted from the speaker 212. And an image recognition unit (second image recognition means) 223 for recognizing an ambient image acquired by the home camera 213. Further, information from the home AI assist unit 240 is also appropriately used for voice recognition and image recognition. Information from the home AI control unit 230 is used for speech synthesis, and information from the home AI assist unit 240 is also used as appropriate.

The home AI control unit 230 identifies a speaker from the recognition results by the speech recognition unit 221 and the image recognition unit 223, and recognizes a speaker, and a speaker recognition unit (second speaker recognition means) 231 for recognizing the attribute; A conversation meaning understanding unit (second conversation meaning understanding means) 232 that understands the meaning of the speaker's conversation by referring to a word map stored in advance from the recognition result by the image recognition unit 223, a voice recognition unit 221, and an image recognition unit A personal emotion estimation unit (second individual emotion estimation means) 233 for estimating the emotional state of the speaker with reference to the emotion map stored in advance from the recognition result by 223, and a conversation with the vehicle outside speaker 20 from the side talk of the vehicle And an information extraction unit (second information extraction means) 234 for extracting information to be transmitted to the person 10. In addition, information from the home AI assist unit 240 is also appropriately used for speaker recognition, conversation meaning understanding, individual emotion estimation, and information extraction.

The home AI assist unit 240 appropriately accesses the database 201 and the cloud 300 to obtain information necessary for speech recognition, image recognition and speech synthesis from the database 201 and the database (big data) 301, and performs speech recognition and image recognition. And analysis and evaluation for speech synthesis. Further, the information exchange is also performed with the home AI control unit 230 to assist the speaker recognition by the home AI control unit 230, the conversation meaning comprehension, and the personal emotion estimation. In addition, the home AI assist unit 240 outputs a command signal to the in-vehicle device.

Similar to the database 101, the database 201 includes, for example, personal attribute data such as data relating to the driver and his family, communication attribute data necessary for communication in the communication unit, vehicle attribute data For example, event management data for managing a specific event, scheduling data such as a schedule of a driver or his family, vital data such as a driver's health state, etc. are stored.
Further, the communication unit 22 includes a communication unit 22A, and the communication unit 22A performs authentication of received data and encryption of start data.

[4. Initial setting]
The in-vehicle AI robot 100 and the home AI robot 200 are initially initialized (also referred to as initialization), and this initialization is performed, for example, as shown in the flowchart of FIG. Note that this initial setting may be performed periodically, for example, every day, or may be performed as needed.

As shown in FIG. 4, first, the vehicle-side personal attribute data is initialized on the side of the in-vehicle AI robot 100 (step S110). For example, face recognition and facial expression recognition of the driver and his family are performed and registered as vehicle-side personal attribute data. Similarly, initialization of the vehicle-side personal attribute data is also performed on the home AI robot 200 side (step S130). For example, face recognition and facial expression recognition of the driver and his family are performed and registered as vehicle-side personal attribute data. Thus, personal attribute data is shared between the vehicle 1 side and the home 2 side.

Next, on the on-vehicle AI robot 100 side, initialization of the vehicle-side communication attribute data is performed (step S112). For example, the setting of the incoming call address and the setting of the authentication method are performed. Similarly, home-side communication attribute data is also initialized on the home AI robot 200 side (step S132). For example, the setting of the incoming call address and the setting of the authentication method are performed. These communication attribute data are also shared between the vehicle 1 side and the home 2 side.

Next, the vehicle attribute data is initialized on the side of the in-vehicle AI robot 100 (step S114). For example, vehicle information such as a state of tire air pressure of the vehicle 1 and information of a battery residual amount (SOC) are registered. These pieces of information are sent from the vehicle side to the home side by information synchronization, and the home-side vehicle attribute data is also initialized on the home AI robot 200 side (step S134). Thus, the vehicle attribute data is shared between the vehicle 1 and the home 2.

Next, on the on-vehicle AI robot 100 side, initialization of the vehicle-side event management data is performed (step S116). For example, event events (normal and emergency) of the vehicle 1 and operations corresponding to the events are registered. At the home AI robot 200 side, home-side event management data is initialized (step S136). For example, an event event at home 2 (normal and emergency) and an operation corresponding to the event are registered. These event management data are also shared between the vehicle 1 side and the home 2 side by information synchronization. The event event of the vehicle 1 includes various event events that occur in relation to the traveling and driving of the vehicle. Also, the event events at home 2 include various event events that occur in the family life.

Next, on the on-vehicle AI robot 100 side, initialization of the vehicle-side schedule data is performed (step S118). For example, behavior data of a vehicle occupant (driver or the like) is registered. At the home AI robot 200 side, home-side schedule data is initialized (step S138). For example, behavior data of a home-side participant (family, etc.) is registered. These schedule data are also shared between the vehicle 1 side and the home 2 side by information synchronization.

Next, on the on-vehicle AI robot 100 side, initialization of vehicle-side vital data is performed (step S120). For example, vital data of the vehicle occupant (driver or passenger) is registered. These pieces of information are sent from the vehicle to the home by information synchronization, and initialization of the vehicle vital data is performed on the home AI robot 200 (step S140). Thus, the vehicle-side vital data is shared between the vehicle 1 and the home 2.

[5. Processing of in-vehicle AI robot]
In the on-vehicle AI robot 100, processing is performed as shown in the flowchart of FIG.
First, the initialization shown in FIG. 4 is performed (step S100). Next, in the standby state (step S200), voice conversation input (step S210), camera image input (step S220), sensor data input (step S230), data input via the cloud (step S240) If there is, each processing is performed.

When there is a speech conversation input, speech recognition is performed (step S212), speaker recognition is performed (step S214), and the meaning of speech is understood (step S216).
If there is a camera image input, image recognition is performed (step S222), and understanding of the emotion of the driver etc. is performed (step S224).

When the sensor data is input, the driver or the like understands the vital sign, and the vehicle environment such as the air pressure or the remaining battery amount (step S232).
When there is data input via the cloud, message understanding (step S244) is performed via message input (step S242).

After such conversation meaning comprehension, emotion comprehension, vital sign comprehension, vehicle environment comprehension comprehension, and message comprehension, processing for selecting information corresponding to AI, response expression, etc. is performed (step S300).
When the corresponding voice response is selected based on the processing result (step S310), voice synthesis is performed (step S312), and spontaneous speech output is performed (step S314).
In addition, when the control of the on-vehicle device such as the on-vehicle air conditioner is selected (step S320), the on-vehicle device is instructed to control (step S322).

Further, when control of the navigation device is selected (step S330), for example, search for service facilities and display thereof (step S332), route search, prediction of arrival time and display thereof, and service reservation are performed (step S334). ).
When access to the database 101 is selected (step S340), the database 101 is accessed to update data, and the information is synchronized (step S342).
When connection to the home AI robot 200 is selected (step S350), a message is transmitted via the cloud (step S352).
In addition, in FIG. 5, the flows described in parallel can be performed simultaneously and progressive, and return returns to the standby state in step S200. However, when the initial setting is updated, the process returns to step S100.

Therefore, the on-vehicle AI robot 100 can extract information to be transmitted to the on-vehicle speaker 20 from the conversation with the on-vehicle speaker 10, and can transmit the information to the home AI robot 200 by communication. Further, the on-vehicle AI robot 100 can receive the extraction information transmitted from the home AI robot 200 by communication, and can transmit the extraction information to the vehicle speaker 10 by conversation. Further, the on-vehicle AI robot 100 can generate conversation contents using information including emotion estimation information of the vehicle speaker 10.

[6. Processing of home AI robot]
In the home AI robot 200, processing is performed as shown in the flowchart of FIG.
First, the initialization shown in FIG. 4 described above is performed (step S400), and then the standby state (step S500) is entered, and voice conversation input (step S510), camera image input (step S520), and sensors If there is data input (step S530) or data input via the cloud (step S540), the respective processing is performed.

If there is voice conversation input, voice recognition is performed (step S512), speaker recognition is performed (step S514), and the meaning of the conversation is understood (step S516).
When there is a camera image input, image recognition is performed (step S522), and understanding of the emotion of the driver etc. is performed (step S524).
When sensor data is input, security data, air conditioning, and illumination data are recognized (step S532).
When there is data input via the cloud, conversation input (step S542), image input (step S544), and sensor data input from the vehicle side (step S546) are performed.

After such conversation meaning comprehension, emotion comprehension, vital sign comprehension, vehicle environment comprehension, and data input, processing for selecting information corresponding to information extraction or reaction expression by AI is performed (step S600).
When the corresponding voice response is selected based on the processing result (step S610), voice synthesis is performed (step S612), and spontaneous speech output is performed (step S614).
Also, when control of the home appliance is selected (step S620), the home appliance is instructed to control (step S622).

When access to the database 101 is selected (step S630), the database 101 is accessed to update data, and information is synchronized (step S632).
When connection to the in-vehicle AI robot 100 is selected (step S640), a message is transmitted via the cloud (step S642).
Also in FIG. 6, the flows described in parallel can be performed simultaneously, and the process returns from the return to the standby state in step S500. Also in this case, when the initial setting is updated, the process returns to step S400.

Therefore, the home AI robot 200 can extract information to be transmitted to the in-vehicle speaker 10 from the conversation with the in-vehicle speaker 20 and transmit the information to the in-vehicle AI robot 100 by communication. In addition, the home AI robot 200 can receive the extracted information transmitted from the on-vehicle AI robot 100 by communication, and can transmit the extracted information to the on-vehicle speaker 20 by conversation. In addition, the home AI robot 200 can generate conversation contents using information including emotion estimation information of the outside speaker 20.

The in-vehicle AI robot 100 and the home AI robot 200 perform information synchronization through the cloud 300 to share information, and each AI robot 100 accesses the cloud 300 at a predetermined cycle to perform in-vehicle AI. The robot 100 acquires information from the home AI robot 200, and the home AI robot 200 acquires information from the on-board AI robot 100 in substantially real time.

[7. Action and effect]
According to the present embodiment, since it is configured as described above, on the vehicle side, the driver who is the vehicle-side speaker talks with the on-vehicle AI robot 100, and on the vehicle outside, it is the outside speaker. The family and the home AI robot 200 have a conversation, and each

AI robot

100, 200 extracts the information to be transmitted between the vehicle side and the home side from each conversation, and the extracted information is stored in the cloud 300, respectively. send.

When information is sent to the cloud 300, the in-vehicle AI robot 100 can acquire information from the home AI robot 200, and the home AI robot 200 can acquire information from the in-vehicle AI robot 100 in substantially real time. The driver on the side 1 and the family on the side 2 at home can exchange information in real time through the

AI robot

100, 200. The driver on the vehicle 1 side and the family on the home 2 side only talk with the

AI robots

100 and 200, so the driver does not disturb the driving of the vehicle and the family disturbs the work such as housework. Can communicate the necessary information.

Specifically, for example, the following information can be transmitted.
For example, when the home AI robot 200 grasps the condition of the home and the contents of conversation and sends it to the in-vehicle AI robot 100, the in-vehicle AI robot 100 determines whether transmission of the received information is unnecessary or not. The driver can be notified of information by selecting an appropriate transmission timing while observing the situation of

The on-vehicle AI robot 100 can also instruct the navigation system the route of the vehicle 1 from the conversation with the driver 10, calculate the fastest route and arrival time from various information, and send it to the home AI robot 200. The home AI robot 200 can determine the necessity of the received information and can transmit necessary information to a home person at an appropriate timing.

The on-vehicle AI robot 100 can also estimate the information on the state of the vehicle and the time until restoration when a vehicle failure occurs, and can send it to the home AI robot 200. Necessary information can be determined, and necessary information can be transmitted to householders at appropriate times.
The home AI robot 200 can grasp the state of the home and conversation contents and send it to the on-vehicle AI robot 100. The on-vehicle AI robot 100 determines the necessity of the received information and determines the necessary information at an appropriate timing. Can be transmitted to the driver.

Also, the home AI robot 200 searches for a route to a predetermined destination and transmits it to the on-vehicle AI robot 100, and the on-vehicle AI robot 10 adds more information based on the received information, and an optimal route is obtained. It can also be set.
Identify the number and attributes of the in-vehicle AI robot 10, send it to the home AI robot 200, and judge whether the information received by the home AI robot 200 is necessary or unnecessary, and transmit it to the home person at an appropriate timing. You can also.

In addition, the home AI robot 200 can grasp sleep time and health status through, for example, a wearable device, and can transmit it to the on-board AI robot 100. The on-board AI robot 10 is in the state of the driver sent from the home AI robot 200 Based on the analysis information, support can be provided, such as changing the setting of the safety function and recommending break points.

When the in-vehicle AI robot 100 transmits the information such as the in-vehicle air-conditioning temperature setting, the result of judgment whether the driver is hot or cold, and the distance / time to the home to the home AI robot 200, the home AI robot 200 transmits the transmitted information. Also, based on the behavior pattern of the target person, it is possible to set the temperature of the home air conditioning, the temperature of the bath, the hot water supply timing, and the like.

[8. Other]
Although the embodiment has been described above, in the embodiment, a home as a facility outside the vehicle is illustrated, but the facility outside the vehicle may be an office or the like.
In the embodiment, although the cloud 300 is used, the present invention is not limited to this, and other media may be used, or direct communication may be performed between the vehicle side and the outside of the vehicle such as at home.
In the above embodiment, the humanoid robot is adopted only for the home AI robot 200. However, a humanoid robot may be adopted for the on-vehicle AI robot 100.
Alternatively, a functional robot equipped with only necessary functions may be adopted as neither the in-vehicle AI robot 100 nor the home AI robot 200, without adopting a humanoid robot.
In the embodiment, an example in which necessary information is exchanged between the vehicle side speaker and the vehicle outside speaker between the vehicle and the facility outside the vehicle is shown, but between the vehicle and another vehicle In this case, necessary information may be exchanged between the vehicle-side speaker and the vehicle-side speaker (the occupant of another vehicle, for example, a driver).

1 vehicle 10 vehicle side speaker (eg driver)
11 in-vehicle device 12 communication device (first communication means)
2 home 20 car outside speakers (eg driver's family)
21 Household appliances (specific equipment)
22 Communication device (second communication means)
32 Communication Device (Third Communication Means)
100 In-vehicle AI robot (In-vehicle conversation robot)
101 first database 110 in-vehicle HMI unit 111 microphone (first voice receiving means)
112 Speaker (1st voice transmission means)
113 In-vehicle camera (first image acquisition means)
120 in-vehicle HMI processing unit 121 speech recognition unit (first speech recognition means)
122 Speech synthesis unit (first speech synthesis means)
123 Image recognition unit (first image recognition means)
130 in-vehicle AI control unit 131 speaker recognition unit (first speaker recognition means)
132 Conversation Meaning Understanding Unit (First Conversation Meaning Understanding Means)
133 Personal Emotion Estimator (First Personal Emotion Estimator)
134 Information Extraction Unit (First Information Extraction Unit)
140 In-vehicle AI assist unit 200 Home AI robot (outside conversation robot)
201 second database 210 home HMI unit 211 microphone (second voice receiving means)
212 Speaker (second voice transmission means)
213 Home camera (second image acquisition means)
220 home HMI processing unit 221 voice recognition unit (second voice recognition means)
222 Speech synthesis unit (second speech synthesis means)
223 Image recognition unit (second image recognition means)
230 home AI control unit 231 speaker recognition unit (second speaker recognition means)
232 Conversational Meaning Understanding Unit (2nd Conversational Meaning Understanding Means)
233 Personal Emotion Estimator (Second Personal Emotion Estimator)
234 Information Extraction Unit (Second Information Extraction Unit)
240 Home AI Assist Unit 300 Cloud Server as an External Server (Cloud)
301 database

Claims

An on-vehicle conversation robot equipped in a vehicle, having artificial intelligence, and exchanging information with the vehicle-side speaker in the vehicle;
First communication means connected to the on-board conversation robot;
An out-of-vehicle conversation robot equipped in a facility or other vehicle outside the vehicle, having artificial intelligence, and exchanging information with a speaker outside the vehicle in the facility or the other vehicle;
And a second communication unit connected to the outside robot.
The on-vehicle conversation robot extracts information to be transmitted to the on-vehicle speaker from the conversation with the on-vehicle speaker, and transmits the information to the on-vehicle conversation robot by the first communication unit. The extraction information transmitted from the on-vehicle conversation robot side is received by the second communication means, and the extraction information is transmitted to the outside speaker of the vehicle by conversation.
The on-vehicle conversation robot extracts information to be transmitted to the on-vehicle speaker from the conversation with the on-vehicle speaker, and transmits the information to the on-vehicle conversation robot by the second communication unit. An information transmission support system for a vehicle, characterized in that the extraction information transmitted from the out-vehicle conversation robot side is received by the first communication means, and the extraction information is transmitted to the vehicle speaker by conversation.
Third communication means for communicating with the outside, the extraction information from the on-board conversation robot side obtained by communication by the third communication means, and the outside-car conversation robot side obtained by communication by the third communication means And a database storing the extracted information from
The information transmission support system for a vehicle according to claim 1, wherein information transmission between the on-vehicle conversation robot and the outside-of-vehicle conversation robot is performed via the external server.
The on-board conversation robot is
A first database storing analysis information for analysis of voice recognition information and analysis of image recognition information;
First voice receiving means for receiving the voice of the vehicle-side speaker;
First image acquisition means for acquiring an ambient image;
First voice recognition means for recognizing voice received by the voice receiving means;
A first image recognition unit that recognizes an image acquired by the first image acquisition unit;
First speaker recognition means for recognizing the vehicle-side speaker from at least one of recognition information of the voice recognition information of the first speech recognition means and image recognition information of the image recognition means and analysis information of the first database When,
First conversation meaning understanding means for understanding the meaning of the conversation of the vehicle side speaker from the recognition information of the first speech recognition means and the analysis information of the first database;
First conversation content generation means for generating a conversation content to be sent back to the vehicle side speaker from the conversation meaning understanding information of the first conversation meaning understanding means and the analysis information of the first database;
First voice synthesizing means for synthesizing a reply voice corresponding to the conversation contents generated by the first conversation contents generating means;
First voice transmitting means for transmitting a reply voice synthesized by the first voice synthesizing means;
The first information extraction means for extracting information to be transmitted to the outside speaker from the conversation meaning comprehension information by the first conversation meaning comprehension means, according to claim 1 or 2, Information transmission support system for vehicles.
The on-board conversation robot is
The system further comprises first emotion estimation means for estimating the state of emotion of the vehicle speaker from the recognition information of at least one of the first speech recognition means and the first image recognition means,
The vehicle information according to claim 3, wherein the first conversation content generation means generates the conversation content to the vehicle side speaker from the information including the emotion estimation information by the first emotion estimation means. Transmission support system.
The vehicle is equipped with on-vehicle equipment mounted on the vehicle and operated in accordance with a command.
The on-board conversation robot is
The vehicle according to claim 3 or 4, characterized in that a command according to information extracted by extracting vehicle operation related information from conversation meaning understanding information by the first conversation meaning understanding means is output to the on-vehicle device. Information transmission support system.
The outside conversation robot is
A second database storing analysis information for analysis of voice recognition information and analysis of image recognition information;
Second voice receiving means for receiving the voice of the vehicle-side speaker;
Second image acquisition means for acquiring an ambient image;
Second voice recognition means for recognizing voice received by the voice receiving means;
A second image recognition unit that recognizes an image acquired by the second image acquisition unit;
Second speaker recognition means for recognizing the vehicle speaker from the recognition information of at least one of the speech recognition information of the second speech recognition means and the image recognition information of the image recognition means and the analysis information of the second database When,
Second conversation meaning understanding means for understanding the meaning of the conversation of the vehicle speaker from the recognition information of the second speech recognition means and the analysis information of the second database;
Second conversation content generation means for generating a conversation content to be sent back to the vehicle side speaker from the conversation meaning understanding information of the second conversation meaning understanding means and the analysis information of the second database;
Second voice synthesis means for synthesizing a reply voice corresponding to the conversation contents generated by the second conversation contents generation means;
Second voice transmitting means for transmitting a reply voice synthesized by the second voice synthesizing means;
The second information extraction means for extracting information to be transmitted to the vehicle side speaker from the conversation meaning understanding information by the second conversation meaning understanding means, The method according to any one of claims 3 to 5, further comprising: The information transmission support system for vehicles of item 1.
The outside conversation robot is
The system further comprises second emotion estimation means for estimating the state of the emotion of the outside speaker from the recognition information of at least one of the second speech recognition means and the second image recognition means,
7. The vehicle information according to claim 6, wherein the second conversation content generation means generates the conversation content to the outside speaker from the information including the emotion estimation information by the second emotion estimation means. Transmission support system.
The facility or the other vehicle is equipped in the facility or in the other vehicle, and is equipped with a specific device that operates in response to a command.
The outside conversation robot is
8. The vehicle according to claim 6, wherein a command corresponding to information extracted from facility operation related information from the conversation meaning understanding information by the second conversation meaning understanding means is output to the specific device. Communication support system.