JP2020144618A - Agent device, control method of agent device, and program - Google Patents

Abstract

To provide an agent device, a control method of the agent device, and a program that allow an occupant to easily use a new function.SOLUTION: An agent device (100) comprises: a plurality of agent function units (150-1 to 150-3) for providing a service including a voice response to speech of an occupant of a vehicle; and a selection unit (122) for selecting an agent function unit corresponding to the speech of the occupant from among the plurality of agent function units. When a new function is added to one agent function unit of the plurality of agent function units and the newly added function is provided for the occupant, the selection unit causes a function of the agent function unit having the newly added function to be provided for the occupant preferentially over the other agent function units already having the same function as the newly added function.SELECTED DRAWING: Figure 2

Description

The present invention relates to an agent device, a control method for the agent device, and a program.

Conventionally, a technique relating to an agent that provides information on driving support according to a request of a occupant, control of a vehicle, other applications, and the like while interacting with a vehicle occupant has been disclosed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2006-335231

In recent years, practical application has been promoted for mounting a plurality of agents on a vehicle. In addition, the functions that can be executed by the agent may be updated sequentially. However, even if a new function is added to an agent, if there is another agent that can execute the new function for some time, the occupant may be made to execute the new function by the agent to which the new function is added. It could be difficult.

The present invention has been made in consideration of such circumstances, and one of the objects of the present invention is to provide an agent device, a control method of the agent device, and a program capable of making new functions easier for occupants to use. To do.

The agent device, the control method of the agent device, and the program according to the present invention have adopted the following configurations.
(1): The agent device according to one aspect of the present invention includes a plurality of agent function units that provide a service including a voice response in response to an utterance of a vehicle occupant, and among the plurality of agent function units. The selection unit includes a selection unit that selects an agent function unit corresponding to the utterance of the occupant, and the selection unit is the new function unit when a new function is added to one agent function unit among the plurality of agent function units. When the function added to is provided to the occupant, the agent to which the new function is added has priority over other agent function units that already have the same function as the newly added function. The function of the functional unit is provided to the occupant.

(2): The agent device according to another aspect of the present invention includes a plurality of agent function units that provide a service including a voice response in response to an utterance of a vehicle occupant, and among the plurality of agent function units. The plurality of agent function units include a vehicle agent function unit that has a function of instructing the vehicle equipment to operate, and the selection unit includes a selection unit that selects an agent function unit corresponding to the utterance of the occupant. When a new function is added to the vehicle agent function unit among the plurality of agent function units and the newly added function is provided to the occupant, the same as the newly added function. The occupant is made to provide the function of the vehicle agent function unit to which the new function is added, preferentially to the other agent function unit that already has the function.

(3): In the embodiment (1) or (2) above, the selection unit is newly added even if the question specifies a specific agent function unit among the plurality of agent function units. When the function is provided to the occupant, the function by the agent function unit to which the new function has been added has priority over other agent function units that already have the same function as the newly added function. Is provided to the occupants.

(4): In any of the above aspects (1) to (3), when a new function is added to at least one agent function unit among the plurality of agent function units, the agent function unit It provides the occupant with information about the newly added function in response to an inquiry that does not specify the details of the new function.

(5): In any of the above aspects (1) to (4), when a new function is added to at least one agent function unit among the plurality of agent function units, the agent function unit It provides the occupant with information about the newly added function when making a response unrelated to the new function.

(6): In the control method of the agent device according to another aspect of the present invention, a computer activates one of a plurality of agent function units, and as a function of the activated agent function unit, a vehicle occupant speaks. Depending on the situation, a service including a voice response is provided, an agent function unit corresponding to the utterance of the occupant is selected from the plurality of agent function units, and one agent function among the plurality of agent function units is used. When a new function is added to the unit, when the newly added function is provided to the occupant, the other agent function unit that already has the same function as the newly added function Preferentially, the occupant is provided with a function by the agent function unit to which the new function is added.

(7): The program according to another aspect of the present invention causes a computer to activate one of a plurality of agent function units, and as a function of the activated agent function unit, responds to a voice of a vehicle occupant. A service including a voice response is provided, an agent function unit corresponding to the utterance of the occupant is selected from the plurality of agent function units, and one agent function unit among the plurality of agent function units is newly used. When a function is added, when the newly added function is provided to the occupant, priority is given to other agent function units that already have the same function as the newly added function. , The occupant is provided with a function by the agent function unit to which the new function is added.

According to the aspects (1) to (7), the new function can be easily used by the user.

It is a block diagram of the agent system 1 including the agent apparatus 100. It is a figure which shows the structure of the agent apparatus 100 which concerns on 1st Embodiment, and the apparatus mounted on the vehicle M. It is a figure which shows the arrangement example of the display / operation apparatus 20. It is a figure which shows the arrangement example of a speaker unit 30. It is a figure which shows an example of the content of the function list information 162. It is a figure for demonstrating the principle that the position where a sound image is localized is determined. It is a figure which shows the configuration of the agent server 200, and a part of the configuration of the agent apparatus 100. It is a figure which shows an example of the dialogue between an agent and an occupant when providing a map search function. It is a figure which shows an example of the agent's answer to the utterance CV3 including a wake-up word. It is a flowchart which shows the series flow of the operation of the agent apparatus 100. It is a flowchart which shows the series flow of the operation of the agent apparatus 100 when the agent function part 150 is prioritized. It is a figure which shows an example of the interaction between an agent and an occupant when providing information about a newly added function. It is a flowchart which shows the series flow of the process which introduces the unexecuted function of the agent apparatus 100.

Hereinafter, the agent device of the present invention, the control method of the agent device, and the embodiment of the program will be described with reference to the drawings. An agent device is a device that realizes a part or all of an agent system. Hereinafter, as an example of the agent device, an agent device mounted on a vehicle (hereinafter referred to as a vehicle M) and having a plurality of types of agent functions will be described. The agent function is, for example, a function of providing various information based on a request (command) included in the utterance of the occupant or mediating a network service while interacting with the occupant of the vehicle M. The functions, processing procedures, controls, output modes and contents of the plurality of types of agents may be different from each other. In addition, some of the agent functions may have a function of controlling equipment in the vehicle (for example, equipment related to driving control and vehicle body control).

Agent functions include, for example, a voice recognition function that recognizes the voice of an occupant (a function that converts voice into text), a natural language processing function (a function that understands the structure and meaning of text), a dialogue management function, and a network. It is realized by integratedly using a network search function or the like that searches for another device or a predetermined database owned by the own device. Some or all of these functions may be realized by AI (Artificial Intelligence) technology. In addition, a part of the configuration for performing these functions (particularly, the voice recognition function and the natural language processing interpretation function) is an agent server capable of communicating with the in-vehicle communication device of the vehicle M or the general-purpose communication device brought into the vehicle M. It may be mounted on (external device). In the following description, it is assumed that a part of the configuration is installed in the agent server, and the agent device and the agent server cooperate to realize the agent system. Further, a service provider (service entity) in which an agent device and an agent server cooperate to appear virtually is called an agent.

<Overall configuration>
FIG. 1 is a configuration diagram of an agent system 1 including an agent device 100. The agent system 1 includes, for example, an agent device 100 and a plurality of agent servers 200-1, 200-2, 200-3, .... The number after the hyphen at the end of the code shall be an identifier for distinguishing agents. When it is not distinguished which agent server it is, it may be simply referred to as an agent server 200. Although three agent servers 200 are shown in FIG. 1, the number of agent servers 200 may be two or four or more. Each agent server 200 is operated by a provider of agent systems different from each other. Therefore, the agents in the present invention are agents realized by different providers. Examples of providers include automobile manufacturers, network service providers, e-commerce businesses, mobile terminal sellers and manufacturers, and any entity (corporation, group, individual, etc.) is the provider of the agent system. Can be.

The agent device 100 communicates with the agent server 200 via the network NW. The network NW includes, for example, a part or all of the Internet, a cellular network, a Wi-Fi network, a WAN (Wide Area Network), a LAN (Local Area Network), a public line, a telephone line, a wireless base station, and the like. Various web servers 300 are connected to the network NW, and the agent server 200 or the agent device 100 can acquire web pages from the various web servers 300 via the network NW.

The agent device 100 interacts with the occupant of the vehicle M, transmits the voice from the occupant to the agent server 200, and presents the answer obtained from the agent server 200 to the occupant in the form of voice output or image display.

<First Embodiment>
[vehicle]
FIG. 2 is a diagram showing the configuration of the agent device 100 according to the first embodiment and the equipment mounted on the vehicle M. The vehicle M includes, for example, one or more microphones 10, a display / operation device 20, a speaker unit 30, a navigation device 40, a vehicle device 50, an in-vehicle communication device 60, an occupant recognition device 80, and an agent device. 100 and are installed. Further, a general-purpose communication device 70 such as a smartphone may be brought into the vehicle interior and used as a communication device. These devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 2 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The microphone 10 is a sound collecting unit that collects sounds emitted in the vehicle interior. The display / operation device 20 is a device (or device group) capable of displaying an image and accepting an input operation. The display / operation device 20 includes, for example, a display device configured as a touch panel. The display / operation device 20 may further include a HUD (Head Up Display) or a mechanical input device. The speaker unit 30 includes, for example, a plurality of speakers (sound output units) arranged at different positions in the vehicle interior. The display / operation device 20 may be shared by the agent device 100 and the navigation device 40. Details of these will be described later.

The navigation device 40 includes a navigation HMI (Human machine Interface), a positioning device such as a GPS (Global Positioning System), a storage device that stores map information, and a control device (navigation controller) that performs route search and the like. .. A part or all of the microphone 10, the display / operation device 20, and the speaker unit 30 may be used as the navigation HMI. The navigation device 40 searches for a route (navigation route) for moving from the position of the vehicle M specified by the positioning device to the destination input by the occupant, so that the vehicle M can travel along the route. , Navi HMI is used to output guidance information. The route search function may be provided in a navigation server accessible via the network NW. In this case, the navigation device 40 acquires a route from the navigation server and outputs guidance information. The agent device 100 may be constructed based on the navigation controller. In that case, the navigation controller and the agent device 100 are integrally configured on the hardware.

The vehicle equipment 50 includes, for example, a driving force output device such as an engine or a traveling motor, an engine start motor, a door lock device, a door opening / closing device, a window, a window opening / closing device, a window opening / closing control device, a seat, and a seat position. Control device, room mirror and its angle position control device, lighting device inside and outside the vehicle and its control device, wiper and defogger and their respective control devices, direction indicator and its control device, air conditioner, mileage information and vehicle position information It includes a vehicle information device that manages information about the vehicle such as tire pressure information and fuel remaining amount information.

The in-vehicle communication device 60 is, for example, a wireless communication device that can access the network NW using a cellular network or a Wi-Fi network.

The occupant recognition device 80 includes, for example, a seating sensor, a vehicle interior camera, an image recognition device, and the like. The seating sensor includes a pressure sensor provided at the bottom of the seat, a tension sensor attached to the seat belt, and the like. The vehicle interior camera is a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera installed in the vehicle interior. The image recognition device analyzes the image of the vehicle interior camera and recognizes the presence or absence of a occupant for each seat, the face orientation, and the like. In the present embodiment, the occupant recognition device 80 is an example of the seating position recognition unit.

FIG. 3 is a diagram showing an arrangement example of the display / operation device 20. The display / operation device 20 includes, for example, a first display 22, a second display 24, and an operation switch ASSY 26. The display / operation device 20 may further include a HUD 28.

The vehicle M includes, for example, a driver's seat DS provided with a steering wheel SW and a passenger seat AS provided in the vehicle width direction (Y direction in the drawing) with respect to the driver's seat DS. The first display 22 is a horizontally long display device extending from an intermediate portion between the driver's seat DS and the passenger's seat AS on the instrument panel to a position facing the left end of the passenger's seat AS. The second display 24 is installed at the middle of the driver's seat DS and the passenger's seat AS in the vehicle width direction and below the first display. For example, both the first display 22 and the second display 24 are configured as a touch panel, and include an LCD (Liquid Crystal Display), an organic EL (Electroluminescence), a plasma display, and the like as display units. The operation switch ASSY26 is a combination of dial switches, button-type switches, and the like. The display / operation device 20 outputs the content of the operation performed by the occupant to the agent device 100. The content displayed by the first display 22 or the second display 24 may be determined by the agent device 100.

FIG. 4 is a diagram showing an arrangement example of the speaker unit 30. The speaker unit 30 includes, for example, speakers 30A to 30H. The speaker 30A is installed on a window pillar (so-called A pillar) on the driver's seat DS side. The speaker 30B is installed under the door near the driver's seat DS. The speaker 30C is installed on the window pillar on the passenger seat AS side. The speaker 30D is installed at the bottom of the door near the passenger seat AS. The speaker 30E is installed at the lower part of the door near the right rear seat BS1 side. The speaker 30F is installed at the lower part of the door near the left rear seat BS2 side. The speaker 30G is installed in the vicinity of the second display 24. The speaker 30H is installed on the ceiling (roof) of the vehicle interior.

In such an arrangement, for example, when the speakers 30A and 30B exclusively output sound, the sound image is localized in the vicinity of the driver's seat DS. Further, when the sound is output exclusively to the speakers 30C and 30D, the sound image is localized in the vicinity of the passenger seat AS. Further, when the sound is output exclusively to the speaker 30E, the sound image is localized in the vicinity of the right rear seat BS1. Further, when the sound is output exclusively to the speaker 30F, the sound image is localized in the vicinity of the left rear seat BS2. Further, when the sound is output exclusively to the speaker 30G, the sound image is localized near the front of the passenger compartment, and when the sound is output exclusively to the speaker 30H, the sound image is localized near the upper part of the passenger compartment. Become. Not limited to this, the speaker unit 30 can localize the sound image at an arbitrary position in the vehicle interior by adjusting the distribution of the sound output from each speaker by using a mixer or an amplifier.

[Agent device]
Returning to FIG. 2, the agent device 100 includes a management unit 110, agent function units 150-1, 150-2, 150-3, a pairing application execution unit 152, and a storage unit 160. The management unit 110 includes, for example, an acoustic processing unit 112, a WU (Wake Up) determination unit 114 for each agent, a display control unit 116, a voice control unit 118, a function identification unit 120, and a selection unit 122. When it is not distinguished which agent function unit it is, it is simply referred to as an agent function unit 150. The three agent function units 150 are shown only as an example corresponding to the number of agent servers 200 in FIG. 1, and the number of agent function units 150 may be two or four or more. It may be. The software layout shown in FIG. 2 is simply shown for the sake of explanation, and is actually modified arbitrarily so that, for example, the management unit 110 may intervene between the agent function unit 150 and the in-vehicle communication device 60. can do.

Each component of the agent device 100 is realized, for example, by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). Some or all of these components are hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may be realized by (including circuits), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or a removable storage device such as a DVD or a CD-ROM. It is stored in a medium (non-transient storage medium) and may be installed by mounting the storage medium in a drive device. The storage unit 160 is realized by the storage device described above. For example, the function list information 162 is stored in the storage unit 160.

FIG. 5 is a diagram showing an example of the contents of the function list information 162. The function list information 162 is information in which a function that can be executed by an agent, a date when the function becomes executable (illustrated, an executable date), and an execution history of the function are associated with each agent. Is. For example, the execution history is associated with information indicating whether the occupant has "executed" or "not executed" the function, and the function used by the occupant even once is "executed". Information indicating is associated. The contents of the function list information 162 are updated by the agent server 200, for example, every time the function is updated (for example, every time a new function is added) or at predetermined time intervals.

In FIG. 5, the agent 1 is associated with information indicating a map search function, a voice reproduction function, and a shiritori function, and the execution history of any of the functions is information indicating “not executed”. .. Further, the agent 2 is associated with information indicating a map search function and a music playback function, the map search function indicates "executed", and the music playback function indicates "not executed". Information. Further, the agent 3 is associated with information indicating a map search function and a music playback function, and the execution history of each function is information indicating "executed". Details of agents 1 to 3 will be described later.

The management unit 110 functions by executing a program such as an OS (Operating System) or middleware.

The sound processing unit 112 of the management unit 110 performs sound processing on the input sound so as to be in a state suitable for recognizing the wakeup word preset for each agent and the function that the agent can execute. I do.

The WU determination unit 114 for each agent exists corresponding to each of the agent function units 150-1, 150-2, and 150-3, and recognizes a wakeup word predetermined for each agent. The WU determination unit 114 for each agent recognizes the meaning of the voice from the voice (voice stream) subjected to the acoustic processing. First, the WU determination unit 114 for each agent detects a voice section based on the amplitude and zero intersection of the voice waveform in the voice stream. The WU determination unit 114 for each agent may perform frame-by-frame speech recognition based on a mixture Gaussian mixture model (GMM) and section detection based on non-speech recognition.

Next, the WU determination unit 114 for each agent converts the voice in the detected voice section into text and converts it into character information. Then, the WU determination unit 114 for each agent determines whether or not the textual character information corresponds to the wakeup word. When it is determined that it is a wakeup word. The WU determination unit 114 for each agent notifies the selection unit of information indicating the corresponding agent function unit 150. The agent server 200 may be equipped with a function corresponding to the WU determination unit 114 for each agent. In this case, when the management unit 110 transmits the voice stream to which the sound processing has been performed by the sound processing unit 112 to the agent server 200 and determines that the agent server 200 is a wakeup word, the management unit 110 follows an instruction from the agent server 200. The agent function unit 150 starts. It should be noted that each agent function unit 150 may be always activated and may determine the wakeup word by itself. In this case, the management unit 110 does not need to include the WU determination unit 114 for each agent.

The function specifying unit 120 identifies the function of the agent requested by the occupant. First, the function specifying unit 120 detects a voice section based on the amplitude and zero intersection of the voice waveform in the voice stream. The function specifying unit 120 may perform frame-by-frame speech recognition based on the mixed Gaussian distribution model and section detection based on non-speech recognition. Next, the function specifying unit 120 converts the voice in the detected voice section into text and converts it into character information. Then, the function specifying unit 120 determines whether or not the textual character information corresponds to the name of the function included in the function column of the function list information 162. When the function specifying unit 120 determines that the character information corresponds to the name of the function, the function specifying unit 120 identifies the function as the function of the agent requested to be provided by the occupant.

The function specifying unit 120 may inquire of each agent function unit 150 each time the function is specified, the name of the function, the release date of the function, the execution history, and the like. In this case, the function list information 162 may not be stored in the storage unit 160.

The selection unit 122 realizes the function specified by the agent function unit 150 or the function identification unit 120 in which the wakeup word is recognized by the WU determination unit 114 for each agent (that is, corresponds to the utterance of the occupant). Select unit 150. The details of the process in which the selection unit 122 selects the agent function unit 150 will be described later. The selection unit 122 transmits an audio stream to the selected agent function unit 150. The selection unit 122 activates the selected agent function unit 150.

The agent function unit 150 causes an agent to appear in cooperation with the corresponding agent server 200, and provides a service including a voice response in response to an utterance of a vehicle occupant. The agent function unit 150 may include one to which the authority to control the vehicle device 50 is granted. Further, the agent function unit 150 may be one that cooperates with the general-purpose communication device 70 via the pairing application execution unit 152 and communicates with the agent server 200. For example, the agent function unit 150-1 is given the authority to control the vehicle device 50. The agent function unit 150-1 communicates with the agent server 200-1 via the vehicle-mounted communication device 60. The agent function unit 150-2 communicates with the agent server 200-2 via the vehicle-mounted communication device 60. The agent function unit 150-3 cooperates with the general-purpose communication device 70 via the pairing application execution unit 152, and communicates with the agent server 200-3. The pairing application execution unit 152 pairs with the general-purpose communication device 70 by, for example, Bluetooth (registered trademark), and connects the agent function unit 150-3 and the general-purpose communication device 70. The agent function unit 150-3 may be connected to the general-purpose communication device 70 by wired communication using USB (Universal Serial Bus) or the like. Hereinafter, the agent 1 in which the agent function unit 150-1 and the agent server 200-1 collaborate to appear, the agent 2 in which the agent function unit 150-2 and the agent server 200-2 collaborate to appear. An agent that the agent function unit 150-3 and the agent server 200-3 collaborate to appear may be referred to as an agent 3.

The display control unit 116 causes the first display 22 or the second display 24 to display an image in response to an instruction from the agent function unit 150. In the following, it is assumed that the first display 22 is used. The display control unit 116 generates, for example, an image of an anthropomorphic agent (hereinafter referred to as an agent image) that communicates with an occupant in the vehicle interior under the control of a part of the agent function unit 150, and the generated agent. The image is displayed on the first display 22. The agent image is, for example, an image of a mode of talking to an occupant. The agent image may include, for example, a facial image such that the facial expression and the facial orientation are recognized by the viewer (occupant) at least. For example, in the agent image, parts imitating eyes and nose are represented in the face area, and the facial expression and face orientation may be recognized based on the positions of the parts in the face area. In addition, the agent image is felt three-dimensionally, and the viewer can recognize the face orientation of the agent by including the head image in the three-dimensional space, or the agent's image can be included by including the image of the main body (body and limbs). The movement, behavior, posture, etc. may be recognized. Further, the agent image may be an animation image.

The voice control unit 118 causes a part or all of the speakers included in the speaker unit 30 to output voice in response to an instruction from the agent function unit 150. The voice control unit 118 may use a plurality of speaker units 30 to control the localization of the sound image of the agent voice at a position corresponding to the display position of the agent image. The position corresponding to the display position of the agent image is, for example, a position where the occupant is expected to feel that the agent image is speaking the agent voice. Specifically, the position is near the display position of the agent image (for example, 2). It is within ~ 3 [cm]). Further, localization of the sound image means, for example, determining the spatial position of the sound source felt by the occupant by adjusting the loudness of the sound transmitted to the left and right ears of the occupant.

FIG. 6 is a diagram for explaining the principle of determining the position where the sound image is localized. Although FIG. 6 shows an example in which the speakers 30B, 30D, and 30G described above are used for simplification of the description, any speaker included in the speaker unit 30 may be used. The audio control unit 118 controls the amplifier (AMP) 32 and the mixer 34 connected to each speaker to localize the sound image. For example, when the sound image is localized at the spatial position MP1 shown in FIG. 6, the voice control unit 118 controls the amplifier 32 and the mixer 34 to cause the speaker 30B to output 5% of the maximum intensity, and the speaker 30D. The output is 80% of the maximum intensity, and the speaker 30G is made to output 15% of the maximum intensity. As a result, from the position of the occupant P, it seems that the sound image is localized at the spatial position MP1 shown in FIG.

Further, when the sound image is localized at the spatial position MP2 shown in FIG. 6, the voice control unit 118 controls the amplifier 32 and the mixer 34 to cause the speaker 30B to output 45% of the maximum intensity, and the speaker 30D. The output of 45% of the maximum intensity is performed, and the speaker 30G is made to output 45% of the maximum intensity. As a result, from the position of the occupant P, it seems that the sound image is localized at the spatial position MP2 shown in FIG. In this way, by adjusting the plurality of speakers provided in the vehicle interior and the loudness of the sound output from each speaker, the position where the sound image is localized can be changed. More specifically, the localization position of the sound image is determined based on the sound characteristics originally possessed by the sound source, the information on the vehicle interior environment, and the head-related transfer function (HRTF). The voice control unit 118 localizes the sound image at a predetermined position by controlling the speaker unit 30 with the optimum output distribution obtained in advance by a sensory test or the like.

[Agent server]
FIG. 7 is a diagram showing a configuration of the agent server 200 and a part of the configuration of the agent device 100. Hereinafter, the operation of the agent function unit 150 and the like together with the configuration of the agent server 200 will be described. Here, the description of the physical communication from the agent device 100 to the network NW will be omitted.

The agent server 200 includes a communication unit 210. The communication unit 210 is a network interface such as a NIC (Network Interface Card). Further, the agent server 200 includes, for example, a voice recognition unit 220, a natural language processing unit 222, a dialogue management unit 224, a network search unit 226, and a response sentence generation unit 228. These components are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device including a non-transient storage medium), or a removable storage medium such as a DVD or a CD-ROM (non-transient). It is stored in a sex storage medium) and may be installed by attaching the storage medium to a drive device.

The agent server 200 includes a storage unit 250. The storage unit 250 is realized by the above-mentioned various storage devices. Data and programs such as a personal profile 252, a dictionary DB (database) 254, a knowledge base DB 256, and a response rule DB 258 are stored in the storage unit 250.

In the agent device 100, the agent function unit 150 transmits a voice stream or a voice stream that has undergone processing such as compression or coding to the agent server 200. When the agent function unit 150 recognizes a voice command capable of local processing (processing that does not go through the agent server 200), the agent function unit 150 may perform the processing requested by the voice command. The voice command capable of local processing is a voice command that can be answered by referring to a storage unit (not shown) included in the agent device 100, or in the case of the agent function unit 150-1, the vehicle device 50 is controlled. It may be a voice command (for example, a command to turn on the air conditioner). Therefore, the agent function unit 150 may have a part of the functions provided in the agent server 200.

When the voice stream is acquired, the voice recognition unit 220 performs voice recognition and outputs textual character information, and the natural language processing unit 222 interprets the character information with reference to the dictionary DB 254. The dictionary DB 254 is associated with abstract semantic information with respect to character information. The dictionary DB 254 may include list information of synonyms and synonyms. The processing of the voice recognition unit 220 and the processing of the natural language processing unit 222 are not clearly separated in stages, and the voice recognition unit 220 corrects the recognition result in response to the processing result of the natural language processing unit 222. It may be done by influencing each other.

For example, when the natural language processing unit 222 recognizes the meanings such as "today's weather" and "how is the weather" as the recognition result, the natural language processing unit 222 generates a command replaced with the standard character information "today's weather". As a result, even if there is a character fluctuation in the voice of the request, it is possible to facilitate the dialogue according to the request. Further, the natural language processing unit 222 may recognize the meaning of character information by using artificial intelligence processing such as machine learning processing using probability, or may generate a command based on the recognition result.

The dialogue management unit 224 determines the content of the utterance to the occupant of the vehicle M based on the processing result (command) of the natural language processing unit 222 with reference to the personal profile 252, the knowledge base DB 256, and the response rule DB 258. The personal profile 252 includes the personal information of the occupants, hobbies and preferences, the history of past dialogues, etc. stored for each occupant. The knowledge base DB 256 is information that defines the relationships between things. The response rule DB 258 is information that defines the actions (answers, device control contents, etc.) that the agent should perform in response to the command.

Further, the dialogue management unit 224 may identify the occupant by collating with the personal profile 252 using the feature information obtained from the voice stream. In this case, in the personal profile 252, for example, personal information is associated with voice feature information. The voice feature information is, for example, information on the characteristics of how to speak such as voice pitch, intonation, and rhythm (sound pitch pattern), and the feature amount based on the Mel Frequency Cepstrum Coefficients. .. The voice feature information is, for example, information obtained by having the occupant utter a predetermined word or sentence at the time of initial registration of the occupant and recognizing the uttered voice.

The dialogue management unit 224 causes the network search unit 226 to perform a search when the command requests information that can be searched via the network NW. The network search unit 226 accesses various web servers 300 via the network NW and acquires desired information. The "information searchable via the network NW" may be, for example, an evaluation result by a general user of a restaurant in the vicinity of the vehicle M, or a weather forecast according to the position of the vehicle M on that day.

The response sentence generation unit 228 generates a response sentence and transmits it to the agent device 100 so that the content of the utterance determined by the dialogue management unit 224 is transmitted to the occupant of the vehicle M. The response sentence generation unit 228 calls the occupant's name or generates a response sentence that resembles the occupant's way of speaking when the occupant is identified as a registered occupant in the personal profile. You may.

When the agent function unit 150 acquires the response sentence, the agent function unit 150 instructs the voice control unit 118 to perform voice synthesis and output the voice. Further, the agent function unit 150 instructs the display control unit 116 to display the image of the agent in accordance with the audio output. In this way, the agent function in which the virtually appearing agent responds to the occupant of the vehicle M is realized.

[About the selection process of the agent function unit 150: No wake word]
Hereinafter, the selection process in which the selection unit 122 selects the agent function unit 150 will be described. FIG. 8 is a diagram showing an example of a dialogue between an agent and an occupant when providing a map search function. First, the occupant makes an utterance CV1 including requesting the agent to provide the map search function. The utterance CV1 is, for example, a word such as "Activate the map search function?". In response to this, the selection unit 122 searches the function list information 162 using, for example, the function specified by the function specifying unit 120 by the above-mentioned process (in this example, the map search function) as a search key, and the function corresponds to the function. Identify the attached agent. In the function list information 162 of FIG. 5, the agents associated with the map search function are the agents of agents 1 to 3.

Next, the selection unit 122 sets the execution history of the function to "execution" even when there is already an agent whose execution history of the function indicates "executed" among the agents associated with the function. Priority is given to agents that indicate "not executed". In the function list information 162 of FIG. 5, the agent 1 indicates that the map search function is "not executed". Therefore, the selection unit 122 preferentially selects and activates the agent function unit 150-2 and the agent function unit 150-3 as the agent function unit that causes the agent function unit 150-1 to respond to the voice of the occupant.

The agent function unit 150 (agent function unit 150-1 in this example) activated by the selection unit 122 sends the response sentence RP1 to the speech CV1 from the corresponding agent server 200 (agent server 200-1 in this example). The voice control unit 118 is instructed to acquire the voice, synthesize the voice in the response sentence RP1, and output the voice. The response sentence RP1 includes, for example, a word that introduces an agent of the agent function unit 150 that executes the requested function in the utterance CV1. Response sentence RP1 is, for example, "Hello, this is △△ (agent 1). I will provide a map search function.", Which is a word such as.

The agent function unit 150-1 provides the requested function (in this example, the map search function) when the utterance CV2 of the occupant with respect to the response sentence RP1 has a positive content. Further, when the occupant's utterance CV2 with respect to the response sentence RP1 has a negative content, the agent function unit 150-1 instructs the selection unit 122 to select the agent function unit 150 again. In this case, the selection unit 122 selects the agent function unit 150 that provides the function requested by the occupant from the agent function unit 150 excluding the agent function unit 150 that has been selected once.

[About the selection process of the agent function unit 150: There is a wake word]
Next, a case where the occupant performs the utterance CV3 including the wake-up word and the request for the provision of the map search function to the agent will be described. FIG. 9 is a diagram showing an example of the agent's response to the utterance CV3 including the wakeup word. The utterance CV3 is, for example, a word such as "'Hey OO (Agent 2)" (Wake Up Word), activate the map search function? " In response to this, the selection unit 122 identifies, for example, the agents to which the map search function is associated are the agents 1 to 3, as described above. Next, when the selection unit 122 already has an agent whose execution history of the function indicates "executed" among the agents associated with the function, and an agent specified by the wakeup word exists. Even so, the agent whose execution history of the function indicates "not executed" is preferentially selected. In the function list information 162 of FIG. 5, the agent 1 indicates that the map search function is "not executed". Therefore, the selection unit 122 preferentially selects and activates the agent function unit 150-2 and the agent function unit 150-3 as the agent function unit that causes the agent function unit 150-1 to respond to the voice of the occupant.

The agent function unit 150 (agent function unit 150-1 in this example) activated by the selection unit 122 sends the response sentence RP2 to the speech CV1 from the corresponding agent server 200 (agent server 200-1 in this example). The voice control unit 118 is instructed to acquire the voice, synthesize the voice in the response sentence RP2, and output the voice. Here, in the response statement RP2, for example, in the utterance CV1, a wakeup word that activates agents 2 to 3 other than the agent (in this example, agent 1) realized by the agent function unit 150 activated by the selection unit 122 is used. If it is included, in order to prevent confusion for the occupants, a word claiming that the activated agent is Agent 1 is included. Further, the response sentence RP2 includes, for example, a word that introduces that the requested function can be executed by the agent function unit 150 activated by the selection unit 122. Response sentence RP2 is, for example, is the word of "Hello, △△ is (agent 1). I also map search function by I was able to use. Do not try to use if you like?", And the like.

The agent function unit 150-1 provides the requested function (in this example, the map search function) when the occupant's utterance CV4 for the response sentence RP2 has a positive content. Further, when the occupant's utterance CV4 for the response sentence RP2 has a negative content, the agent function unit 150-1 instructs the selection unit 122 to select the agent function unit 150 again. In this case, the selection unit 122 selects the agent function unit 150 that provides the function requested by the occupant from the agent function unit 150 excluding the agent function unit 150 that has been selected once.

As described above, according to the agent device 100 of the present embodiment, the agent having the new function can preferentially respond to the occupant, and the new function can be easily used by the occupant.

[Operation flow]
FIG. 10 is a flowchart showing a series of flow of operations of the agent device 100. First, the sound processing unit 112 performs sound processing on the sound picked up by the microphone 10 (step S100). Next, the function specifying unit 120 identifies the function of the agent requested by the occupant based on the acoustically processed voice stream (step S102). The selection unit 122 determines whether or not there is an agent capable of executing the function specified by the function specifying unit 120 (step S104). When there is no agent capable of realizing the specified function, the selection unit 122 selects / activates the agent function unit 150 according to a predetermined rule, and provides an audio stream to the activated agent function unit 150 (step S106). The predetermined rule is, for example, a rule for selecting the agent function unit 150 based on a predetermined selection order, or a rule for randomly selecting the agent function unit 150.

In response to this, the agent server 200 generates a response statement for replying to the occupant that the function cannot be provided, and provides the response statement to the management unit 110. Next, the agent function unit 150 is provided by the agent server 200 and acquires a response statement (step S108). Next, the agent function unit 150 determines whether or not the task of the agent has been completed (step S110). For example, the agent function unit 150 determines that the task has been completed when a response sentence to the occupant's utterance is provided. The voice control unit 118 performs voice synthesis on the response sentence acquired by the agent function unit 150-1 and outputs the voice (step S112).

When the selection unit 122 determines that there is an agent capable of realizing the specified function, the selection unit 122 determines whether or not there is an agent whose execution history of the function indicates "not executed" among the agents ( Step S114). When the selection unit 122 determines that there is no agent whose execution history indicates "not executed", the selection unit 122 realizes the requested function from the agent function whose execution history indicates "executed" according to a predetermined rule. The agent function unit 150 is selected (step S116). The selection unit 122 provides an audio stream to the selected agent function unit 150 (step S118).

In response to this, the agent server 200 generates a response statement for replying to the occupant that the agent provides the requested function, and provides the response statement to the management unit 110. Next, the selected agent function unit 150 acquires the response statement provided by the agent server 200 (step S120). Next, the agent function unit 150 determines whether or not the task of the agent has been completed (step S122). For example, the agent function unit 150 determines that the task has been completed when a response sentence to the occupant's utterance is provided. The voice control unit 118 performs voice synthesis on the response sentence acquired by the agent function unit 150 and outputs the voice (step S124).

When the selection unit 122 determines that there is an agent indicating "not executed", the selection unit 122 provides an audio stream to the agent function unit 150 that realizes the specified agent (step S126). When the selection unit 122 determines that there is an agent indicating "not executed", the selection unit 122 sets the agent function unit 150 that realizes the requested function among the agent function units 150 that realize the specified agent according to a predetermined rule. You may choose.

In response to this, the agent server 200 generates a response statement for replying to the occupant that the agent provides the requested function, and provides the response statement to the management unit 110. Next, the agent function unit 150 acquires the response statement provided by the agent server 200 (step S128). Next, the agent function unit 150 determines whether or not the task of the agent has been completed (step S130). The voice control unit 118 performs voice synthesis on the response sentence acquired by the agent function unit 150 and outputs the voice (step S132).

[Priority of agent function unit 150]
In addition, when there are a plurality of agent function units 150 indicating that the function requested by the occupant is "not executed", the selection unit 122 sets the agent function unit 150 based on the priority assigned to each agent function unit 150. You may choose. Among the plurality of agent function units 150, the agent function unit 150 having a high priority is, for example, a vehicle agent function unit having a function of instructing the vehicle device 50 to operate (in this example, the agent function unit 150-1). ). Hereinafter, the agent function unit 150 having the highest priority is the agent function unit 150-1, and the priority relationship with other agent function units 150 is as follows: agent function unit 150-1> agent function unit 150-2>. It is assumed that the agent function unit 150-3.

For example, in the selection unit 122, when the function requested by the occupant is the "music playback function", the agents whose execution history indicates "not executed" are agents 1 and 2, but the agent function that realizes the agent 1. Since the unit 150-1 has a higher priority than the agent function unit 150-2 that realizes the agent 2, the agent function unit 150-1 is selected.

As described above, according to the agent device 100 of the present embodiment, it is possible to give priority to the specific agent to deal with the occupant and increase the chance for the occupant to interact with the familiar agent.

[Operation flow]
FIG. 11 is a flowchart showing a series of flow of operations of the agent device 100 when the agent function unit 150 is prioritized. The same step numbers are assigned to the same processes as those shown in FIG. 10, and the description thereof will be omitted.

When the selection unit 122 determines that there is an agent whose function execution history indicates "not executed", the selection unit 122 determines whether or not the agent includes an agent having a high priority (in this example, agent 1) (in this example, the agent 1). Step S200). When the selection unit 122 determines that the agent 1 is included in the agent, the selection unit 122 provides an audio stream to the agent function unit 150-1 that realizes the agent 1 having a high priority (step S202). In response to this, the agent server 200-1 generates a response statement for replying to the occupant that the agent 1 provides the requested function, and provides the response statement to the management unit 110. Next, the agent function unit 150 acquires the response statement provided by the agent server 200 (step S204). Next, the agent function unit 150 determines whether or not the task of the agent has been completed (step S206). For example, the agent function unit 150 determines that the task has been completed when a response sentence to the occupant's utterance is provided. The voice control unit 118 performs voice synthesis on the response sentence acquired by the agent function unit 150 and outputs the voice (step S208).

When the selection unit 122 determines in step S114 that there is no agent whose function execution history indicates "not executed", or when it is determined that the agent 1 is not included in the agents capable of realizing the specified function. (Step S210) The predetermined rule is, for example, a rule for selecting the agent function unit 150 based on a predetermined selection order, or a rule for selecting the agent function unit 150 that realizes the requested function according to a predetermined rule. This is a rule for randomly selecting the agent function unit 150, or a rule for selecting the agent function unit 150 that realizes an agent having a high priority among agents whose execution history indicates "executed". The selection unit 122 provides an audio stream to the agent function unit 150 that realizes the selected agent (step S212).

In response to this, the agent server 200 generates a response statement for replying to the occupant that the agent provides the requested function, and provides the response statement to the management unit 110. Next, the agent function unit 150 acquires the response statement provided by the agent server 200 (step S214). Next, the agent function unit 150 determines whether or not the task of the agent has been completed (step S216). The voice control unit 118 performs voice synthesis on the response sentence acquired by the agent function unit 150 and outputs the voice (step S218).

[Processing to provide information about newly added functions: When there is an inquiry]
Further, when a new function is added, the agent function unit 150 may provide the occupant with information on the newly added function. FIG. 12 is a diagram showing an example of a dialogue between an agent and an occupant when providing information on a newly added function. First, the occupant makes an utterance CV3 inquiring the agent about the newly added function of the agent. The utterance CV3 is, for example, a word such as "Are there any new functions?" In response to this, the function specifying unit 120 determines whether or not the textualized character information includes words such as "new function". For example, when the textualized character information includes a word such as "new function", the function specifying unit 120 determines that the occupant is inquiring about the newly added function of the agent.

When it is determined by the function specifying unit 120 that the occupant is inquiring about the newly added function of the agent, the selection unit 122 selects a function whose execution history is "not executed" in the function list information 162. Identify. In FIG. 5, the function whose execution history is “not executed” is, for example, a shiritori function that can be executed by the agent 1. The selection unit 122 selects and activates the agent function unit 150-1 as the agent function unit that responds to the voice of the occupant.

The agent function unit 150 (agent function unit 150-1 in this example) activated by the selection unit 122 sends the response sentence RP2 to the speech CV3 from the corresponding agent server 200 (agent server 200-1 in this example). The voice control unit 118 is instructed to acquire the voice, synthesize the voice in the response sentence RP2, and output the voice. The response statement RP2 includes, for example, a word that introduces that the newly added function can be executed by the agent function unit 150 activated by the selection unit 122. Response sentence RP2 is, for example, "Hello, this is △△ (agent 1). I" shiritori function "can now be run. Would you like to use?" Is a word such as.

The agent function unit 150-1 provides the requested function (in this example, the shiritori function) when the occupant's utterance CV4 for the response sentence RP2 has a positive content. Further, when the occupant's utterance CV4 for the response sentence RP2 has a negative content, the agent function unit 150-1 instructs the selection unit 122 to select the agent function unit 150 again. In this case, the selection unit 122 is a function excluding the function once selected, selects a function whose usage history is "not executed", and selects an agent function unit 150 capable of executing the function.

As described above, according to the agent device 100 of the present embodiment, the new function can be introduced to the occupant and the new function can be easily used by the occupant.

[Operation flow]
FIG. 13 is a flowchart showing a series of processes for introducing the unexecuted functions of the agent device 100. First, the sound processing unit 112 performs sound processing on the sound picked up by the microphone 10 (step S300). Next, the function specifying unit 120 determines whether or not the occupant has inquired about the additional function based on the acoustically processed voice stream (step S302). The agent device 100 ends the processing of the flowchart of FIG. 13 when the occupant has not inquired about the additional function. When it is determined that the occupant has inquired about the additional function, the function specifying unit 120 determines whether or not there is an unexecuted agent function based on the function list information 162 (step S304). When the function specifying unit 120 determines that there is no unexecuted agent function, the voice control unit 118 performs voice synthesis on the response sentence notifying that there is no additional function and outputs the voice (step S306). For example, the function specifying unit 120 instructs the agent function unit 150 to generate a response sentence notifying that there is no additional function, and receives the response sentence provided by the agent function unit 150. The response statement notifying that there is no additional function may be provided by the agent function unit 150 having the highest priority, or may be provided by another agent function unit 150.

The function specifying unit 120 provides an audio stream to the agent function unit 150 having an unexecuted function (step S308). In response to this, the agent server 200 generates a response statement for replying to the occupant that the agent provides the requested function, and provides the response statement to the management unit 110. Next, the agent function unit 150 acquires the response statement provided by the agent function unit 150 (step S310). Next, the agent function unit 150 determines whether or not the task of the agent has been completed (step S312). The voice control unit 118 performs voice synthesis on the response sentence acquired by the agent function unit 150 and outputs the voice (step S314).

[Processing to provide information about newly added functions: When there is no inquiry]
In the above description, the case where the agent function unit 150 provides the occupant with information on the newly added function when the occupant inquires about the additional function has been described, but the present invention is not limited to this. The agent function unit 150 may provide the occupant with information on the newly added function, for example, when making a response (for example, chatting) unrelated to the newly added function. For example, when the newly added function is the "shiritori function" and the agent function unit 150 responds to the occupant regarding the "map search function", the agent function unit 150 responds to the map search function. After finishing, by responding such as "By the way, I can execute the" Shiritori function ". Do you want to use it?", The information about the newly added function is given to the occupants. May be provided.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 ... Agent system, 10 ... Microphone, 20 ... Display / operation device, 22 ... 1st display, 24 ... 2nd display, 30 ... Speaker unit, 32 ... Amplifier, 34 ... Mixer, 40 ... Navigation device, 50 ... Vehicle equipment , 60 ... In-vehicle communication device, 70 ... General-purpose communication device, 80 ... Crew recognition device, 100 ... Agent device, 110 ... Management unit, 112 ... Sound processing unit, 114 ... WU judgment unit for each agent, 116 ... Display control unit, 118 ... Voice control unit, 120 ... Function identification unit, 122 ... Selection unit, 150, 150-1, 150-2, 150-3 ... Agent function unit, 152 ... Pairing application execution unit, 160 ... Storage unit, 162 ... Function List information, 200, 200-1, 200-2, 200-3 ... Agent server, 210 ... Communication unit, 220 ... Voice recognition unit, 222 ... Natural language processing unit, 224 ... Dialogue management unit, 226 ... Network search unit, 228 ... Response sentence generator, 250 ... Storage unit, 252 ... Personal profile, 300 ... Web server

Claims (7)

Multiple agent function units that provide services including voice responses in response to vehicle occupants' utterances
Among the plurality of agent function units, a selection unit for selecting an agent function unit corresponding to the utterance of the occupant is provided.
The selection unit is newly added when a new function is added to one agent function unit among the plurality of agent function units and when the newly added function is provided to the occupant. Priority is given to other agent function units that already have the same functions as the above functions, and the crew members are made to provide the functions of the agent function units to which the new functions have been added.
Agent device. Multiple agent function units that provide services including voice responses in response to vehicle occupants' utterances
Among the plurality of agent function units, a selection unit for selecting an agent function unit corresponding to the utterance of the occupant is provided.
The plurality of agent function units include a vehicle agent function unit having a function of instructing the vehicle equipment to operate.
The selection unit is newly added when a new function is added to the vehicle agent function unit among the plurality of agent function units and when the newly added function is provided to the occupant. The occupant is made to provide the function of the vehicle agent function unit to which the new function is added, preferentially to other agent function units that already have the same function as the above function.
Agent device. The selection unit is the newly added function when the newly added function is provided to the occupant even if the question specifies a specific agent function unit among the plurality of agent function units. Priority is given to other agent function units that already have the same function as the above, and the crew member is made to provide the function of the agent function unit to which the new function is added.
The agent device according to claim 1 or 2. When a new function is added to at least one agent function unit among the plurality of agent function units, the agent function unit responds to an inquiry that does not specify the details of the new function and adds the new function unit. To provide the occupants with information about the functions performed,
The agent device according to any one of claims 1 to 3. When a new function is added to at least one agent function unit among the plurality of agent function units, the agent function unit newly performs the response unrelated to the new function. To provide the occupants with information about the added features,
The agent device according to any one of claims 1 to 4. The computer activates one of the plurality of agent function units, and as a function of the activated agent function unit,
Providing services including voice responses in response to vehicle occupants' utterances
From the plurality of agent function units, the agent function unit corresponding to the utterance of the occupant is selected.
When a new function is added to one of the plurality of agent function units and the newly added function is provided to the occupant, the same function as the newly added function is provided. The occupant is made to provide the function of the agent function unit to which the new function is added, preferentially to the other agent function unit that already has the above.
How to control the agent device. The computer is activated by one of a plurality of agent function units, and as a function of the activated agent function unit,
Provide services including voice response in response to vehicle occupants' utterances
Among the plurality of agent function units, the agent function unit corresponding to the utterance of the occupant is selected.
When a new function is added to one of the plurality of agent function units and the newly added function is provided to the occupant, the same function as the newly added function is provided. The occupant is made to provide the function of the agent function unit to which the new function is added, preferentially to the other agent function unit that already has the above.
program.

Images