JP2021026124A - Voice interactive device, voice interactive method, and program - Google Patents

Abstract

To provide a voice interactive device, a voice interactive method and a program capable of expressing accuracy of voice recognition results.SOLUTION: In a vehicle, a voice interactive device includes: a voice acquisition unit which acquires voice information; a first information acquisition unit which receives and acquires external response information for the voice information acquired by the voice acquisition unit from an external device; a storage unit which stores internal response information for specific voice information; a second information acquisition unit which acquires the internal response information for the voice information acquired by the voice acquisition unit from the storage unit; and an output control unit for outputting at least one of the external response information and the internal response information to an output unit. The output control unit causes the output unit to output a sound effect together with the internal response information, when outputting the internal response information.SELECTED DRAWING: Figure 2

Description

The present invention relates to a voice dialogue device, a voice dialogue method, and a program.

Conventionally, there are voice dialogue systems that interact with a user and provide an answer according to a request made by the user, particularly an agent system having a voice dialogue function. In a voice dialogue system, for example, a local voice dialogue device acquires a user's utterance, sends it to a server, and provides the user with an answer obtained from the server. In addition, when the reception environment is poor, or when the user's request is simple and the answer is short, there is also a hybrid format in which the answer is provided by the voice dialogue device itself instead of the answer from the server. ..

On the other hand, for example, in an in-vehicle device mounted on a vehicle, if the reception environment is poor and the error rate becomes large, it causes noise. Conventionally, there is an information receiver that adds pseudo noise with an adjusted noise level to a voice whose volume level has been adjusted due to deterioration of a reception state (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2007-173967

The information receiver described in Patent Document 1 avoids a silent state and avoids a silent state by adding pseudo noise whose volume level is adjusted to the voice whose volume level is adjusted due to deterioration of the reception state. The mixed result can be heard by the user. However, since this information receiver adds pseudo noise based on the error rate of the received digital audio broadcast signal, for example, it indicates the accuracy of how well the response to the user's request corresponds. It was not possible, and users sometimes had difficulty understanding why the answer was.

The present invention has been made in consideration of such circumstances, and an object of the present invention is to provide a voice dialogue device, a voice dialogue method, and a program capable of informing the user of the accuracy of a response to a user's request. Let's try.

The voice dialogue device, the voice dialogue method, and the program according to the present invention have adopted the following configurations.
(1): The voice dialogue device according to one aspect of the present invention receives and acquires external response information to the voice information acquired by the voice acquisition unit from a voice acquisition unit that acquires voice information and an external device. A second information acquisition unit that acquires internal response information to the voice information acquired by the voice acquisition unit from the first information acquisition unit, a storage unit that stores internal response information for specific voice information, and the storage unit. A unit and an output control unit that outputs at least one of the external response information and the internal response information to the output unit are provided, and the output control unit together with the internal response information when outputting the internal response information. It is a voice dialogue device that outputs a sound effect to the output unit.

(2): In the aspect of (1) above, the communication between the determination unit that determines whether to output the internal response information or the external response information, and the first information acquisition unit and the external device. It further includes a communication quality acquisition unit that acquires communication quality information related to quality, and the determination unit determines response information to be output based on the communication quality information acquired by the communication quality acquisition unit, and the output control. The unit outputs the internal response information or the external response information determined by the determination unit to the output unit.

(3): In the aspect of (2) above, the determination unit determines to output the internal response information when the communication quality information acquired by the communication quality acquisition unit is equal to or lower than the first determination quality. Is.

(4): In the aspect of (3) above, the determination unit determines that the external response information is output when the communication quality information acquired by the communication quality acquisition unit exceeds the first determination quality. The output control unit outputs a sound effect to the output unit together with the external response information when the communication quality information acquired by the communication quality acquisition unit is equal to or lower than the second determination quality.

(5): In any of the above aspects (1) to (4), the output control unit outputs the sound effect overlaid on the internal response information.

(6): In any of the above (1) to (5), when the vehicle is mounted on a vehicle and the output control unit outputs information about the vehicle-mounted device as the internal response information. Does not output the sound effect.

(7): The voice dialogue method according to one aspect of the present invention acquires voice information, receives and acquires external response information to the voice information from an external device, and quality of communication with the external device. The internal response information for the voice information is acquired from the storage unit that acquires the communication quality information regarding the voice information and stores the internal response information for the specific voice information, and at least one of the external response information and the internal response information is output as an output unit. This is a voice dialogue method in which a sound effect is output to the output unit together with the internal response information when the internal response information is output.

(8): In the program according to one aspect of the present invention, a computer of a voice dialogue device is made to acquire voice information, and external response information to the voice information is received and acquired from the external device, and the program is connected to the external device. Of the external response information and the internal response information, the communication quality information regarding the quality of communication between the two is acquired, and the internal response information for the voice information is acquired from the storage unit that stores the internal response information for the specific voice information. When the process of outputting at least one of them to the output unit is executed and the internal response information is output, the program executes the process of outputting the sound effect to the output unit together with the internal response information.

According to (1) to (8), it is possible to inform the user of the accuracy of the response to the user's request.
According to (2) to (5), it is possible to make the user recognize that the accuracy of the response to the user's request is low.
According to (6), it is possible to make it easier for the user to recognize the information about the vehicle equipment.

It is a block diagram of the agent system 1 including the voice dialogue apparatus 100. It is a figure which shows the structure of the voice dialogue apparatus 100, and the apparatus mounted on the vehicle M. It is a figure which shows the configuration of the agent server 200, and a part of the configuration of a voice dialogue device 100. It is a flowchart which shows an example of the flow of the process executed in the voice dialogue apparatus 100. It is a flowchart which shows an example of the flow of the process executed in the voice dialogue apparatus 100. It is explanatory drawing which shows an example of the display of the display / operation device 20 and the output of a peaker 30. It is explanatory drawing which shows an example of the display of the display / operation device 20 and the output of a peaker 30.

Hereinafter, embodiments of the voice dialogue device, the voice dialogue method, and the program of the present invention will be described with reference to the drawings. The voice dialogue device includes, for example, an agent function. The agent function is, for example, a function of providing various information based on a request (command) included in the utterance of the occupant or mediating a network service while interacting with the occupant who is the user of the vehicle M. Is. The agent may be singular or multiple types. The functions, processing procedures, controls, output modes and contents of each of the plurality of types of agents may be different. In addition, some of the agent functions may have a function of controlling equipment in the vehicle (for example, equipment related to driving control and vehicle body control).

Agent functions include, for example, a voice recognition function that recognizes the voice of an occupant (a function that converts voice into text), a natural language processing function (a function that understands the structure and meaning of text), a dialogue management function, and a search for other devices. Alternatively, it is realized by integratedly using a network search function or the like for searching a predetermined database owned by the own device. Some or all of these functions may be realized by AI (Artificial Intelligence) technology. In addition, a part of the configuration for performing these functions (particularly, the voice recognition function and the natural language processing interpretation function) communicates with the in-vehicle communication device of the vehicle M or the general-purpose communication terminal (mobile terminal) brought into the vehicle M. It may be installed in a possible agent server. In the following description, it is assumed that a part of the configuration is installed in the agent server, and the voice dialogue device and the agent server cooperate to realize the agent function. Further, a service provider (service entity) in which a voice dialogue device and an agent server cooperate to appear virtually is called an agent. The agent server is an example of an "external device".

[overall structure]
FIG. 1 is a configuration diagram of an agent system 1 including a voice dialogue device 100. The agent system 1 includes, for example, a voice dialogue device 100 and an agent server 200. The agent server 200 may be singular or plural. When a plurality of agent servers 200 exist, the plurality of agent servers 200 are operated by providers of different agent systems. The agents in this case are operated by providers of different agent systems.

The voice dialogue device 100 is provided by an agent system provider having an agent server 200. The agent server 200 is a parent server of the agent function unit 150 in the voice dialogue device 100. Examples of the agent system provider include automobile manufacturers, network service providers, e-commerce businesses, mobile terminal sellers and manufacturers, and any entity (corporation, group, individual, etc.) is the agent system. Can be a provider of

The voice dialogue device 100 communicates with the agent server 200 via the network NW. The network NW includes, for example, a part or all of the Internet, a cellular network, a Wi-Fi network, a WAN (Wide Area Network), a LAN (Local Area Network), a public line, a telephone line, a wireless base station, and the like. Various web servers 500 are connected to the network NW. Both the voice dialogue device 100 and the agent server 200 can acquire web pages from various web servers 500 via the network NW.

The voice dialogue device 100 interacts with the occupant of the vehicle M, generates response information such as a response sentence based on the voice from the occupant, transmits information about the voice from the occupant to the agent server 200, and causes an agent. Response sentences are obtained from the server 200, and these response sentences are presented to the occupants in the form of voice output or image display.

[vehicle]
FIG. 2 is a diagram showing the configuration of the voice dialogue device 100 and the equipment mounted on the vehicle M. The vehicle M is equipped with, for example, one or more microphones 10, a display / operation device 20, a speaker 30, a navigation device 40, a vehicle device 50, an in-vehicle communication device 60, and a voice dialogue device 100. .. In addition, a general-purpose communication terminal is brought into the vehicle interior and used as a communication device. These devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 2 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The microphone 10 is a sound collecting unit that collects sounds emitted in the vehicle interior. The display / operation device 20 is a device (or a group of devices) capable of displaying an image and accepting an input operation. The display / operation device 20 includes, for example, a display device configured as a touch panel. The display / operation device 20 may further include a HUD (Head Up Display) or a mechanical input device. The speakers 30 are arranged at a plurality of positions different from each other in the vehicle interior, for example. The display / operation device 20 may be shared by the voice dialogue device 100 and the navigation device 40. The display / operation device 20 and the speaker 30 are examples of the “output unit”.

The navigation device 40 includes a navigation HMI (Human machine Interface), a positioning device such as a GPS (Global Positioning System), a storage device that stores map information, and a control device (navigation controller) that performs route search and the like. .. A part or all of the microphone 10, the display / operation device 20, and the speaker 30 may be used as the navigation HMI. The navigation device 40 searches for a route (navigation route) for moving from the position of the vehicle M specified by the positioning device to the destination input by the occupant, so that the vehicle M can travel along the route. , Navi HMI is used to output guidance information. The route search function may be provided in a navigation server accessible via the network NW. In this case, the navigation device 40 acquires a route from the navigation server and outputs guidance information. The voice dialogue device 100 may be constructed based on the navigation controller. In that case, the navigation controller and the voice dialogue device 100 are integrally configured on the hardware.

The vehicle device 50 includes, for example, a driving force output device such as an engine or a traveling motor, an engine start motor, a door lock device, a door opening / closing device, a window, a window opening / closing device, a window opening / closing control device, a seat, and a seat position. Control device, room mirror and its angle position control device, lighting device inside and outside the vehicle and its control device, wiper and defogger and their respective control devices, direction indicator and its control device, air conditioner, mileage and tire pressure information And vehicle information devices such as fuel level information.

The in-vehicle communication device 60 is, for example, a wireless communication device that can access the network NW using a cellular network or a Wi-Fi network.

The voice dialogue device 100 includes, for example, a management unit 110 and an agent function unit 150. The management unit 110 includes, for example, an sound processing unit 112, a WU (Wake Up) determination unit 114, a display control unit 116, and a voice control unit 118. The agent function unit 150 includes, for example, a voice acquisition unit 151, a first information acquisition unit 152, a second information acquisition unit 153, a communication quality acquisition unit 154, a determination unit 155, an output control unit 156, and an in-vehicle device. It includes a command unit 157 and a storage unit 160. The configuration of each functional unit shown in FIG. 2 is simply shown for the sake of explanation. In practice, for example, the management unit 110 may intervene between the agent functional unit 150 and the in-vehicle communication device 60. It can be modified arbitrarily.

Each component of the voice dialogue device 100 is realized, for example, by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). Some or all of these components are hardware (circuit parts;) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit). It may be realized by (including circuits), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or a removable storage device such as a DVD or a CD-ROM. It is stored in a medium (non-transient storage medium) and may be installed by mounting the storage medium in a drive device. The storage unit 160 of the agent function unit 150 stores, for example, a local dictionary DB (database) 162, a local knowledge base DB 164, and a local response rule DB 166. The storage unit 160 is realized by the above-mentioned various storage devices included in the voice dialogue device 100.

,
The management unit 110 functions by executing a program such as an OS (Operating System) or middleware. The sound processing unit 112 of the management unit 110 performs sound processing on the input voice so as to be in a state suitable for recognizing a wakeup word preset for the voice dialogue device 100. Generate an audio stream.

The WU determination unit 114 recognizes a predetermined wake-up word for the voice dialogue device 100. The WU determination unit 114 recognizes the meaning of voice from the voice stream generated by the sound processing unit 112. First, the WU determination unit 114 detects the voice section as voice information based on the amplitude and zero intersection of the voice waveform in the voice stream. The WU determination unit 114 may perform frame-by-frame speech recognition based on a mixture Gaussian mixture model (GMM) and section detection based on non-speech recognition.

Next, the WU determination unit 114 converts the voice in the voice information of the detected voice section into text and converts it into character information. Then, the WU determination unit 114 determines whether or not the textualized character information corresponds to the wakeup word. When it is determined that the word is a wakeup word, the WU determination unit 114 activates the agent function unit 150. The WU determination unit 114 outputs character information determined to be not a wakeup word to the agent function unit 150.

The agent server 200 may be equipped with a function corresponding to the WU determination unit 114. In this case, when the management unit 110 transmits the voice stream to which the sound processing has been performed by the sound processing unit 112 to the agent server 200 and determines that the agent server 200 is a wakeup word, the management unit 110 follows an instruction from the agent server 200. The agent function unit 150 starts. The agent function unit 150 may be always activated and may determine the wakeup word by itself. In this case, the management unit 110 does not need to include the WU determination unit 114.

The display control unit 116 causes the display / operation device 20 to display an image in response to an instruction from the agent function unit 150. Under the control of the agent function unit 150, the display control unit 116 generates, for example, an image of an anthropomorphic agent (hereinafter referred to as an agent image) that communicates with the occupant in the vehicle interior, and displays the generated agent image. -Display on the operating device 20. The agent image is, for example, an image of a mode of talking to an occupant. The agent image may include, for example, a facial image such that the facial expression and the facial orientation are recognized by the viewer (occupant) at least. For example, in the agent image, parts imitating eyes and nose are represented in the face area, and the facial expression and face orientation may be recognized based on the positions of the parts in the face area. In addition, the agent image is felt three-dimensionally, and the viewer can recognize the face orientation of the agent by including the head image in the three-dimensional space, or the agent's image can be included by including the image of the main body (body and limbs). The movement, behavior, posture, etc. may be recognized. Further, the agent image may be an animation image.

The voice control unit 118 causes the speaker 30 to output voice in response to an instruction from the agent function unit 150. The voice control unit 118 may use a plurality of speakers 30 to control the localization of the sound image of the agent voice at a position corresponding to the display position of the agent image. The voice control unit 118 is an example of an “output control unit”.

By executing an in-vehicle application program (hereinafter referred to as an in-vehicle agent application) for providing a service including a voice response, an agent appears in cooperation with the agent server 200, and is based on the utterance of a vehicle occupant. Provide services including voice responses according to utterance information. The agent function unit 150 is authorized to control the vehicle device 50, but may not be authorized to control the vehicle device 50.

The voice acquisition unit 151 acquires and recognizes the character information output by the WU determination unit 114 of the management unit 110. The voice acquisition unit 151 acquires and recognizes voice information in the form of character information. The voice acquisition unit 151 interprets the meaning of the recognized character information with reference to the local dictionary DB 162 stored in the storage unit 160. The local dictionary DB 162 may include list information of synonyms and synonyms. The process of recognizing character information and the process of interpreting meaning are not clearly separated in stages, and they are performed by interacting with each other, such as correcting the recognition result of character recognition based on the result of semantic interpretation. You can.

For example, when the recognition result recognizes meanings such as "today's weather" and "how is the weather", the voice acquisition unit 151 generates a command replaced with the standard character information "today's weather". As a result, even if the voice of the request has character fluctuations, it is possible to facilitate the dialogue that meets the request. The voice acquisition unit 151 transmits the recognition result of character recognition to the agent server 200 by using the in-vehicle communication device 60. The voice acquisition unit 151 becomes a part of a transmission unit that transmits information or a communication unit that transmits and receives information.

The first information acquisition unit 152 receives and acquires the server response statement transmitted by the agent server 200 by using the in-vehicle communication device 60. Decision process information regarding the process of generating the server response statement is added to the server response statement. The first information acquisition unit 152 outputs the acquired server response statement to the determination unit 155, and outputs the determination process information to the communication quality acquisition unit 154. The first information acquisition unit 152 becomes a part of a receiving unit that receives information or a communication unit that transmits and receives information. The generation and transmission of the server response statement, the determination process information, and the like will be described when the agent server 200 is described.

The second information acquisition unit 153 determines and acquires a response sentence to the occupant of the vehicle M with reference to the local knowledge base DB 164 and the local response rule DB 166 based on the recognition result recognized by the voice acquisition unit 151. The local knowledge base DB 164 is information that defines the relationships between things. The local response rule DB 166 is information that defines an operation (answer, device control content, etc.) that the agent should perform in response to the command.

The second information acquisition unit 153 generates and acquires a local response sentence so that the determined response sentence is transmitted to the occupant of the vehicle M. The second information acquisition unit 153 may determine and acquire a local response sentence that calls the occupant's name or makes the occupant speak in a manner similar to that of the occupant. The second information acquisition unit 153 outputs the local response statement to the determination unit 155.

The communication quality acquisition unit 154 acquires the communication quality as the communication quality information of the in-vehicle communication device 60. The communication quality of the in-vehicle communication device 60 is, for example, a quality based on information such as a transmission / reception state of the vehicle M, a transmission / reception state of the agent server 200, presence / absence of communication restriction, reception success / failure, and timeout. The communication quality acquired by the communication quality acquisition unit 154 may be, for example, the reception quality of the in-vehicle communication device 60 or the reception quality of the information transmitted by the agent server 200 in the agent function unit 150. Alternatively, the quality of communication between the vehicle-mounted communication device 60 and the agent server 200, for example, the transmission quality of the agent server 200 may be used. Further, the communication quality acquisition unit 154 acquires the determination process information added to the server response statement acquired by the first information acquisition unit 152, and outputs it to the determination unit 155.

When the determination unit 155 determines that the local response sentence output by the second information acquisition unit 153 is a prompt answer response sentence, the determination unit 155 outputs the prompt answer response sentence to the output control unit 156. The prompt answer response sentence is a response sentence to the recognition result that has a relatively simple content and has no difference or is difficult to attach to the local response sentence and the server response sentence as the response content to the recognition result recognized by the voice acquisition unit 151. is there. Examples of the prompt response sentence include a recognition result regarding the operation of the vehicle device, for example, a command response sentence regarding the operation of the vehicle device such as "open the window" in response to the recognition result "open the window". .. Alternatively, as an immediate answer response sentence, for example, a simple response sentence such as "9:15" may be given to the recognition result of a simple question such as "what time is it now?". As the response sentence other than the prompt answer sentence, for example, there are innumerable expressions such as a response sentence such as weather information, which is difficult to generate only with the information stored in the agent server 200, a place name, a place name, and a song name. In addition, there is a response statement that requires a huge dictionary database to understand the intention.

When the response sentence output by the second information acquisition unit 153 is a command response sentence related to the operation of the vehicle device, the determination unit 155 outputs the command response sentence to the output control unit 156 and outputs the command information to the in-vehicle device command unit. Output to 157. The command response statement is a response statement that responds to the recognition result recognized as the command statement related to the operation of the vehicle equipment. The command information is information that causes the vehicle equipment to perform an operation corresponding to the command response statement. The command information is, for example, information for opening the window of the vehicle when the command response statement is "open the window".

The determination unit 155 responds to the recognition result acquired by the voice acquisition unit 151 based on the communication quality acquired by the communication quality acquisition unit 154 when the response sentence output by the second information acquisition unit 153 is not an immediate answer response sentence. It is determined whether the statement is a server response statement output by the first information acquisition unit 152 or a local response statement output by the second information acquisition unit 153. When the communication quality acquired by the communication quality acquisition unit 154 exceeds the first determination quality, the determination unit 155 outputs a response sentence to the recognition result acquired by the voice acquisition unit 151 to the server output by the first information acquisition unit 152. Make it a response statement. When the communication quality acquired by the communication quality acquisition unit 154 is equal to or lower than the first determination quality, the determination unit 155 outputs a response sentence to the recognition result acquired by the voice acquisition unit 151 to the local output by the second information acquisition unit 153. Make it a response statement. As an example of the case where the communication quality is equal to or lower than the first judgment quality, the waiting time set until the server response statement is acquired before the first information acquisition unit 153 acquires the server response statement exceeds a predetermined time. In that case, if the transmission / reception status of the vehicle M or the agent server 200 is poor or the communication is restricted and the server response statement cannot be received, the server response statement can be received, but the received server response statement is incomplete. There are cases.

When the determination unit 155 determines that the communication quality acquired by the communication quality acquisition unit 154 exceeds the first determination quality and the response statement is the server response statement, the determination unit 155 is based on the determination process information output by the communication quality acquisition unit 154. , It is determined whether or not the communication quality is equal to or lower than the second determination quality. As an example of the case where the communication quality is equal to or lower than the second determination quality, for example, there is a case where sufficient information is not obtained in the process of generating the server response statement in the agent server 200. When the determination unit 155 determines that the communication quality is equal to or lower than the second determination quality, the determination unit 156 causes the output control unit 156 to output noise information.

The determination unit 155 may use either the server response statement or the local response statement as the response statement for the recognition result, or both. When the response statement for the recognition result is both the server response statement and the local response statement, the determination unit 155 may determine whether to output the server response statement or the local response statement first based on an appropriate criterion.

The output control unit 156 is determined by the determination unit 155 as a response sentence to the recognition result, and outputs a response sentence to be output by the speaker 30 to the management unit 110. For example, when the determination unit 155 determines that the local response statement is to be output, the output control unit 156 outputs the local response statement as internal response information to the management unit 110. The management unit 110, which has output the internal response information by the output control unit 156, uses the voice control unit 118 to control the speaker 30 so as to output a local response statement. Further, when the determination unit 155 determines that the server response statement is to be output, the output control unit 156 outputs the server response statement to the management unit 110 as external response information. The management unit 110 to which the external response information is output by the output control unit 156 controls the display / operation device 20 by using the display control unit 116 or uses the voice control unit 118 so as to output the server response statement. It controls the speaker 30. In this way, the output control unit 156 causes the display / operation device 20 to display the local response statement or the server response statement determined by the determination unit 155, and outputs the local response statement to the speaker 30.

When the output control unit 156 outputs the internal response information for outputting the local response statement to the management unit 110, the output control unit 156 also outputs the noise information for outputting the noise (noise sound) as a sound effect to the speaker 30. To do. When the determination unit 155 determines that the communication quality is equal to or lower than the second determination quality, the output control unit 156 outputs the external response information for outputting the server response statement to the management unit 110. Noise information for outputting noise as a sound effect is also output to 30. The voice control unit 118 of the management unit 110, which outputs the noise information together with the internal response information or the external response information by the output control unit 156, outputs the local response statement or the server response statement by superimposing (overlapping) the noise. Controls the speaker 30. The output control unit 156 outputs information for outputting a sound effect such as noise before or after the local response statement or the server response statement instead of outputting the local response statement or the server response statement by superimposing noise. It may be.

When the prompt answer response sentence is output by the second information acquisition unit 153, the output control unit 156 outputs the prompt answer response sentence to the management unit 110 as internal response information. The management unit 110 to which the internal response information is output by the output control unit 156 controls the speaker 30 so as to output a command response sentence by using the voice control unit 118.

The in-vehicle device command unit 157 controls the vehicle device 50 based on the command information when the command information is output by the second information acquisition unit 153. The control of the vehicle equipment 50 here includes, for example, opening / closing of doors, opening / closing of windows, control of seat position, and the like. The vehicle device 50 may be controlled by specifying the control target, for example, opening and closing the door of the driver's seat.

[Agent server]
FIG. 3 is a diagram showing a configuration of the agent server 200 and a part of the configuration of the voice dialogue device 100. Hereinafter, the operation of the agent function unit 150 and the like together with the configuration of the agent server 200 will be described. Here, the description of the physical communication from the voice dialogue device 100 to the network NW will be omitted.

The agent server 200 includes a communication unit 210. The communication unit 210 is a network interface such as a NIC (Network Interface Card). Further, the agent server 200 includes, for example, a natural language processing unit 222, a dialogue management unit 224, a network search unit 226, and a response sentence generation unit 228. These components are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium (non-transient) such as a DVD or a CD-ROM. It is stored in a sex storage medium) and may be installed by attaching the storage medium to a drive device.

The agent server 200 includes a storage unit 250. The storage unit 250 is realized by the above-mentioned various storage devices included in the agent server 200. Data and programs such as a personal profile 252, a dictionary DB 254, a knowledge base DB 256, and a response rule DB 258 are stored in the storage unit 250. The data and programs such as the personal profile 252, the dictionary DB 254, the knowledge base DB 256, and the response rule DB 258 correspond to the in-vehicle agent application. Although the personal profile 252 is for each of the plurality of users, the agent function unit 150 stores information corresponding to the personal profile of the user of the vehicle M. Further, the local dictionary DB 162, the local knowledge base DB 164, and the local response rule 166 DB stored in the storage unit 160 of the agent function unit 150 are simpler than the dictionary DB 254, the knowledge base DB 256, and the response rule DB 258. Therefore, the local response statement that can be generated by the agent function unit 150 is simpler than the server response statement generated by the agent server 200, and is likely to be a response statement with low accuracy as a response to the user's request. ..

When the agent function unit 150 recognizes a voice command capable of local processing (processing that does not go through the agent server 200) in the voice dialogue device 100, the agent function unit 150 may perform the processing requested by the voice command. The voice command capable of local processing is a voice command that can be answered by referring to the storage unit 160 included in the voice dialogue device 100, or a voice command that controls the vehicle device 50 (for example, turning on the air conditioner). Command, etc.) Therefore, the agent function unit 150 has a part of the functions included in the agent server 200.

The natural language processing unit 222 interprets the meaning of the character information transmitted by the agent function unit 150 with reference to the dictionary DB 254. The dictionary DB 254 is associated with abstract semantic information with respect to character information. The dictionary DB 254 may include list information of synonyms and synonyms. A voice recognition unit may be provided in the agent server 200, a voice stream may be transmitted from the agent function unit 150, the voice recognition unit may perform voice recognition, convert it into text, and perform processing in the natural language processing unit 222. In this case, the processing of the voice recognition unit and the processing of the natural language processing unit 222 are not clearly separated in stages, and the voice recognition unit corrects the recognition result in response to the processing result of the natural language processing unit 222. It may be done by interacting with each other.

The natural language processing unit 222 generates a command in which the obtained recognition result is replaced with standard character information in the same manner as the voice acquisition unit 151, for example. Even if the natural language processing unit 222 and the voice acquisition unit 151 recognize the meaning of character information by using artificial intelligence processing such as machine learning processing using probability, or generate a command based on the recognition result, for example. Good.

The dialogue management unit 224 determines a response sentence to the occupant of the vehicle M based on the processing result (command) of the natural language processing unit 222 with reference to the personal profile 252, the knowledge base DB 256, and the response rule DB 258. The personal profile 252 includes occupant's personal information, hobbies and preferences, history of past dialogues, etc. stored for each occupant. The knowledge base DB 256 is information that defines the relationships between things. The response rule DB 258 is information that defines the actions (answers, device control contents, etc.) that the agent should perform in response to the command.

Further, the dialogue management unit 224 may identify the occupant by collating with the personal profile 252 using the feature information obtained from the voice stream. In this case, in the personal profile 252, for example, personal information is associated with voice feature information. The voice feature information is, for example, information on speaking features such as voice pitch, intonation, and rhythm (sound pitch pattern), and feature quantities based on Mel Frequency Cepstrum Coefficients and the like. .. The voice feature information is, for example, information obtained by having the occupant utter a predetermined word or sentence at the time of initial registration of the occupant and recognizing the uttered voice.

The dialogue management unit 224 causes the network search unit 226 to perform a search when the command requests information that can be searched via the network NW. The network search unit 226 accesses various web servers 500 via the network NW and acquires desired information. The "information searchable via the network NW" may be, for example, an evaluation result by a general user of a restaurant in the vicinity of the vehicle M, or a weather forecast according to the position of the vehicle M on that day.

The dialogue management unit 224 generates decision process information indicating the decision process until the response sentence is decided. The dialogue management unit 224 determines whether or not sufficient information for determining the server response statement has been obtained in the process of determining the response statement. For example, if the agent server 200 cannot communicate with the various web servers 500 and cannot obtain the information to be obtained from the various web servers 500, it is determined that sufficient information has not been obtained, and the server responds. Generates decision-making process information that indicates that sufficient information was not obtained during the statement generation process.

The response sentence generation unit 228 generates a server response sentence and transmits it to the voice dialogue device 100 so that the response sentence determined by the dialogue management unit 224 is transmitted to the occupants of the vehicle M. The response sentence generation unit 228 generates a server response sentence that calls the name of the occupant or makes the occupant speak in a manner similar to the occupant's speech when the occupant is identified as the occupant registered in the personal profile. You may.

When the agent function unit 150 generates a local response sentence or acquires a server response sentence, the agent function unit 150 instructs the voice control unit 118 to perform voice synthesis and output the voice. Further, the agent function unit 150 instructs the display control unit 116 to display the image of the agent in accordance with the audio output. In this way, the agent function in which the virtually appearing agent responds to the occupant of the vehicle M is realized.

[Processing in the voice dialogue device 100]
Next, an example of processing in the voice dialogue device 100 will be described. The voice dialogue device 100 starts communication with the agent server 200 when the occupant of the vehicle M starts the dialogue. The agent server 200 generates an answer and provides it to the voice dialogue device 100. The voice dialogue device 100 presents the answer obtained from the agent server 200 to the occupant in the form of voice output or image display.

4 to 6 are flowcharts showing an example of the flow of processing executed in the voice dialogue device 100. In the voice dialogue device 100, the WU determination unit 114 detects the voice section of the voice uttered by the occupant, and determines whether or not the wake-up word (WU word) is acquired from the text information obtained by converting the detected voice section into text. (Step S101).

If it is determined that the wakeup word has not been acquired, the WU determination unit 114 repeats the process of step S101. When the WU determination unit 114 determines that the wakeup word has been acquired, the voice acquisition unit 151 determines whether or not the voice information has been acquired (step S103). If it is determined that the voice information has not been acquired, the voice acquisition unit 151 repeats the process of step S103.

When it is determined that the voice information has been acquired, the voice acquisition unit 151 recognizes the character information and transmits the character information as the recognition result to the agent server 200 (step S105). Subsequently, the second information acquisition unit 153 generates and acquires a local response sentence based on the character information recognized by the voice acquisition unit 151 (step S107).

Subsequently, the determination unit 155 determines whether or not the local response sentence acquired by the second information acquisition unit 153 is an immediate answer response sentence (step S109). When it is determined that the local response statement is not an immediate response statement, the determination unit 155 performs a process of determining the response statement (step S111). The process of determining the response statement in step S111 will be described later.

Subsequently, the output control unit 156 outputs the local response statement or the server response statement determined by the determination unit 155 to the management unit 110 as a response statement (step S113), displays the response statement on the display / operation device 20, and causes the speaker. Output to 30. When the determination unit 155 determines that the noise information is to be output, the output control unit 156 outputs the noise information to the management unit 110 together with the local response statement, and displays the response statement on the display / operation device 20. Noise is superimposed on the response text and output to the speaker 30. In this way, the voice dialogue device 100 ends the process shown in FIG.

In step S109, when the determination unit 155 determines that the acquired local response statement is a prompt response statement, the determination unit 155 sets the local response statement as the response statement (step S115). Subsequently, the second information acquisition unit 153 also determines whether or not the prompt answer response sentence is a command response sentence (step S117). When it is determined that the prompt answer response sentence is a command response sentence, the second information acquisition unit 153 outputs the command information to the in-vehicle device command unit 157 (step S119). The in-vehicle device command unit 157 controls the vehicle device 50 based on the command information output by the second information acquisition unit 153. When it is determined that the prompt answer response sentence is not a command response sentence, the second information acquisition unit 153 skips the process of step S119. After that, the output control unit 156 outputs the local response sentence (quick answer response sentence) as a response sentence to the management unit 110 (step S113), displays the response sentence on the display / operation device 20, and outputs the response sentence to the speaker 30. In this way, the voice dialogue device 100 ends the process shown in FIG.

Subsequently, the process shown in step S111 will be described with reference to FIG. In the process shown in step S111, as shown in FIG. 5, the first information acquisition unit 152 determines whether or not the server response statement transmitted by the agent server 200 has been received and acquired (step S201).

When it is determined that the server response statement transmitted by the agent server 200 has not been acquired, the first information acquisition unit 152 determines whether or not the determination time has elapsed after transmitting the recognition result (step S203). ). If it is determined that the determination time has not elapsed, the first information acquisition unit 152 repeats the process according to step S201.

When it is determined that the determination time has elapsed, the first information acquisition unit 152 determines that the communication quality is equal to or lower than the first determination quality, and determines the local response statement acquired by the second information acquisition unit 153 as the response statement. At the same time, it is determined that the output control unit 156 outputs noise information (step S205). In this way, the voice dialogue device 100 ends the process shown in FIG. 5 and proceeds to the process of step S113 shown in FIG.

When the first information acquisition unit 152 determines in step S201 that the server response statement transmitted by the agent server 200 has not been acquired, the communication quality acquisition unit 154 acquires the communication quality of the vehicle-mounted communication device 60 (step). S207). Subsequently, the determination unit 155 determines whether or not the communication quality acquired by the communication quality acquisition unit 154 is equal to or lower than the second determination quality based on the determination process information output by the communication quality acquisition unit 154 (step). S209).

When the communication quality acquisition unit 154 determines that the communication quality acquired is equal to or lower than the second determination quality, the determination unit 155 determines the server response statement acquired by the second information acquisition unit 153 as a response statement and outputs control. It is determined that the noise information is output to the unit 156 (step S211). In this way, the voice dialogue device 100 ends the process shown in FIG. 5 and proceeds to the process of step S113 shown in FIG.

When it is determined that the communication quality acquired by the communication quality acquisition unit 154 is not less than or equal to the second determination quality (exceeds the second determination quality), the determination unit 155 responds to the server response statement acquired by the first information acquisition unit 152. (Step S213). In this way, the voice dialogue device 100 ends the process shown in FIG. 5 and proceeds to the process of step S113 shown in FIG.

In the voice dialogue device 100 of the embodiment, the local response sentence is a response sentence generated by using the local dictionary DB 162, the local knowledge base DB 164, and the local response rule DB 166 stored in the voice dialogue device 100, and is generated by the agent server 200. It is likely that the response to the user's request is less accurate than the generated server response statement.

For example, as shown in FIGS. 6 and 7, it is assumed that the user U asks "Tomorrow's weather" as a request through dialogue. In response to this question, when the response statement is a server response statement, for example, as shown in FIG. 6, the display / operation device 20 is displayed with the agent image E as "clear in the morning, cloudy in the afternoon, and in the evening." The text character "There will be a shower" is displayed, and the voice corresponding to the text character displayed on the display / operation device 20 is output from the speaker 30.

On the other hand, when the response statement is a local response statement, for example, as shown in FIG. 7, the display / operation device 20 displays the text character “cloudy” together with the agent image E, and the speaker 30 Outputs the voice corresponding to the text characters displayed on the display / operation device 20. As described above, when the response statement is a local response statement, the response statement is less complete than when the response statement is a server response statement, and the accuracy as a response to the user's request is low.

In the voice dialogue device 100, noise is superimposed when the local response sentence output by the speaker 30 is output. Specifically, as shown in FIG. 6, when the server response statement is the answer, the voice is output without adding noise, whereas when the local response statement shown in FIG. 7 is answered, the voice is output. , "Za" and "Za" noises are added to output sound from the speaker 30. Further, even when the response statement is a server response statement, for example, when the agent server 200 does not obtain sufficient information in the process of generating the server response statement, the noises of "Za" and "Za" are similarly obtained. Is given to output sound from the speaker 30. As described above, when the voice dialogue device 100 outputs the response sentence determined by utilizing the maximum capacity of the voice dialogue system 1, the response sentence is not added with noise and the question is asked again. If there is room for improvement, the speaker 30 is made to output sound with noise. Therefore, when the local response statement is output, the response statement output by the speaker 30 is not a highly accurate response statement as an answer to the user's request, but a better response statement is obtained by asking the question again, for example. The user can be made aware that there is room for acquisition. In this way, the voice dialogue device 100 can inform the user of the accuracy as a response to the user's request.

In the above embodiment, whether or not to add noise to the local response statement is determined regardless of the content of the response statement, but whether or not to add noise in consideration of the content of the response statement is determined. You may judge. For example, if the response statement contains negative expressions such as "I don't know" or "I can't", noise may be added to the response statement, or such negative expressions may be added to the local response statement. If is included, noise may be added to the local response statement. In addition, when adding noise to the response sentence, noise may be superimposed on the response sentence, and noise sounds may be included before and after the response sentence. Further, instead of adding noise as a sound effect, a chime sound may be output. Further, the voice dialogue device 100 may have a dialogue with the user without the appearance of the agent image. Further, when outputting the sound effect, a display that gives a negative impression to the user is displayed, such as displaying the agent image E shown in FIG. 7 with a cloudy face or changing the font of the text character to a dark impression font. You may do it.

In the above embodiment, the voice dialogue device 100 is mounted on the vehicle M, but may be other than the one mounted on the vehicle M. For example, the voice dialogue device 100 is mounted on a smartphone, a smart speaker, or the like. May be good. Further, in the above embodiment, the voice dialogue device 100 adds noise to the local response statement and outputs the noise together with the local response statement, but the noise is output at a time other than when the response statement is output. It may be. For example, the voice dialogue device 100 may output noise during the processing period of the system or while waiting for voice recognition. By outputting noise, it is possible to remind the user that the accuracy of the response to the user's request is low.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 ... Agent system 20 ... Display / operation device 30 ... Speaker 50 ... Vehicle equipment 60 ... In-vehicle communication device 100 ... Voice dialogue device 150 ... Agent function unit 151 ... Voice acquisition unit 152 ... First information acquisition unit 153 ... Second information acquisition Unit 154 ... Communication quality acquisition unit 155 ... Judgment unit 156 ... Output control unit 157 ... In-vehicle device command unit 200 ... Agent server M ... Vehicle

Claims (8)

A voice acquisition unit that acquires voice information,
A first information acquisition unit that receives and acquires external response information to the voice information acquired by the voice acquisition unit from an external device.
A storage unit that stores internal response information for specific voice information,
A second information acquisition unit that acquires internal response information to the voice information acquired by the voice acquisition unit from the storage unit.
An output control unit that outputs at least one of the external response information and the internal response information to the output unit.
With
When the output control unit outputs the internal response information, the output control unit outputs a sound effect to the output unit together with the internal response information.
Voice dialogue device. A determination unit that determines whether to output the internal response information or the external response information,
A communication quality acquisition unit for acquiring communication quality information regarding the quality of communication between the first information acquisition unit and the external device is further provided.
The determination unit determines the response information to be output based on the communication quality information acquired by the communication quality acquisition unit.
The output control unit causes the output unit to output the internal response information or the external response information determined by the determination unit.
The voice dialogue device according to claim 1. The determination unit determines that the internal response information is output when the communication quality information acquired by the communication quality acquisition unit is equal to or lower than the first determination quality.
The voice dialogue device according to claim 2. The determination unit determines that the external response information is output when the communication quality information acquired by the communication quality acquisition unit exceeds the first determination quality.
When the communication quality information acquired by the communication quality acquisition unit is equal to or lower than the second determination quality, the output control unit outputs a sound effect to the output unit together with the external response information.
The voice dialogue device according to claim 3. The voice dialogue device according to any one of claims 1 to 4, wherein the output control unit outputs the sound effect overlaid on the internal response information. Installed in vehicles equipped with in-vehicle devices,
When the output control unit outputs information about the in-vehicle device as the internal response information, the output control unit does not output the sound effect.
The voice dialogue device according to any one of claims 1 to 5. The computer of the voice dialogue device
Get voice information,
Receives and acquires external response information to the voice information from an external device,
Acquire communication quality information regarding the quality of communication with the external device,
The internal response information for the voice information is acquired from the storage unit that stores the internal response information for the specific voice information.
At least one of the external response information and the internal response information is output to the output unit.
When the internal response information is output, the sound effect is output to the output unit together with the internal response information.
Voice dialogue method. On the computer of the voice dialogue device,
Get voice information,
External response information to the voice information is received from an external device and acquired.
Acquire communication quality information regarding the quality of communication with the external device.
The internal response information for the voice information is acquired from the storage unit that stores the internal response information for the specific voice information.
A process of outputting at least one of the external response information and the internal response information to the output unit is executed.
When the internal response information is output, a process of outputting a sound effect to the output unit together with the internal response information is executed.
program.

Images