JP2021089310A - Voice operation device, voice operation system and voice operation method - Google Patents

Abstract

To provide technology for improving usability while an excess increase of a communication load and a processing load is suppressed in a voice operation device which transmits voice data of speech uttered by a user to a voice recognition engine and operates a related apparatus according to an operation command generated on the basis of a voice recognition processing result by the voice recognition engine.SOLUTION: A voice operation device comprises: a voice acquisition part for acquiring voice data on speech uttered by a user; a related word detection part for detecting an operation related word related to a related apparatus operated in accordance with a previous operation command when voice data which the voice acquisition part newly acquires is voice data without a starting word; and a voice data transmission part for transmitting voice data without a starting word to a voice recognition engine when the related word detection part detects the operation related word from the voice data without a starting word.SELECTED DRAWING: Figure 12

Description

The present invention relates to a voice control device, a voice control system, and a voice control method.

There is known a voice control device that operates an in-vehicle device by recognizing a voice of an occupant. Further, a voice dialogue system that performs voice dialogue between an in-vehicle terminal and a server is known (see, for example, Patent Documents 1 and 2).

Further, an intelligent personal assistant (also referred to as an AI assistant) that performs a task or a service in response to a request from a user's utterance is known.

Japanese Unexamined Patent Publication No. 2005-250379 Japanese Unexamined Patent Publication No. 2001-249685

When voice recognition of the user's utterance content using a voice recognition engine such as AI Assistant, the user uses "Alexa (registered trademark)", "Hey Siri (registered trademark)", etc.
After issuing a specific activation word (WuW: Wake-up Word), it is necessary to issue a request such as "○○". That is, the user needs to utter the activation word every time the voice is input, and it cannot be said that the usability is high. On the other hand, if an attempt is made to voice-recognize all utterances that do not include the activation word by the user, there is a concern that the communication load and the processing load will increase excessively.

Therefore, an object of the present invention is a voice operation in which voice data of a speech uttered by a user is transmitted to a voice recognition engine and a related device is operated according to an operation command generated based on a voice recognition processing result by the voice recognition engine. The purpose of the device is to provide a technique for improving usability while suppressing an excessive increase in communication load and processing load.

In order to solve the above problems, the present invention adopts the following configuration. That is, the present invention transmits to the voice recognition engine at least the voice data including the speech following the activation word when the voice data of the speech uttered by the user includes an activation word for activating the speech recognition engine. , A voice operation device that operates related devices in response to operation commands generated based on the result of voice recognition processing by the voice recognition engine, and a voice acquisition unit that acquires voice data related to speech made by the user. When the voice data newly acquired by the voice acquisition unit is voice data without a start word that does not include a start word, the start that newly acquires an operation-related word related to the related device operated in response to the previous operation command. When the related word detection unit that detects from the wordless voice data and the related word detection unit detect the operation related word from the activation wordless voice data, the voice that transmits the activation wordless voice data to the voice recognition engine. It includes a data transmission unit.

Further, the voice operation device according to the present invention further includes a voice storage unit that stores voice data acquired by the voice acquisition unit, and the related word detection unit is the previous voice data stored in the voice storage unit. Therefore, the related device operated in response to the previous operation command may be specified by detecting a predetermined device-specific related word associated with each related device.

Further, the voice operation device according to the present invention further includes a command information storage unit that stores operation command information related to the operation command generated based on the voice recognition processing result of the voice data by the voice recognition engine, and detects the related word. The unit may specify the related device operated in response to the previous operation command based on the operation command information stored in the command information storage unit.

Further, the present invention can be specified as a voice operation system. That is, the voice operation system according to the present invention transmits to the voice operation device based on any of the above-mentioned voice operation devices, the voice recognition engine, and the voice recognition processing result of the voice data by the voice recognition engine. Includes an operation command generation server that generates operation commands for.

Further, the present invention can be specified as a voice operation method. That is, the present invention transmits to the voice recognition engine at least the voice data including the speech following the activation word when the voice data of the speech uttered by the user includes an activation word for activating the speech recognition engine. , A voice operation method in which the voice operation device executes the operation of the related device in response to the operation command generated based on the voice recognition processing result by the voice recognition engine, and acquires the voice data related to the speech uttered by the user. When the voice acquisition process and the voice data newly acquired in the voice acquisition process are voice data without a start word that does not include a start word, operation-related words related to the related device operated in response to the previous operation command are displayed. When the related word detection step of detecting from the newly acquired voice data without activation word and the operation related word are detected from the voice data without activation word in the related word detection step, the voice data without activation word is recognized by the voice. Includes a voice data transmission process to be transmitted to the engine. In the above-mentioned voice operation method, the computer of the voice operation device executes the processing related to each of the above-mentioned steps.

Further, the present invention may be a program for causing a computer to execute a process related to each step in the voice operation method. Further, the present invention may be a storage medium in which the program can be read by a computer and stored non-temporarily.

According to the present invention, in a voice operation device that transmits voice data of a speech uttered by a user to a voice recognition engine and operates a related device in response to an operation command generated based on the voice recognition processing result by the voice recognition engine. , It is possible to provide a technique for improving usability while suppressing an excessive increase in communication load and processing load.

FIG. 1 is a schematic configuration diagram of a voice operation system according to the first embodiment. FIG. 2 is a hardware configuration diagram of the voice operation device according to the first embodiment. FIG. 3 is a hardware configuration diagram of the voice recognition engine according to the first embodiment. FIG. 4 is a hardware configuration diagram of the command generation server according to the first embodiment. FIG. 5 is a diagram showing an example of the utterance content of the speaker and the utterance content of the voice assistant. FIG. 6 is a functional block diagram of the voice operation device according to the first embodiment. FIG. 7 is a functional block diagram of the voice recognition engine according to the first embodiment. FIG. 8 is a functional block diagram of the command generation server according to the first embodiment. FIG. 9 is a diagram showing an example of a data structure of operation command data. FIG. 10 is a diagram showing an example of the data structure of the device-specific related word definition table TB1. FIG. 11 is a diagram showing an example of a data structure of operation command data. FIG. 12 is a flowchart showing a control content when the voice operation device transmits voice data to the voice recognition engine.

<Embodiment 1>
Hereinafter, embodiments of the present invention will be exemplified with reference to the drawings.

FIG. 1 is a schematic configuration diagram of the voice operation system 100 according to the first embodiment. As shown in FIG. 1, the voice operation system 100 includes a voice operation device 10, a voice recognition engine 30, and a command generation server 50. The voice control device 10, the voice recognition engine 30, and the command generation server 50 are connected to each other so as to be able to communicate with each other via the communication line N. The communication line N is, for example, a communication network such as the Internet. Further, the communication line N may be at least a part of a line using a wireless communication method such as WiFi or LTE. In the present embodiment, a large number of voice operation devices 10 are connected to the voice recognition engine 30 and the command generation server 50 via the communication line N.

The voice control device 10 is an in-vehicle device mounted on the vehicle 1, and is a control device for operating various related devices 2 mounted on the vehicle 1 by voice input. The voice control device 10 may form, for example, a part of an AVN machine (in-vehicle audio / visual / navigation integrated machine). The related device 2 (see FIG. 2) is a device to be operated by the voice control device 10, and examples thereof include an air conditioner (air conditioner) 2A, an audio device 2B, and a headlight (lighting device) 2C. However, the related device 2 is not limited to the above example, and the related device 2 may include a wiper, a blinker, a power window, a door lock device of the vehicle 1, and the like.

The voice control device 10 is connected to each related device 2 via an in-vehicle network (CAN, LIN, etc.). For example, the voice control device 10 controls each related device 2 according to the utterance content of the user (speaker) who gets on the vehicle 1. Further, the vehicle 1 is provided with a microphone (microphone) 3 and a speaker 4, and these are connected to the voice operation device 10 via an in-vehicle network. The positions and number of microphones 3 and speakers 4 provided in the vehicle 1 are not particularly limited.

FIG. 2 is a hardware configuration diagram of the voice operation device 10 according to the first embodiment. The voice control device 10 is a computer having a processor 12, a memory 13, an input / output IF (interface) 14, and a communication IF (interface) 15 connected to each other by a connection bus 11. The processor 12 is a central processing unit that controls the entire device by processing the input information and outputting the processing result. The processor 12 is also called a CPU (Central Processing Unit) or an MPU (Micro-processing unit). The processor 12 is not limited to a single processor, and may have a multiprocessor configuration. Further, it may be a multi-core configuration having a plurality of cores in a single chip connected by a single socket.

The memory 13 includes, for example, a main storage device and an auxiliary storage device. The main storage device is used as a work area of the processor 12, a storage area for temporarily storing information processed by the processor 12, and a buffer area for communication data. The main storage device is a storage medium for the processor 12 to cache programs and data and expand a work area. The main storage device includes, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory. The auxiliary storage device is a storage medium that stores a program executed by the processor 12, data used for information processing, operation setting information, and the like. The auxiliary storage device is, for example, an HDD (Hard-disk Drive), an SSD (Solid State Drive), an EPROM (Erasable Programmable ROM), a flash memory, a USB memory, a memory card, or the like. Further, the auxiliary storage device in the memory 13 is provided with a voice recognition dictionary DB as a storage destination of data referred to by each processing unit of the voice operation device 10.

The input / output IF 14 is an interface for inputting / outputting data to / from each related device 2 connected to the voice operating device 10. The voice control device 10 is connected to a microphone 3, a speaker 4, or the like via, for example, an input / output IF14. The microphone 3 is an input device for inputting voices (utterances) emitted by the occupants of the vehicle 1. The information input from the microphone 3 is notified to the processor 12 via the connection bus 11. The speaker 4 is an output device that outputs audio data processed by the processor 12 or the like as audio.

The communication IF 15 is an interface for communicating with another device via the communication line N. The communication IF15 may perform communication by a wireless communication method such as WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution), WiFi, or Bluetooth (registered trademark).

Although the specific processing content will be described later, in the voice operation device 10, a predetermined activation word for activating the voice recognition engine 30 is added to the voice data (speech data) of the speech uttered by the occupant (user, speaker) of the vehicle 1. If it is included, at least the voice data including the speech following the activation word is transmitted to the voice recognition engine 30, and the operation command generated based on the voice recognition processing result by the voice recognition engine 30 is acquired and acquired. The related device 2 is operated according to the operation command. The activation word is a specific word predetermined in association with the voice recognition engine 30 or the like, and serves as a trigger for transmitting voice data from the voice operation device 10 in order to start the voice recognition engine 30 in the standby state, for example. .. The activation word may also be referred to as WuW (Wake Up Word), wake up word, or wake word. Examples of such an activation word include "Alexa (registered trademark)" and "Hey Siri (registered trademark)".
In addition, in this specification, the "starting word" is not limited to one word, and may be a term, a phrase or the like including a plurality of words. Further, as the "startup word", various things such as "hello, my car", "high, my vehicle" and the like can be adopted.

In the voice operation system 100 according to the present embodiment, voice data is transmitted from the voice operation device 10 to the voice recognition engine 30 triggered by the occupant issuing an activation word as described above. FIG. 3 is a hardware configuration diagram of the voice recognition engine 30 according to the first embodiment. The voice recognition engine 30 is a computer having a processor 32, a memory 33, a communication IF 34, and the like connected to each other by a connection bus 31. The processor 32 is a central processing unit that controls the entire device by processing the input information and outputting the processing result. The processor 32 is also called a CPU (Central Processing Unit) or an MPU (Micro-processing unit). The processor 32 is not limited to a single processor, and may have a multiprocessor configuration. Further, it may be a multi-core configuration having a plurality of cores in a single chip connected by a single socket.

The memory 33 includes a main storage device and an auxiliary storage device. The main storage device is used as a work area of the processor 32, a storage area for temporarily storing information processed by the processor 32, and a buffer area for communication data. The main storage device is a storage medium for the processor 32 to cache programs and data and expand a work area. The main storage device includes, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory. The auxiliary storage device is a storage medium that stores a program executed by the processor 32, data used for information processing, operation setting information, and the like. The auxiliary storage device is, for example, an HDD (Hard-disk Drive), an SSD (Solid State Drive), an EPROM (Erasable Programmable ROM), a flash memory, a USB memory, a memory card, or the like. The communication IF 34 is an interface for communicating with another device via the communication line N. The voice recognition engine 30 may exist on the cloud.

The voice recognition engine 30 that receives the voice data from the voice control device 10 converts the voice data into text (character string) data representing the utterance content of the speaker by performing voice recognition processing on the voice data. Further, the voice recognition engine 30 performs natural language analysis processing on the obtained text data to generate speaker request data which is data indicating the request contents of the speaker, and commands the speaker request data. It is transmitted to the generation server 50. FIG. 4 is a hardware configuration diagram of the command generation server 50 according to the first embodiment. The command generation server 50 is a computer having a processor 52, a memory 53, a communication IF 54, and the like connected to each other by a connection bus 51. The processor 52 is a central processing unit that controls the entire device by processing the input information and outputting the processing result. The processor 52 is also called a CPU (Central Processing Unit) or an MPU (Micro-processing unit). The processor 52 is not limited to a single processor, and may have a multiprocessor configuration. Further, it may be a multi-core configuration having a plurality of cores in a single chip connected by a single socket.

The memory 53 includes a main storage device and an auxiliary storage device. The main storage device is used as a work area of the processor 52, a storage area for temporarily storing information processed by the processor 52, and a buffer area for communication data. The main storage device is a storage medium for the processor 52 to cache programs and data and expand a work area. The main storage device includes, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory. The auxiliary storage device is a storage medium that stores a program executed by the processor 52, data used for information processing, operation setting information, and the like. The auxiliary storage device is, for example, an HDD (Hard-disk Drive), an SSD (Solid State Drive), an EPROM (Erasable Programmable ROM), a flash memory, a USB memory, a memory card, or the like. The communication IF 54 is an interface for communicating with another device via the communication line N. The command generation server 50 may be configured as a web server existing on the cloud.

The command generation server 50 generates operation command data to be transmitted to the voice operation device 10 based on the speaker request data received from the voice recognition engine 30. The operation command data is data in which information regarding the operation content of the related device 2 to be operated by the voice operation device 10 is stored. Of course, this operation command data reflects the request of the speaker. The operation command data generated by the command generation server 50 is transmitted from the command generation server 50 to the voice operation device 10 via the communication line N. The voice operation device 10 operates the target related device 2 based on the operation command data received from the command generation server 50 in accordance with the operation command data indicating the request contents of the speaker. Further, the command generation server 50 generates response voice generation text data for generating a response voice corresponding to the generated operation command data, and transmits the response voice generation text data to the voice recognition engine 30. The voice recognition engine 30 performs voice synthesis processing on the response voice generation text data received from the command generation server 50, so that the response voice generation text data is voice data (hereinafter referred to as "response voice data"). Convert to. The voice recognition engine 30 transmits the generated response voice data to the voice control device 10. Then, the voice operation device 10 announces to the occupant by outputting the voice corresponding to the response voice data received from the voice recognition engine 30 (hereinafter, referred to as “voice assistant utterance”) to the speaker 4.

Next, specific control contents in the voice operation system 100 will be described. The voice operation system 100 is activated, for example, when the ACC (accessory) power supply of the vehicle 1 is switched on.

FIG. 5 is a diagram showing an example of the utterance content of the speaker and the utterance content of the voice assistant. FIG. 6 is a functional block diagram of the voice operation device 10 according to the first embodiment. In the voice operation device 10, when the processor 12 executes the application program, the voice acquisition unit 21, the activation word detection unit 22, the voice data transmission unit 23, the related word detection unit 24, the command acquisition unit 25, and the response voice acquisition unit are used. It functions as each processing unit such as 26, an operation processing unit 27, and a response voice output unit 28. However, at least a part of the processing of each of the above processing units may be provided by a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), or the like. Further, at least a part of each of the above processing units may be a dedicated large scale integration (LSI) such as a field-programmable gate array (FPGA) or other digital circuits. Further, an analog circuit may be included in at least a part of each of the above processing units.

The voice acquisition unit 21 acquires voice data by receiving the utterance of the occupant (speaker) input to the microphone 3 via the input / output IF14. Here, the memory 13 has a voice storage unit 131 that stores voice data related to the utterance of the occupant acquired from the microphone 3. The voice storage unit 131, which is a storage area, is a storage area allocated to a part of the memory 13. The voice acquisition unit 21 stores the voice data of the speaker acquired from the microphone 3 in the voice storage unit 131.

The activation word detection unit 22 is based on the voice data of the speaker newly acquired by the voice acquisition unit 21 (that is, the latest voice data stored in the voice storage unit 131 of the memory 13), and the activation word is added to the voice data. By determining whether or not is included, the presence or absence of an activation word in the voice data is detected. The activation word detection unit 22 detects the activation word, for example, by performing voice recognition processing on the voice data. The voice recognition process can be performed by referring to the voice recognition dictionary DB (database) according to an algorithm corresponding to a known voice signal matching model. The voice recognition dictionary DB and the voice signal matching model used for the voice recognition process may be stored in, for example, the auxiliary storage device of the memory 13.

In FIG. 5, with the ACC (accessory) power of the vehicle 1 turned on, the occupant "○○ (starting word), raise the temperature of the air conditioner by 3 degrees"("○○" on the left is the starting word (WuW). ) Is shown. The same applies hereinafter.). In this case, the utterance "○○ (starting word), raise the temperature of the air conditioner by 3 degrees" is picked up by the microphone 3, the voice data of the utterance is acquired by the voice acquisition unit 21, and the voice is stored in the memory 13. It is stored in the unit 131. Then, "○○ (starting word)" in the voice data "○○ (starting word), raise the temperature of the air conditioner by 3 degrees" is detected by the starting word detecting unit 22.

The voice data transmission unit 23 performs a process of transmitting voice data to the voice recognition engine 30. When the activation word detection unit 22 detects the activation word, the voice data transmission unit 23 transmits voice data including at least the utterance following the activation word to the voice recognition engine 30. Here, among the voice data, the voice data corresponding to the utterance part following the activation word, that is, the utterance part of "raising the temperature of the air conditioner by 3 degrees" (hereinafter referred to as "request content voice data") is voice data. The transmission unit 23 transmits to the voice recognition engine 30. Further, the voice data transmission unit 23 transmits the ID identification information to the voice recognition engine 30 together with the request content voice data. The request content voice data and ID identification information are transmitted to the voice recognition engine 30 via the communication IF 15 of the voice operation device 10 and the communication line N.

The ID identification information is identification information assigned to each voice operation device 10 in order to identify the voice operation device 10 mounted on the vehicle 1. By transmitting the ID identification information to the voice recognition engine 30 together with the request content voice data, the voice recognition engine 30 can specify the source of the received request content voice data. The related word detection unit 24, the command acquisition unit 25, the operation processing unit 27, and the response voice output unit 28 in the voice operation device 10 will be described later.

FIG. 7 is a functional block diagram of the voice recognition engine 30 according to the first embodiment. In the voice recognition engine 30, the processor 32 executes an application program to function as each processing unit such as a data acquisition unit 36, an analysis processing unit 37, and a voice synthesis unit 38.

The data acquisition unit 36 of the voice recognition engine 30 acquires the request content voice data and the ID identification information transmitted by the voice data transmission unit 23 of the voice operation device 10. Then, the data acquisition unit 36 delivers the request content voice data to the analysis processing unit 37, and the analysis processing unit 37 performs voice recognition processing and natural language analysis processing on the received request content voice data. For example, the analysis processing unit 37 converts the request content voice data into text data by performing voice recognition processing on the request content voice data. Further, the analysis processing unit 37 interprets the intention of the speaker's request included in the text data by performing natural language analysis on the text data. The natural language analysis executed by the analysis processing unit 37 may include morphological analysis processing, syntax analysis processing, semantic analysis processing, context analysis processing, and the like.

As described above, the analysis processing unit 37 generates speaker request data indicating the request content of the speaker based on the request content voice data. The analysis processing unit 37 associates the generated speaker request data with the ID identification information and transmits the generated speaker request data to the command generation server 50. The speaker request data and the ID identification information are transmitted to the command generation server 50 via the communication IF 34 of the voice recognition engine 30 and the communication line N. The function of the voice synthesis unit 38 in the voice recognition engine 30 will be described later. Further, the voice recognition engine 30 may store the request content voice data and the ID identification information received from the voice operation device 10 in the storage area of the memory 33. Further, the analysis processing unit 37 in the voice recognition engine 30 may be realized by artificial intelligence (AI). That is, the machine learning model may be stored in the memory 33 of the voice recognition engine 30, and the analysis processing unit 37 may generate the speaker request data using the machine learning model. In such a configuration, the voice recognition engine 30 is constructed as a so-called AI assistant.

FIG. 8 is a functional block diagram of the command generation server 50 according to the first embodiment. In the command generation server 50, when the processor 52 executes the application program, it functions as each processing unit such as the command generation unit 56 and the response voice text generation unit 57.

The command generation unit 56 of the command generation server 50 generates operation command data to be transmitted to the voice operation device 10 based on the speaker request data and the ID identification information received from the voice recognition engine 30. FIG. 9 is a diagram showing an example of a data structure of operation command data. In the example shown in FIG. 9, the operation command data is stored in association with the related device 2 (Object) to be operated by the operation command, the operation content (Action), and the ID identification information. In the example here, the related device 2 (Object) to be operated is "air conditioner", the operation content (Action) is "raised three times", and the conditioner ID identification information is No1. However, the data structure of the operation command data shown in FIG. 9 is an example, and is not limited to the above example. The command generation unit 56 transmits the operation command data to the voice operation device 10 corresponding to the ID identification information. The operation command data is transmitted to the voice operation device 10 via the communication IF 54 of the command generation server 50 and the communication line N.

Further, the response voice text generation unit 57 transmits the response voice to the voice recognition engine 30 based on the speaker request data received from the voice recognition engine 30 or the operation command data generated by the command generation unit 56. Generate text data for generation. The response voice generation text data is text data corresponding to the operation command data, and is text (character string) data that is the source of the voice assistant utterance to be voice output from the speaker 4 of the voice operation device 10. The text data for generating the response voice includes the related device 2 (Object) to be operated and the operation content (Action) stored in the operation command data. In the example here, the text (character string) "Raise the temperature of the air conditioner by 3 degrees" is included. The response voice text generation unit 57 transmits the response voice generation text data together with the ID identification information to the voice recognition engine 30. The response voice generation text data and the ID identification information are transmitted to the voice recognition engine 30 via the communication IF 54 of the command generation server 50 and the communication line N.

In the voice recognition engine 30 that has received the response voice generation text data and the ID identification information from the command generation server 50, the voice synthesis unit 38 performs voice synthesis processing on the response voice generation text data. As a result, the text data for generating the response voice is converted into the response voice data. The voice synthesis unit 38 of the voice recognition engine 30 transmits the response voice data converted from the response voice generation text data to the voice operation device 10 corresponding to the ID identification information. The response voice data is transmitted to the voice control device 10 via the communication IF 34 of the voice recognition engine 30 and the communication line N.

As described above, the voice operation device 10 receives the response voice data from the voice recognition engine 30, and receives the operation command data from the command generation server 50. Here, the command acquisition unit 25 of the voice operation device 10 receives the operation command data transmitted from the command generation server 50. The memory 13 in the voice operation device 10 has a command information storage unit 132 that stores operation command data (operation command information). The command information storage unit 132 is a storage area allocated to a part of the memory 13. The command acquisition unit 25 of the voice operation device 10 stores the acquired operation command data in the command information storage unit 132.

The operation processing unit 27 of the voice operation device 10 reads the latest operation command data stored in the command information storage unit 132, and operates the related device 2 to be operated according to the operation command data. In the example here, as described with reference to FIG. 9, the related device 2 (Object) to be operated is the “air conditioner 2A”, and the operation content (Action) is the processing content of “raising three times”. There is. Therefore, the operation processing unit 27 outputs a control signal to the air conditioner 2A so that the set temperature of the air conditioner 2A is changed to a temperature 3 degrees higher according to the operation content defined in the operation command data. To do. As a result, the set temperature of the air conditioner 2A is changed according to the operation command data.

Further, the response voice acquisition unit 26 of the voice operation device 10 acquires the response voice data transmitted from the voice recognition engine 30. The response voice acquisition unit 26 delivers the acquired response voice data to the response voice output unit 28. The response voice output unit 28 outputs a control signal to the speaker 4, and causes the speaker 4 to output a voice assistant utterance based on the response voice data received from the response voice acquisition unit 26. In the example here, the voice "Raise the temperature of the air conditioner by 3 degrees" is output from the speaker 4.

In the voice operation system 100 of the present embodiment, the voice recognition engine 30 goes into the standby state again after a certain period of time (for example, several seconds) has elapsed from the end of one session. The term "session" as used herein means that the voice recognition engine 30 is activated by receiving voice data from the voice control device 10, transmits the generated speaker request data to the command generation server 50, and then responds to the voice from the command generation server 50. It refers to a series of processes from receiving the generation text data to transmitting the response voice data generated from the response voice generation text data to the voice operation device 10. Here, the session started by the speaker's utterance "○○ (starting word), raising the temperature of the air conditioner by 3 degrees" up to the above is referred to as a "first session".

Here, in the conventional voice operation system, when the related device is operated by the voice input of the speaker, the speaker utters the activation word (WuW) each time the voice is input, and then the activation word is continued. It was necessary to speak the request contents, and it could not be said that the convenience of the user was high. On the other hand, even when the voice data of the speaker acquired by the voice control device is the voice data without the activation word that does not include the activation word (WuW), the voice data without the activation word is always transmitted to the voice recognition engine 30. Attempting to execute voice recognition processing or language analysis processing on voice data without a start word causes an excessive increase in communication load and processing load. Therefore, in the voice operation device 10 of the present embodiment, even when the speaker omits the utterance of the activation word (WuW), the voice operation of the related device 2 can be performed without causing an excessive increase in the communication load and the processing load. It is possible. Hereinafter, the characteristic processing contents of the voice operation device 10 will be described.

In the example shown in FIG. 5, after the first session is completed, the occupant of the vehicle 1 utters "lower the temperature by 1 degree". This utterance does not include the activation word. In this case, the voice data of the speaker newly acquired by the voice acquisition unit 21 of the voice operation device 10 is the voice data without the activation word that does not include the activation word, and the voice data without the activation word is the voice storage unit 131 of the memory 13. Is remembered in. Therefore, in this case, the activation word detection unit 22 does not detect the activation word from the voice data without the activation word.

In this way, when the voice data of the speaker newly acquired by the voice acquisition unit 21 is the voice data without the activation word, the related word detection unit 24 in the voice operation device 10 operates in response to the previous operation command. The operation-related words related to the related device 2 are detected from the newly acquired voice data without the activation word. Hereinafter, each process in the "second session" started when the speaker utters "lower the temperature by 1 degree" without uttering the activation word will be described.

In the second session, the related word detection unit 24 sets the previous voice data stored in the voice storage unit 131 of the memory 13 (in the control example shown in FIG. 5, “〇〇 (starting word), the temperature of the air conditioner 3”. (Applicable to the utterance "Raise the degree"), by detecting the device-specific related words that are predetermined and related to the related device 2 in association with each related device 2, the operation is performed according to the previous operation command. Identify the related device 2 that has been used.

Here, the device-specific word definition table TB1 shown in FIG. 10 is stored in the memory 13 of the voice control device 10. In the device-specific related word definition table TB1, the correspondence relationship between the related device 2 and the related device 2 that are voice-operated by the voice control device 10 and the predetermined device-specific related words is defined. The related device 2 is an air conditioner 2A, an audio device 2B, a headlight 2C, or the like.

In the example shown in FIG. 10, as the related words for each device corresponding to the air conditioner 2A, "air conditioner (air conditioner)", "temperature", "degree", "raise", "lower", "stop", Words such as "attach" are defined. In addition, as device-specific related words corresponding to audio device 2B, "audio", "CD", "DVD," volume "," volume "," play "," stop "," track up "," track down " Words such as "" are defined. Further, as the device-specific related words corresponding to the headlight 2C, words such as "headlight", "light", "lighting", and "bright and dark" are defined. However, the device-specific related words set for each of these related devices 2 are exemplary.

As described above, the related word detection unit 24 refers to the device-specific related word definition table TB1 stored in the memory 13 and detects the device-specific related word from the previous voice data stored in the voice storage unit 131. To do. In the example here, "air conditioner", "temperature", and "degree" included in the previous utterance "○○ (starting word), raise the temperature of the air conditioner by 3 degrees" are detected as related words for each device. .. Then, the related word detection unit 24 reads out the related device 2 associated with the detected device-specific related words (“air conditioner”, “temperature”, “degree”) by referring to the definition table TB1. As a result, the related word detection unit 24 can identify the related device 2 operated in response to the previous operation command. In the control example here, the air conditioner 2A is specified as the related device 2 operated in response to the previous operation command.

Then, the related word detection unit 24 in the voice operation device 10 detects the operation-related word related to the related device 2 operated in response to the previous operation command from the newly acquired voice data without the activation word. As described above, since the related device 2 operated in response to the previous operation command is specified by the air conditioner 2A, the related word detection unit 24 is the "air conditioner (air conditioner)" related to the air conditioner 2A. , "Temperature", "Degree", "Raise", "Lower", "Stop", "Turn", etc. are included in the newly acquired voice data without activation word. As a result, the related word detection unit 24 detects the words "temperature" and "degree" as operation-related words from the voice data without the activation word related to the utterance "lower the temperature by 1 degree".

As described above, when the related word detection unit 24 detects the operation-related word related to the related device 2 operated in response to the previous operation command from the newly acquired voice data without the activation word, the voice data is transmitted. The unit 23 transmits the voice data without the activation word and the ID identification information to the voice recognition engine 30. In this way, the voice recognition engine 30 in the standby state is started by triggering the transmission of the voice data without the start word and the ID identification information from the voice operation device 10 to the voice recognition engine 30.

Each process executed in the voice recognition engine 30 is substantially the same as in the first session. That is, the analysis processing unit 37 of the voice recognition engine 30 performs voice recognition processing on the voice data without the start word, thereby converting the voice data without the start word "lower the temperature by 1 degree" into text data. , The intention of the speaker's request included in the text data is interpreted by performing natural language analysis processing on this text data. In this way, the analysis processing unit 37 generates speaker request data indicating the content of the speaker's request based on the voice data without the activation word. The speaker request data generated by the voice recognition engine 30 is transmitted to the command generation server 50 in association with the ID identification information, and the command generation server 50 generates operation command data and response voice generation text data.

Here, the speaker request data received by the command generation server 50 from the voice recognition engine 30 does not store information corresponding to the related device 2 (Object) to be operated. Therefore, the command generation server 50 complements the information corresponding to the related device 2 (Object) when generating the operation command data and the text data for generating the response voice.

The memory 53 of the command generation server 50 has a command data storage unit 151 that stores the operation command data generated by the command generation unit 56. The command data storage unit 151 is a storage area allocated to a part of the memory 53. The command generation unit 56 of the command generation server 50 reads the operation command data generated in the previous session from the command data storage unit 151. Then, the command generation unit 56 generates information corresponding to the related device 2 (Object) to be operated, which is blank in the speaker request data newly received from the voice recognition engine 30, in the previous session. Read from the operation command data that was executed and complete it. As a result, the operation command data shown in FIG. 11 is generated. In the control example here, as shown in FIG. 11, the related device 2 (Object) to be operated is “air conditioner” and the operation content (Action) is “lowered once”. In FIG. 11, the parentheses attached to the air conditioner mean that they are complemented from the operation command data generated in the previous session. Then, the command generation unit 56 transmits the generated operation command data to the voice operation device 10 corresponding to the ID identification information as in the first session.

Further, the response voice text generation unit 57 of the command generation server 50 is the response voice generation text data having the content corresponding to the operation command data based on the operation command data generated by the command generation unit 56, as in the first session. Is generated, and the generated text data for generating the response voice is transmitted to the voice recognition engine 30 together with the ID identification information. Then, the voice recognition engine 30 that received the response voice generation text data and the ID identification information from the command generation server 50 generated the response voice data based on the response voice generation text data by the voice synthesis unit 38. The response voice data is transmitted to the voice operation device 10 corresponding to the ID identification information.

Then, the voice operation device 10 that has received the operation command data from the command generation server 50 operates the related device 2 to be operated based on the operation command data, as in the first session. Specifically, the operation processing unit 27 of the voice operation device 10 is an air conditioner so that the set temperature of the air conditioner 2A is set to a temperature one degree lower according to the operation content defined in the operation command data. Operate the conditioner 2A. Further, the response voice acquisition unit 26 of the voice operation device 10 delivers the response voice data acquired from the voice recognition engine 30 to the response voice output unit 28, and the response voice output unit 28 is a speaker based on the response voice data. Let 4 output the voice assistant speech. As a result, the voice "lower the temperature of the air conditioner by 1 degree" is output from the speaker 4. This ends the second session.

Next, the third session will be described with reference to FIG. The third session is started when the occupant says "○○ (startup word), play the CD" when the voice recognition engine 30 is in the standby state after the second session is completed. .. In this way, when the occupant's utterance includes the activation word, the same processing as in the first session is performed. That is, the request content voice data corresponding to the utterance portion of "playing the CD" is transmitted from the voice operation device 10 to the voice recognition engine 30. The processing contents after that are the same as in the first session. That is, finally, the voice operation device 10 receives the operation command data from the command generation server 50 and the response voice data from the voice recognition engine 30. Then, the operation processing unit 27 of the voice operation device 10 outputs a control signal to the audio device 2B to reproduce the CD. Further, the response voice output unit 28 of the voice control device 10 operates the speaker 4 based on the response voice data, so that the voice assistant utterance "play a CD" is output from the speaker 4. This ends the third session.

In the third session, if the occupant says "Turn off the air conditioner" instead of saying "○○ (starting word), play the CD", the starting word is not included in the utterance. The process of detecting the operation-related word from the voice data without the activation word is performed by the related word detection unit 24. In this case, the related word detection unit 24 refers to the device-specific word definition table TB1 stored in the memory 13, and is related to each device from the previous (second session) voice data stored in the voice storage unit 131. Detect words. Specifically, the words "temperature" and "degree" included in the utterance "lower the temperature by 1 degree" are detected as related words for each device, and the related device 2 operated in response to the previous operation command is an air conditioner. Specified as conditioner 2A.

Then, the related word detection unit 24 of the voice operation device 10 acquires the operation-related word related to the related device 2 operated in response to the previous operation command, and newly acquired voice data without an activation word "turn off the air conditioner". Detect from. As a result, the related word detection unit 24 detects the words "air conditioner" and "stop" related to the air conditioner 2A as operation-related words. In this way, the related word detection unit 24 detects the operation-related word from the voice data without the activation word, so that the voice data without the activation word is transmitted to the voice recognition engine 30. The subsequent processing contents are the same as the processing contents explained in the second session, and thus the description thereof will be omitted.

Next, in the third session, a case where the occupant says "play the CD" instead of saying "OO (startup word), play the CD" will be described. In this case, since the utterance does not include the activation word, the related word detection unit 24 performs a process of detecting the operation-related word from the voice data without the activation word. That is, the related word detection unit 24 refers to the device-specific related word definition table TB1 stored in the memory 13, and refers to the device-specific related word from the previous (second session) voice data stored in the voice storage unit 131. Is detected. Specifically, the words "temperature" and "degree" included in the utterance "lower the temperature by 1 degree" are detected as related words for each device, and the related device 2 operated in response to the previous operation command is an air conditioner. Specified as conditioner 2A.

Then, the related word detection unit 24 of the voice operation device 10 newly acquired the operation-related word related to the related device 2 (conditioner 2A) operated in response to the previous operation command, "plays the CD". It is detected from the voice data without the activation word. In this case, since the operation-related word related to the air conditioner 2A is not included in the voice data without the activation word "play the CD", the operation-related word is not detected by the related word detection unit 24. .. Therefore, the voice data without the activation word "play the CD" is not transmitted to the voice recognition engine 30, and the voice recognition engine 30 is not started. That is, in this case, the voice control device 10 does not respond to the utterance of the speaker and does not operate the audio device 2B.

In the above description of the second session, the related word detection unit 24 detects a predetermined device-specific related word associated with each related device 2 from the previous voice data stored in the voice storage unit 131. By doing so, the related device 2 operated in response to the previous operation command is specified, but the related word detection unit 24 may specify the related device 2 operated in response to the previous operation command by another process. For example, the related word detection unit 24 may specify the related device 2 operated in response to the previous operation command based on the operation command data stored in the command information storage unit 132.

Next, the control flow executed by the processor 12 when the voice data is transmitted from the voice control device 10 to the voice recognition engine 30 when the occupant of the vehicle 1 speaks will be described. FIG. 12 is a flowchart showing a control content when the voice operation device 10 transmits voice data to the voice recognition engine 30. In step S10, the processor 12 (voice acquisition unit 21) newly acquires the voice data spoken by the occupant of the vehicle 1 (voice acquisition step). Further, the processor 12 (voice acquisition unit 21) stores the acquired voice data of the speaker in the voice storage unit 131 of the memory 13.

Next, in step S20, the processor 12 (starting word detection unit 22) determines whether or not the newly acquired voice data includes the starting word. When the processor 12 (starting word detection unit 22) determines that the newly acquired voice data includes the starting word, the processor 12 detects the starting word from the voice data and proceeds to step S30. On the other hand, if the newly acquired voice data does not include the start word and the processor 12 (start word detection unit 22) does not detect the start word from the voice data, the process proceeds to step S40. In this case, it means that the newly acquired voice data is the voice data without the activation word.

In step S30, the processor 12 (voice data transmission unit 23) transmits the request content voice data regarding the utterance portion following the activation word among the newly acquired voice data to the voice recognition engine 30 together with the ID identification information. To do. As a result, the voice recognition engine 30 that has received the request content voice data from the voice operation device 10 performs voice recognition processing, natural language analysis processing, or the like on the request content voice data to indicate the utterance indicating the request content of the speaker. Person request data can be generated. When the process of step S3 is completed, the processor 12 ends the routine related to this control flow.

In step S40, the processor 12 (related word detection unit 24) detects an operation-related word related to the related device 2 operated in response to the previous operation command from the newly acquired voice data without an activation word (related word). Detection process). In this step, the processor 12 (related word detection unit 24) is a device that is determined in advance in association with each related device 2 from the previous voice data stored in the voice storage unit 131 of the memory 13 as described above. By detecting another related word, the related device 2 operated in response to the previous operation command can be specified. Alternatively, the processor 12 (related word detection unit 24) may specify the related device 2 operated in response to the previous operation command based on the operation command data stored in the command information storage unit 132 of the memory 13. ..

Then, in step S40, the processor 12 (related word detection unit 24) includes the operation-related words related to the related device 2 operated in response to the previous operation command in the newly acquired voice data without the activation word. Determine if it is. Then, the processor 12 (related word detection unit 24) detects the operation-related word from the newly acquired voice data without activation word when the operation-related word is not included in the newly acquired voice data without activation word, that is, from the newly acquired voice data without activation word. If not, it is determined that the voice data without the activation word is the utterance content not related to the operation by the previous operation command, so the routine related to this control flow is terminated. In this case, the voice data without the activation word is not transmitted from the voice operation device 10 to the voice recognition engine 30, and the voice recognition engine 30 is not started.

On the other hand, in step S40, when the processor 12 (related word detection unit 24) determines that the newly acquired voice data without activation words includes operation-related words, the processor 12 determines the operation-related words from the voice data without activation words. Detect and proceed to step S50. In step S50, the processor 12 (voice data transmission unit 23) transmits the newly acquired voice data without an activation word to the voice recognition engine 30 together with the ID identification information (voice data transmission step). As a result, the voice recognition engine 30 that receives the voice data without the activation word from the voice operation device 10 performs voice recognition processing, natural language analysis processing, and the like on the voice data without the activation word to satisfy the request contents of the speaker. It is possible to generate the speaker request data to be shown. When the process of step S5 is completed, the processor 12 ends the routine related to this control flow.

As described in the above control flow, the voice operation device 10 according to the present embodiment acquires voice data related to the utterance uttered by the occupant (user) of the vehicle 1, and the newly acquired voice data does not include the activation word. When the voice data has no activation word, the operation-related word related to the related device operated in response to the previous operation command is detected from the newly acquired voice data without the activation word. Then, when the operation-related word is detected from the voice data without the activation word, the voice data without the activation word is transmitted to the voice recognition engine. Therefore, when the user operates the related device 2 by voice while the voice recognition engine 30 is in the standby state, the user operates the related device 2 by voice input without uttering the activation word (WuW) each time the voice operation is performed. It is possible to improve the convenience of the user. That is, it is possible to provide the voice operation device 10 and the voice operation system 100 having excellent usability.

Further, when the user's utterance does not include the activation word, and if the utterance content is not related to the previous voice operation, the voice data without the activation word is not transmitted to the voice recognition engine 30, and the utterance content is the previous voice operation. Since the voice recognition engine 30 is started by transmitting the voice data without a start word to the voice recognition engine 30 when it is related to the above, the related equipment is suppressed while suppressing an excessive increase in communication load and processing load. 2 voice operations can be realized. From the above, the voice operation device 10 according to the present embodiment and the voice operation system 100 including the voice operation device 10 can improve usability while suppressing an excessive increase in communication load and processing load as compared with the conventional case. Further, according to the voice operation device 10 and the voice operation system 100 including the voice operation device 10, the existing voice recognition engine can be used as the voice recognition engine 30 without major modification, so that the development cost and construction of the system can be increased. The cost can be suppressed.

Further, when the processor 12 (related word detection unit 24) of the voice operation device 10 detects the operation-related word related to the related device 2 operated in response to the previous operation command from the newly acquired voice data without the activation word. , The association operated in response to the previous operation command by detecting a predetermined device-specific related word associated with each related device 2 from the previous voice data stored in the voice storage unit 131 of the memory 13. The device 2 is specified. According to this, the related device 2 to be operated by the previous operation command can be easily specified based on the device-specific related word associated with each related device 2. Therefore, when the user speaks without uttering the activation word, it should be easily determined whether the utterance content is related to the operation by the previous operation command, and the related device 2 should be operated in response to the utterance. It can be easily determined whether or not it is possible.

Further, when the processor 12 (related word detection unit 24) of the voice operation device 10 detects the operation-related word related to the related device 2 operated in response to the previous operation command from the newly acquired voice data without the activation word. , The related device 2 operated in response to the previous operation command is specified based on the operation command data stored in the command information storage unit 132 of the memory 13. In this way, by referring to the operation command data stored in the command information storage unit 132, the related device 2 to be operated by the previous operation command can be easily specified. Therefore, when the user speaks without uttering the activation word, it should be easily determined whether the utterance content is related to the operation by the previous operation command, and the related device 2 should be operated in response to the utterance. It can be easily determined whether or not it is possible.

Although the embodiments of the present invention have been described above, these are merely examples, and the present invention is not limited thereto, as long as the above configurations are combined and the like does not deviate from the scope of the claims. , Various changes are possible based on the knowledge of those skilled in the art.

For example, in the above embodiment, the example in which the voice control device 10 is configured as an in-vehicle device mounted on a vehicle has been described, but the present invention is not limited to this. That is, the voice operation device 10 does not have to be an in-vehicle device, and may be, for example, a smartphone, a tablet PC, a smart speaker, or the like. Further, in the voice operation system 100, the voice recognition engine 30 and the command generation server 50 may be provided as a single server device. Further, the function of the voice recognition engine 30 may be provided in the voice operation device 10, or the function of the command generation server 50 may be provided in the voice recognition engine 30.

Further, each process realized by the voice operation device 10, the voice recognition engine 30, and the command generation server 50 in the present embodiment is realized by executing various application programs stored in the memory by each of these processors. There is. Further, a program for causing each processor in the voice operation device 10, the voice recognition engine 30, and the command generation server 50 to execute each of the above-described processes can be recorded on a computer-readable recording medium. Here, the computer-readable recording medium means a recording medium that can be read from a computer by accumulating information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action. Among such recording media, those that can be removed from a computer include, for example, flexible discs, magneto-optical discs, CD-ROMs, CD-R / Ws, DVDs, DATs, 8 mm tapes, memory cards, and the like. In addition, there are hard disks, ROMs, and the like as recording media fixed to the computer.

1 ... Vehicle 2 ... Related equipment 3 ... Microphone 4 ... Speaker 10 ... Voice control device 21 ... Voice acquisition unit 22 ... Activation word detection unit 23 ... Voice data transmission Unit 24 ... Related word detection unit 25 ... Command acquisition unit 26 ... Response voice acquisition unit 27 ... Operation processing unit 28 ... Response voice output unit 30 ... Voice recognition engine 50 ... -Command generation server 100 ... Voice operation system

Claims (5)

When the voice data of the speech uttered by the user includes a start word for activating the voice recognition engine, at least the voice data including the utterance following the start word is transmitted to the voice recognition engine, and the voice recognition engine transmits the voice data. It is a voice operation device that operates related devices according to the operation commands generated based on the voice recognition processing result.
A voice acquisition unit that acquires voice data related to utterances made by the user,
When the voice data newly acquired by the voice acquisition unit is voice data without a start word that does not include a start word, the start that newly acquires an operation-related word related to the related device operated in response to the previous operation command. Related word detector that detects from wordless voice data,
When the related word detection unit detects an operation-related word from the voice data without the activation word, the voice data transmission unit that transmits the voice data without the activation word to the voice recognition engine
To prepare
Voice control device. A voice storage unit for storing voice data acquired by the voice acquisition unit is further provided.
The related word detection unit operates according to the previous operation command by detecting a predetermined device-specific related word associated with each related device from the previous voice data stored in the voice storage unit. Identify related equipment
The voice operating device according to claim 1. Further, a command information storage unit for storing operation command information related to the operation command generated based on the voice recognition processing result of the voice data by the voice recognition engine is provided.
The related word detection unit identifies the related device operated in response to the previous operation command based on the operation command information stored in the command information storage unit.
The voice operating device according to claim 1. An operation command for transmitting to the voice operation device based on the voice operation device according to any one of claims 1 to 3, the voice recognition engine, and the voice recognition processing result of voice data by the voice recognition engine. To generate an operation command generation server, including a voice operation system. When the voice data of the speech uttered by the user includes a start word for activating the voice recognition engine, at least the voice data including the utterance following the start word is transmitted to the voice recognition engine, and the voice recognition engine transmits the voice data. It is a voice operation method in which the voice operation device executes the operation of the related device according to the operation command generated based on the voice recognition processing result.
A voice acquisition process that acquires voice data related to utterances made by the user,
When the voice data newly acquired in the voice acquisition process is voice data without a start word that does not include a start word, the operation-related word related to the related device operated in response to the previous operation command is newly acquired and started. Related word detection process to detect from wordless voice data,
A voice data transmission step of transmitting the voice data without the activation word to the voice recognition engine when an operation-related word is detected from the voice data without the activation word in the related word detection step.
including,
Voice operation method.

Images