JP2022017239A - Voice operation support system, voice operation system, voice processing device, voice operation support device, voice operation support method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make full use of household electrical appliances.

SOLUTION: A voice operation support system includes voice data reception means, answer information generation means, and voice data transmission means. The voice data reception means receives first voice data based on user's voice from a voice input/output device 3. When the received first voice data indicates an inquiry about the operation of a household electrical appliance 2, the answer information generation means generates, for each model of the household electrical appliance 2, answer information for the inquiry by referring to an operation explanation information database associated with information indicating an explanation about an operation peculiar to the model. The voice data transmission means transmits second voice data indicating the voice based on the generated answer information to the voice input/output device 3.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a voice operation support system, a voice operation system, a voice processing device, a voice operation support device, a voice operation support method, and a program.

Conventionally, various proposals have been made regarding a system that recognizes a user's voice input via a voice input device installed in the house by a server outside the house and controls home appliances in the house based on the recognition result. (For example, Patent Document 1).

In recent years, this type of system is operated by a manufacturer that sells voice input devices, and is operated by a server that performs voice recognition (hereinafter referred to as the first server) and a manufacturer that sells home appliances, and controls home appliances. A configuration for providing a voice operation service is being developed in cooperation with a server (hereinafter referred to as a second server).

Specifically, in the above system, the first server and a plurality of second servers corresponding to each of a plurality of different manufacturers are connected via the Internet, and the manufacturers are operated by common voice operations regardless of the manufacturers. Allows similar control for different home appliances of the same type (eg, air conditioners).

International Publication No. 2014/171144

However, the above-mentioned common voice operations are limited to those corresponding to basic operations such as power on / off and operation start / end. For this reason, the user cannot easily perform the operation peculiar to the manufacturer of the home electric appliance, and the actual situation is that the home electric appliance owned by the user cannot be fully utilized.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a voice operation support system or the like capable of fully utilizing home appliances.

In order to achieve the above object, the voice operation support system according to the present invention is
A voice data receiving means for receiving the first voice data based on the user's voice from the voice input / output device, and
When the received first voice data indicates an inquiry about the operation of the home electric appliance, refer to the operation explanation information database associated with the information indicating the explanation about the operation peculiar to the model for each model of the home electric appliance. Then, the answer information generation means for generating the answer information for the inquiry,
A voice data transmission means for transmitting a second voice data indicating a voice based on the response information to the voice input / output device is provided.

According to the present invention, the home electric appliance can be fully utilized.

The figure which shows the whole structure of the voice operation system including the voice operation support system which concerns on Embodiment 1 of this invention. A block diagram showing a hardware configuration of an audio input / output device according to the first embodiment. A block diagram showing a functional configuration of the audio input / output device according to the first embodiment. A block diagram showing a hardware configuration of the voice processing device according to the first embodiment. The figure for demonstrating the secondary storage device included in the voice processing apparatus which concerns on Embodiment 1. The figure for demonstrating the apparatus operation information DB stored in the secondary storage apparatus included in the voice processing apparatus which concerns on Embodiment 1. The figure for demonstrating the customer equipment information DB stored in the secondary storage device included in the voice processing apparatus which concerns on Embodiment 1. A block diagram showing a hardware configuration of the device control device according to the first embodiment. The figure for demonstrating the secondary storage device included in the device control device which concerns on Embodiment 1. The figure for demonstrating the device operation information DB stored in the secondary storage device included in the device control device which concerns on Embodiment 1. The figure for demonstrating the customer equipment information DB stored in the secondary storage device included in the device control device which concerns on Embodiment 1. A block diagram showing a hardware configuration of the voice operation support device according to the first embodiment. The figure for demonstrating the secondary storage device included in the voice operation support device which concerns on Embodiment 1. The figure for demonstrating the operation explanation information DB (air conditioner) stored in the secondary storage device included in the voice operation support device which concerns on Embodiment 1. The figure for demonstrating the operation explanation information DB (television) stored in the secondary storage device included in the voice operation support device which concerns on Embodiment 1. The figure for demonstrating the apparatus information DB stored in the secondary storage apparatus included in the voice operation support apparatus which concerns on Embodiment 1. The block diagram which shows the functional structure of the voice processing apparatus which concerns on Embodiment 1. The figure which shows the structure of the operation information in Embodiment 1. The figure which shows an example of the correspondence relation between the utterance content of a user, and the generated operation information in Embodiment 1. The figure which shows the structure of the request information in Embodiment 1. The figure for demonstrating the request number in Embodiment 1. In the first embodiment, a diagram showing an example of a correspondence relationship between a user's utterance content and a generated request information. A flowchart showing a procedure of voice processing in the first embodiment. A flowchart showing a procedure of operation information generation processing in the first embodiment. A flowchart showing a procedure of request information generation processing in the first embodiment. A block diagram showing a functional configuration of the device control device according to the first embodiment. A block diagram showing a functional configuration of the voice operation support device according to the first embodiment. The figure which shows an example of the correspondence relation between the request information and the generated answer information in Embodiment 1. A flowchart showing a procedure of voice operation support processing in the first embodiment. The figure which shows the whole structure of the voice operation system including the voice operation support system which concerns on Embodiment 2 of this invention. The block diagram which shows the functional structure of the voice processing apparatus which concerns on Embodiment 2. The figure which shows the structure of the 2nd request information in Embodiment 2. The figure for demonstrating the request number in Embodiment 2. In the second embodiment, a diagram showing an example of the correspondence relationship between the utterance content of the user and the generated second request information. The figure for demonstrating the secondary storage device provided in the voice operation support device which concerns on Embodiment 2. The figure for demonstrating the operation record information DB stored in the secondary storage device included in the voice operation support device which concerns on Embodiment 2. A block diagram showing a functional configuration of the voice operation support device according to the second embodiment. The figure for demonstrating the usage tendency in Embodiment 2. The figure for demonstrating the usage tendency in Embodiment 2. The figure which shows an example of the correspondence relation between the 2nd request information and the generated advice information in Embodiment 2. The figure which shows the whole structure of the voice operation system including the voice operation support system which concerns on Embodiment 3 of this invention. The figure for demonstrating the secondary storage device included in the voice processing apparatus which concerns on Embodiment 3. The figure for demonstrating the customer equipment information DB stored in the secondary storage device included in the voice processing apparatus which concerns on Embodiment 3. The figure for demonstrating the operation history DB stored in the secondary storage device included in the voice processing apparatus which concerns on Embodiment 3. The figure for demonstrating the operation history DB stored in the secondary storage device included in the voice processing apparatus which concerns on Embodiment 3. The figure for demonstrating the operation sequence DB stored in the secondary storage device included in the voice processing apparatus which concerns on Embodiment 3. The block diagram which shows the functional structure of the voice processing apparatus which concerns on Embodiment 3. The figure for demonstrating each parameter for calculating a continuous index in Embodiment 3. The figure which shows an example of the continuous operation table in Embodiment 3.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Embodiment 1.
FIG. 1 is a diagram showing an overall configuration of a voice operation system 1 including a voice operation support system according to the first embodiment of the present invention. The voice operation system 1 is a system that enables the home appliance 2 installed in the house H to be controlled by voice operation, and includes one or a plurality of home appliances 2, a voice input / output device 3, a home gateway 4, and voice. It includes a processing device 5, a device control device 6, and a voice operation support device 7. Further, the voice operation support system according to the first embodiment is a system for supporting voice operation by the user, and is a voice input / output device 3, a home gateway 4, a voice processing device 5, and a voice operation support device 7. It is composed of.

Each home appliance 2 is an air conditioner, a television, an IH (Induction Heating) cooker, etc., and is installed in the house H (including the site). Each home appliance 2 is referred to as a so-called IoT (Internet of Things) device, information home appliance, net home appliance, smart home appliance, etc., and is connected to the Internet via a home gateway 4 having a function as a broadband router.

The audio input / output device 3 is an example of the audio input / output device according to the present invention. The voice input / output device 3 is a device that provides various services based on the voice input by the user's utterance, and is a so-called AI speaker (also referred to as a smart speaker). As shown in FIG. 2, the audio input / output device 3 includes a microphone 30, an A / D converter 31, a speaker 32, a communication interface 33, a CPU (Central Processing Unit) 34, and a ROM (Read Only Memory) 35. , A RAM (Random Access Memory) 36, and a secondary storage device 37. These components are connected to each other via the bus 38.

The microphone 30 inputs the voice emitted by the user, generates an analog voice signal from the input voice, and outputs the analog voice signal to the A / D converter 31. The A / D converter 31 removes noise from the analog audio signal output from the microphone 30, generates a digital audio signal, and supplies the digital audio signal to the CPU 34. The speaker 32 outputs the voice corresponding to the digital voice signal supplied from the CPU 34.

The communication interface 33 includes a network card for communicating with the home gateway 4 by wire or wirelessly and communicating with the voice processing device 5 connected to the Internet via the home gateway 4.

The CPU 34 comprehensively controls the audio input / output device 3. Details of the functions of the audio input / output device 3 realized by the CPU 34 will be described later. The ROM 35 stores a plurality of firmwares and data used when executing these firmwares. The RAM 36 is used as a work area of the CPU 34.

The secondary storage device 37 includes an EEPROM (Electrically Erasable Programmable Read-Only Memory), a readable / writable non-volatile semiconductor memory such as a flash memory, or an HDD (Hard Disk Drive). The secondary storage device 37 stores the audio input / output program and the data used when the audio input / output program is executed. The voice input / output program inputs the user's voice and transmits the voice data based on the input voice to the voice processing device 5, and receives the voice data from the voice processing device 5 and outputs the voice based on the received voice data. It is a program that describes the process to be output.

The audio input / output device 3 functionally includes an audio input unit 300, a transmission unit 301, a reception unit 302, and an audio output unit 303, as shown in FIG. These functional units are realized by the CPU 34 executing the above-mentioned audio input / output program stored in the secondary storage device 37.

The voice input unit 300 inputs the voice spoken by the user and generates voice data based on the input voice. Specifically, the voice input unit 300 applies PCM (Pulse Code Modulation) to an analog voice signal indicating the user's voice to convert it into a digital voice signal, and further applies lossy compression to the voice. Generate data.

The transmission unit 301 transmits the voice data generated by the voice input unit 300 to the voice processing device 5 via the home gateway 4.

The receiving unit 302 receives the voice data transmitted from the voice processing device 5 via the home gateway 4. The voice output unit 303 outputs voice based on the voice data received by the reception unit 302.

Returning to FIG. 1, the voice processing device 5 is an example of the voice processing device according to the present invention. The voice processing device 5 is a server computer installed and operated by a manufacturer, a sales company, or the like of the voice input / output device 3, and is connected to the Internet. The voice processing device 5 cooperates with the voice input / output device 3 and the device control device 6 described later to realize a voice operation for the home electric appliance 2. Further, the voice processing device 5 cooperates with the voice input / output device 3 and the voice operation support device 7 described later to provide a service for supporting the voice operation to the user. As shown in FIG. 4, the voice processing device 5 includes a communication interface 50, a CPU 51, a ROM 52, a RAM 53, and a secondary storage device 54. These components are connected to each other via the bus 55.

The communication interface 50 is a device for connecting to the Internet and communicating with other devices such as a home gateway 4, a device control device 6, and a voice operation support device 7. The CPU 51 comprehensively controls the voice processing device 5. Details of the functions of the voice processing device 5 realized by the CPU 51 will be described later.

The ROM 52 stores a plurality of firmwares and data used when executing these firmwares. The RAM 53 is used as a work area of the CPU 51.

The secondary storage device 54 is a large-capacity storage device including a readable / writable non-volatile semiconductor memory such as an EEPROM and a flash memory or an HDD. As shown in FIG. 5, the secondary storage device 54 stores the voice processing program 540, the device operation information DB 541, the customer information DB 542, and the customer device information DB 543. In addition to this, the secondary storage device 54 stores a voice recognition DB or the like, which is a database for voice recognition.

The voice processing program 540 is a program related to voice processing for realizing a voice operation service. The voice processing program 540 describes voice data analysis processing, device operation-related processing, operation support-related processing, and the like.

The voice data analysis process includes a process of analyzing the voice data transmitted from the voice input / output device 3 using the above voice recognition DB and acquiring the contents of the voice. In the device operation-related processing, operation information for the home electric appliance 2 is generated based on the acquired voice content and transmitted to the device control device 6, and voice data based on the result information from the device control device 6 is input by voice. The process of transmitting to the output device 3 is included. The operation support-related processing is based on a process of transmitting request information for requesting an answer to an inquiry from a user regarding the operation of the home electric appliance 2 to the voice operation support device 7, and a process of transmitting the request information from the voice operation support device 7. It includes a process of transmitting voice data to the voice input / output device 3.

The device operation information DB 541 is a database in which information related to all operations corresponding to the model of the home electric appliance 2 is registered for each model. Specifically, as shown in FIG. 6, a record including items such as an operation number 5410, a category 5411, an operation name 5412, and a setting content 5413 is registered in the device operation information DB 541 for each model of the home electric appliance 2. There is. In the present embodiment, the model means a type (for example, an air conditioner, a television, an IH cooker, etc.) based on the essential functions of the home electric appliance 2.

The operation number is stored in the operation number 5410. The operation number is a number uniquely assigned to each operation in the model. In the category 5411, the category name corresponding to the operation number stored in the operation number 5410 is stored. The category name is a name indicating the classification due to the nature of the operation in the model. For example, when the model of the home electric appliance 2 is an air conditioner, any one of "basic operation", "energy saving operation", and "watching operation" is stored in the category 5411. Further, for example, when the model of the home electric appliance 2 is a television, any one of "basic operation", "energy saving operation", and "dubbing operation" is stored in the category 5411.

In the present embodiment, the operations whose category name corresponds to "basic operation" (for example, in the case of an air conditioner, 1: start of operation 2: end of operation) are common among manufacturers for the home electric appliance 2 belonging to the model. It means that it is an operation. That is, it means that the operation corresponding to the "basic operation" can be supported by all the home electric appliances 2 belonging to the model of all the manufacturers who support the voice operation service.

On the other hand, operations whose category name is other than "basic operation" (for example, in an air conditioner, 3: smart stop, 4: hybrid operation setting, 5: high temperature watching, 6: low temperature watching, 7: good sleep watching) are described above. It means that the home appliance 2 belonging to the model of any one of the manufacturers can be used, that is, the operation is unique to the manufacturer.

The operation name is stored in the operation name 5412. The operation name is the name of the operation corresponding to the operation number stored in the operation number 5410, and is also the content of the voice spoken by the user to perform the operation. The setting contents are stored in the setting contents 5413. The setting content is a content that can be set for the home electric appliance 2 at the time of the operation. The device operation information DB 541 is linked to the device operation information DB held by the device control device 6, for example, by appropriately coordinating the voice processing device 5 and the device control device 6 with OAuth 2.0. As a result, the contents of the device operation information DB 541 are updated.

Returning to FIG. 5, the customer information DB 542 is a database in which personal information of each customer who has subscribed to the voice operation service is registered. The customer information DB 542 includes the name, address, gender, date of birth of each customer, the IP (Internet Protocol) address of the voice input / output device 3 installed in the customer's house (that is, the customer's house H), and the like. Information is registered.

The customer device information DB 543 is a database in which information about all the home electric appliances 2 installed in each customer's house is registered. Specifically, as shown in FIG. 7, in the customer device information DB 543, a record including items such as device ID 5430, model name 5431, model name 5432, installation location 5433, and corresponding operation 5434 is recorded in the customer's home for each customer. It is registered for the number of all home appliances 2 installed in.

The device ID 5430 stores the device ID of the home electric appliance 2. The device ID is an ID (identification) assigned to identify each home electric appliance 2 in the voice operation service. The model name 5431 stores the name of the model to which the home appliance 2 belongs (for example, "air conditioner", "television", "IH cooker", etc.). The model name (also referred to as a model number) of the home electric appliance 2 is stored in the model name 5432. In the installation place 5433, a name indicating the installation place of the home electric appliance 2 (for example, "living room", "bedroom", "kitchen", etc.) is stored. In the corresponding operation 5434, the operation numbers of all the operations that can be operated for the home electric appliance 2 are stored.

The customer information DB 542 and the customer device information DB 543 are, for example, a customer information DB and a customer device information DB held by the device control device 6 by appropriately coordinating the voice processing device 5 and the device control device 6 by OAut 2.0. Linked with. As a result, the contents of the customer information DB 542 and the customer device information DB 543 are updated.

Returning to FIG. 1, the device control device 6 is a server computer installed and operated by a manufacturer, a sales company, or the like of the home electric appliance 2, and is connected to the Internet. The device control device 6 cooperates with the voice processing device 5 to realize voice operation for the home electric appliance 2.

As shown in FIG. 8, the device control device 6 includes a communication interface 60, a CPU 61, a ROM 62, a RAM 63, and a secondary storage device 64. These components are connected to each other via a bus 65.

The communication interface 60 is a device for connecting to the Internet and communicating with other devices such as a home gateway 4 and a voice processing device 5. The CPU 61 comprehensively controls the device control device 6. Details of the functions of the device control device 6 realized by the CPU 61 will be described later.

The ROM 62 stores a plurality of firmwares and data used when executing these firmwares. The RAM 63 is used as a work area of the CPU 61.

The secondary storage device 64 is a large-capacity storage device including a readable / writable non-volatile semiconductor memory such as an EEPROM and a flash memory or an HDD. As shown in FIG. 9, the secondary storage device 64 stores the device control program 640, the device operation information DB 641, the customer information DB 642, and the customer device information DB 643.

The device control program 640 is a program for realizing a voice operation for the home electric appliance 2 in cooperation with the voice processing device 5. The device control program 640 includes processing for controlling the corresponding home electric appliance 2 based on the operation information from the voice processing device 5, processing for transmitting result information indicating the control result of the home electric appliance 2 to the voice processing device 5, and the like. Is described.

The device operation information DB 641 is a database in which information related to the operation of the home electric appliance 2 of the company is registered. Specifically, as shown in FIG. 10, a record including items such as a model name 6410, an operation number 6411, an operation name 6412, and a setting content 6413 is registered in the device operation information DB 641 for each model of the home electric appliance 2. ing. The model name 6410 stores the model name of the home electric appliance 2. The operation number is stored in the operation number 6411. The operation name 6412 stores the operation name corresponding to the operation number stored in the operation number 6411. The setting content 6413 stores the setting content corresponding to the operation. The contents of each record of the device operation information DB 641 may be automatically generated from the data of the instruction manual of the home electric appliance 2 of each model by a DB construction unit (not shown) and set in each record. ..

As described above, for example, by operating the device control device 6 and the voice processing device 5 in cooperation with each other by OAuth 2.0, the device operation information DB 641 of the device control device 6 and the device operation information DB 541 of the voice processing device 5 are operated in cooperation with each other. Is linked to. As a result, the contents of the device operation information DB 541 of the voice processing device 5 are updated.

Returning to FIG. 9, the customer information DB 642 is a database in which personal information of each customer who is a purchaser of the home electric appliance 2 of the company and is subscribed to the voice operation service is registered. Information such as the name, address, gender, and date of birth of each customer is registered in the customer information DB 642.

The customer device information DB 643 is a database in which information about the company's home electric appliances 2 installed in each customer's home is registered. Specifically, as shown in FIG. 11, the customer device information DB 643 contains a record including items such as a device ID 6430, a model name 6431, a model name 6432, an installation location 6433, a corresponding operation 6434, and a communication address 6435 for each customer. Is registered for the number of home appliances 2 of the company installed in the customer's house.

The device ID 6430 stores the device ID assigned to the home electric appliance 2. The model name 6431 stores the name of the model to which the home electric appliance 2 belongs. The model name of the home electric appliance 2 is stored in the model name 6432. The installation location 6433 stores a name indicating the installation location of the home electric appliance 2. In the corresponding operation 6434, the operation numbers of all the operations that can be operated for the home electric appliance 2 are stored. The IP address of the home electric appliance 2 is stored in the communication address 6435.

A user who has purchased a home appliance 2 activates an application program (hereinafter referred to as a voice control application) for using a voice operation service pre-installed on a smartphone, tablet terminal, etc., and his / her personal information and the home appliance 2 A predetermined procedure (hereinafter referred to as a registration procedure) for registering information related to the device control device 6 is performed. As a result, the personal information of the user and the information about the home electric appliance 2 of the user's home are transmitted from the smartphone, tablet terminal, or the like to the device control device 6. The above voice operation application can be downloaded from the device control device 6, other program distribution server, or the like to the smartphone, tablet terminal, or the like.

In the above registration procedure, the model name, model name and IP address are acquired from the home electric appliance 2 by communication via the home gateway 4 by the smartphone, tablet terminal or the like that has activated the voice operation application.

As described above, for example, by operating the device control device 6 and the voice processing device 5 in cooperation with each other by OOut 2.0, the customer information DB 642, the customer device information DB 643, and the voice processing device 5 of the device control device 6 can be operated in cooperation with each other. The customer information DB 542 and the customer device information DB 543 are linked. As a result, the contents of the customer information DB 542 and the customer device information DB 543 of the voice processing device 5 are updated.

Returning to FIG. 1, the voice operation support device 7 is an example of the voice operation support device according to the present invention. The voice operation support device 7 is a server computer operated jointly by a plurality of manufacturers co-sponsoring the voice operation service or by a third-party organization, and is connected to the Internet. The voice operation support device 7 realizes a voice operation support service in cooperation with the voice input / output device 3 and the voice processing device 5. As shown in FIG. 12, the voice operation support device 7 includes a communication interface 70, a CPU 71, a ROM 72, a RAM 73, and a secondary storage device 74. These components are connected to each other via the bus 75.

The communication interface 70 is a device for connecting to the Internet and communicating with other devices such as the voice processing device 5. The CPU 71 comprehensively controls the voice operation support device 7. Details of the functions of the voice operation support device 7 realized by the CPU 71 will be described later. The ROM 72 stores a plurality of firmwares and data used when executing these firmwares. The RAM 73 is used as a work area of the CPU 71.

The secondary storage device 74 is a large-capacity storage device including a readable / writable non-volatile semiconductor memory such as an EEPROM and a flash memory or an HDD. As shown in FIG. 13, the secondary storage device 74 stores the voice operation support program 740, the operation explanation information DB 741, and the device information DB 742. The secondary storage device 74 is an example of a storage device according to the present invention.

The voice operation support program 740 is a program in which processes related to voice operation support are described. The operation explanation information DB 741 is a database in which information for explaining to the user is registered for each operation of each model of the home electric appliance 2. In detail, as shown in FIGS. 14 and 15, the operation explanation information DB 741 contains the operation number 7410, the category 7411, the operation name 7412, the outline explanation 7413, the detailed explanation 7414, and the precautions 7415 for each model of the home electric appliance 2. A record containing items such as is registered. The content of each record of the operation explanation information DB 741 may be automatically generated from the data of the instruction manual of the home electric appliance 2 of each model by a DB construction unit (not shown) and set in each record. ..

The operation number is stored in the operation number 7410. In the category 7411, the category name corresponding to the operation number stored in the operation number 7410 is stored. The operation name corresponding to the operation number is stored in the operation name 7412. The schematic description 7413 stores information indicating a brief description of the operation. The detailed description 7414 stores information indicating a detailed description of the operation. The precautions 7415 stores information indicating matters to be noted about the operation.

Returning to FIG. 13, the device information DB 742 is a database in which information about home appliances 2 of all customers is registered. Specifically, as shown in FIG. 16, in the device information DB 742, all the home appliances in which the device ID 7420 and the record including the items such as the model name 7421, the installation location 7422, and the corresponding operation 7423 are installed in each customer's house are installed. The total number of devices 2 is registered.

The device ID 7420 stores the device ID assigned to the home electric appliance 2. The model name 7421 stores the name of the model to which the home electric appliance 2 belongs. The installation location 7422 stores a name indicating the installation location of the home electric appliance 2. In the corresponding operation 7423, the operation numbers of all the operations that can be operated for the home electric appliance 2 are stored. The device information DB 742 is linked to the customer device information DB 643 held by the device control device 6, for example, by appropriately coordinating the voice operation support device 7 and the device control device 6 with OAuth 2.0. As a result, the contents of the device information DB 742 are updated.

Subsequently, the function of the voice processing device 5 will be described. As shown in FIG. 17, functionally, the voice processing device 5 includes a voice data receiving unit 500, a voice data analysis unit 501, an operation information generation unit 502, an operation information transmission unit 503, and a request information generation unit. It includes a request information transmission unit 505, a result information reception unit 506, a voice data generation unit 507, a voice data transmission unit 508, and a response information reception unit 509. These functional units are realized by the CPU 51 executing the voice processing program 540 stored in the secondary storage device 54.

The voice data receiving unit 500 is an example of the voice data receiving means according to the present invention. The voice data receiving unit 500 receives voice data transmitted from the voice input / output device 3 via the home gateway 4. The voice data is an example of the first voice data according to the present invention. The voice data analysis unit 501 analyzes the voice data received by the voice data receiving unit 500 and generates text data (hereinafter referred to as voice content data) indicating the speech content of the user. When the generated voice content data indicates an operation for the home electric appliance 2, the voice data analysis unit 501 supplies the voice content data to the operation information generation unit 502. On the other hand, when the generated audio content data indicates an inquiry about the operation of the home electric appliance 2, the audio data analysis unit 501 supplies the audio content data to the request information generation unit 504.

The operation information generation unit 502 generates operation information based on the voice content data supplied from the voice data analysis unit 501, the device operation information DB 541, the customer information DB 542, and the customer device information DB 543. As shown in FIG. 18, the operation information includes items such as the device ID 5020, the operation number 5021, and the setting content 5022. The device ID 5020 stores the device ID of the target home electric appliance 2. The operation number 5021 stores the operation number corresponding to the target operation. The setting content 5022 stores the setting content instructed by the user in the operation.

The operation information generation unit 502 identifies the home electric appliance 2 that is the target of the voice operation in the customer's house from the keywords indicating each of the place and the model included in the voice content data. For example, when the voice content data includes "living room" and "air conditioner", the operation information generation unit 502 identifies the home appliance 2 having the device ID "R111" as the home appliance 2 subject to the voice operation. do.

Further, the operation information generation unit 502 uses the device ID of the identified home electric appliance 2 and keywords indicating the operation name and / or the setting content included in the audio content data (for example, "operation start", "operation end", "power supply". The operation number and / or the setting content is identified from "on", "power off", "high temperature watching on", etc.). FIG. 19 shows an example of the correspondence between the utterance content of the user and the generated operation information.

The operation information transmission unit 503 transmits the operation information generated by the operation information generation unit 502 to the device control device 6.

The request information generation unit 504 generates request information based on the voice content data supplied from the voice data analysis unit 501, the device operation information DB 541, the customer information DB 542, and the customer device information DB 543. As shown in FIG. 20, the request information includes items such as device ID 5040, operation number 5041, category 5042, and request number 5043. The device ID 5040 stores the device ID of the target home electric appliance 2. The operation number corresponding to the target operation is stored in the operation number 5041. However, in the case of a query that does not specify an operation, predetermined dummy data is stored in the operation number 5041.

The target category name is stored in the category 5042. However, in the case of a query that does not specify a category, predetermined dummy data is stored in the category 5042. The request number is stored in the request number 5043. As shown in FIG. 21, the request number is a number uniquely assigned to each request content.

The request information generation unit 504 identifies the home electric appliance 2 that is the target of the inquiry at the customer's home from the keywords indicating each of the location and the model included in the audio content data. For example, when the audio content data includes "living room" and "air conditioner", the request information generation unit 504 identifies the home appliance 2 having the device ID "R111" as the home appliance 2 to be inquired about. do. The request information generation unit 504 stores the identified device ID in the device ID 5040.

The request information generation unit 504 determines whether or not a keyword indicating an operation name or a category name exists in the audio content data. If neither keyword exists, the request information generation unit 504 stores "100" in the request number 5043. Further, the request information generation unit 504 stores dummy data in each of the operation number 5041 and the category 5042.

When the keyword indicating the category name exists in the audio content data, the request information generation unit 504 stores "101" in the request number 5043. Further, the request information generation unit 504 stores dummy data in the operation number 5041, and stores a keyword indicating the category name in the category 5042.

When the keyword indicating the operation name is present in the voice content data, the request information generation unit 504 identifies the operation number based on the device ID of the identified home electric appliance 2 and the operation name. The request information generation unit 504 stores the identified operation number in the operation number 5041. The request information generation unit 504 determines whether or not "details" or "notes" exist in the audio content data. If neither of them exists, the request information generation unit 504 stores "102" in the request number 5043 and stores dummy data in the category 5042.

When "details" are present in the audio content data, the request information generation unit 504 stores "103" in the request number 5043 and stores dummy data in the category 5042. When "Caution" exists in the audio content data, the request information generation unit 504 stores "104" in the request number 5043 and stores the dummy data in the category 5042.

As described above, the request information generation unit 504 generates request information according to the content of the user's utterance. FIG. 22 shows an example of the correspondence between the utterance content of the user and the generated request information.

The request information transmission unit 505 transmits the request information generated by the request information generation unit 504 to the voice operation support device 7. The request information generation unit 504 and the request information transmission unit 505 are examples of the response request means according to the present invention.

The result information receiving unit 506 receives the result information from the device control device 6. The result information is information indicating whether or not the target home electric appliance 2 has been normally operated according to the previously transmitted operation information. The result information receiving unit 506 supplies the received result information to the voice data generation unit 507. The voice data generation unit 507 generates voice data by a known voice synthesis process based on the result information supplied from the result information receiving unit 506, and supplies the generated voice data to the voice data transmission unit 508. The voice data is an example of the second voice data according to the present invention. The voice data transmission unit 508 transmits the voice data supplied from the voice data generation unit 507 to the voice input / output device 3. The voice data transmission unit 508 is an example of the voice data transmission means according to the present invention.

The response information receiving unit 509 receives the response information from the voice operation support device 7. The answer information is information generated by the voice operation support device 7 in response to the previously transmitted request information, and is information indicating an answer to the user's inquiry. The answer information receiving unit 509 supplies the received answer information to the voice data generation unit 507. The voice data generation unit 507 generates voice data by a known voice synthesis process based on the response information supplied from the response information receiving unit 509, and supplies the generated voice data to the voice data transmission unit 508. The voice data transmission unit 508 transmits the voice data supplied from the voice data generation unit 507 to the voice input / output device 3.

FIG. 23 is a flowchart showing a procedure of voice processing executed by the voice processing device 5. When the voice data transmitted from the voice input / output device 3 is received by the voice data receiving unit 500 (step S101; YES), the voice data analysis unit 501 analyzes the voice data (step S102), and the voice content. Generate data.

When the voice content data generated by the voice data analysis unit 501 indicates an operation for the home electric appliance 2 (step S103; YES), the voice processing device 5 executes the operation information generation process (step S104). On the other hand, when the above-mentioned voice content data does not indicate an operation for the home electric appliance 2, that is, indicates an inquiry about the operation of the home electric appliance 2 (step S103; NO), the voice processing device 5 performs a request information generation process. Is executed (step S105).

FIG. 24 is a flowchart showing the procedure of the operation information generation process. The operation information generation unit 502 determines whether or not the voice content data generated by the voice data analysis unit 501 lacks the information required for generating the operation information (step S201).

For example, if the audio content data does not include any of the keywords indicating the model of the target home electric appliance 2 and the keywords indicating the installation location, the operation information generation unit 502 lacks the necessary information. It is determined that it is.

When the necessary information is not insufficient (step S201; NO), the operation information generation unit 502 generates the operation information as described above (step S202). The operation information transmission unit 503 transmits the operation information generated by the operation information generation unit 502 to the device control device 6 (step S203).

The result information receiving unit 506 receives the result information from the device control device 6 (step S204). The voice data generation unit 507 generates voice data based on the result information received by the result information receiving unit 506 (step S205).

On the other hand, when the necessary information is insufficient (step S201; YES), the operation information generation unit 502 generates inquiry information for inquiring the user regarding the insufficient information (step S206). For example, the inquiry information includes text data such as "Which of the air conditioners to be operated is the air conditioner in the living room or the air conditioner in the bedroom?"

The voice data generation unit 507 generates voice data based on the inquiry information generated by the operation information generation unit 502 (step S207).

The voice data transmission unit 508 transmits the voice data generated by the voice data generation unit 507 to the voice input / output device 3 (step S208). After that, the process returns to step S101 in the flowchart of FIG.

When the voice data based on the inquiry information is transmitted to the voice input / output device 3, when the user obtains a reply indicating the supplementary information, the voice processing device 5 receives the supplementary information and the previously generated voice content data. Operation information is generated based on the above and transmitted to the device control device 6.

FIG. 25 is a flowchart showing the procedure of the request information generation process. The request information generation unit 504 determines whether or not the audio content data generated by the audio data analysis unit 501 lacks the information required for generating the request information (step S301).

For example, if the audio content data does not include any of the keywords indicating the model of the target home electric appliance 2 and the keywords indicating the installation location, the request information generation unit 504 lacks the necessary information. It is determined that it is.

When the necessary information is not insufficient (step S301; NO), the request information generation unit 504 generates the request information as described above (step S302). The request information transmission unit 505 transmits the request information generated by the request information generation unit 504 to the voice operation support device 7 (step S303).

The response information receiving unit 509 receives the response information from the voice operation support device 7 (step S304). The voice data generation unit 507 generates voice data based on the response information received by the response information receiving unit 509 (step S305).

On the other hand, when the necessary information is insufficient (step S301; YES), the request information generation unit 504 generates inquiry information for inquiring the user regarding the insufficient information (step S306). For example, the inquiry information includes text data such as "Which air conditioner you are inquiring about is a living room air conditioner or a bedroom air conditioner?"

The voice data generation unit 507 generates voice data based on the inquiry information generated by the request information generation unit 504 (step S307).

The voice data transmission unit 508 transmits the voice data generated by the voice data generation unit 507 to the voice input / output device 3 (step S308). After that, the process returns to step S101 in the flowchart of FIG.

When the voice data based on the inquiry information is transmitted to the voice input / output device 3, when the user obtains a reply indicating the supplementary information, the voice processing device 5 receives the supplementary information and the previously generated voice content data. Request information is generated based on the above and transmitted to the voice operation support device 7.

Subsequently, the function of the device control device 6 will be described. As shown in FIG. 26, the device control device 6 functionally includes an operation information receiving unit 600, a control command generation unit 601, a control command transmitting unit 602, a response data receiving unit 603, and a result information transmitting unit. It is equipped with 604. These functional units are realized by the CPU 61 executing the device control program 640.

The operation information receiving unit 600 receives the operation information (see FIG. 18) transmitted from the voice processing device 5. The control command generation unit 601 generates a control command for controlling the target home electric appliance 2 based on the operation information received by the operation information receiving unit 600, the device operation information DB 641, and the customer device information DB 643. The control command transmission unit 602 transmits the control command generated by the control command generation unit 601 to the home electric appliance 2. The control command generation unit 601 is an example of the control command generation means according to the present invention, and the control command transmission unit 602 is an example of the control command transmission means according to the present invention.

The response data receiving unit 603 receives the response data sent from the home electric appliance 2 at the transmission destination after the control command transmitting unit 602 transmits the control command. The response data includes information indicating whether or not the operation according to the control command has been performed. When the response data includes information indicating that the operation according to the control command has been performed, the response data receiving unit 603 notifies the result information transmitting unit 604 that the home electric appliance 2 has been normally operated. do.

On the other hand, when the response data includes information indicating that the operation according to the control command is not performed, the response data receiving unit 603 transmits the result information that the home electric appliance 2 is not normally operated. Notify unit 604. Further, if the response data receiving unit 603 cannot receive the response data from the home electric appliance 2 even after a predetermined time has elapsed after the control command is transmitted to the home electric appliance 2, the home electric appliance 2 is normal. Notifies the result information transmission unit 604 that the operation has not been performed.

The result information transmission unit 604 transmits the result information according to the content notified from the response data reception unit 603 to the voice processing device 5.

Subsequently, the function of the voice operation support device 7 will be described. As shown in FIG. 27, the voice operation support device 7 functionally includes a request information receiving unit 700, a request information analysis unit 701, an answer information generation unit 702, and an answer information transmission unit 703. These functional units are realized by the CPU 71 executing the voice operation support program 740.

The request information receiving unit 700 receives the request information (see FIG. 20) transmitted from the voice processing device 5. The request information analysis unit 701 analyzes the request information received by the request information receiving unit 700 with reference to the operation explanation information DB 741 and the device information DB 742. Specifically, the request information analysis unit 701 retrieves the data stored in each of the device ID 5040, the operation number 5041, the category 5042, and the request number 5043 of the request information. Further, the request information analysis unit 701 refers to the device information DB 742 and acquires all the operation numbers corresponding to the home appliance 2 of the device ID stored in the device ID 5040.

The request information analysis unit 701 supplies the analysis result information including each fetched data (however, dummy data is excluded) and all the acquired operation numbers to the answer information generation unit 702. Further, the request information analysis unit 701 checks the correctness of the inquiry content indicated by the request information, and notifies the response information generation unit 702 of the check result. Specifically, when the operation number is stored in the operation number 5041, the request information analysis unit 701 checks whether or not the operation number corresponds to the home electric appliance 2. When the operation number corresponds to the home electric appliance 2, the request information analysis unit 701 notifies the answer information generation unit 702 of the check result indicating that the inquiry content is correct. On the other hand, when the operation number does not correspond to the home electric appliance 2, the request information analysis unit 701 notifies the response information generation unit 702 of the check result indicating that the inquiry content is invalid.

Further, when the category name is stored in the category 5042, the request information analysis unit 701 refers to the operation explanation information DB 741 and checks whether or not the category name corresponds to the home electric appliance 2. When the category name corresponds to the home electric appliance 2, the request information analysis unit 701 notifies the answer information generation unit 702 of the check result indicating that the inquiry content is correct. On the other hand, when the category name does not correspond to the home electric appliance 2, the request information analysis unit 701 notifies the response information generation unit 702 of the check result indicating that the inquiry content is invalid.

Further, when the dummy data is stored in any of the operation number 5041 and the category 5042, the request information analysis unit 701 checks whether or not "100" is stored in the request number 5043. When "100" is stored in the request number 5043, that is, when the request content is "all operation explanations", the request information analysis unit 701 outputs a check result indicating that the inquiry content is correct to the response information generation unit 702. Notify to. On the other hand, when the request number 5043 stores a request number other than "100", that is, when the request content is other than "all operation explanations", the request information analysis unit 701 indicates that the inquiry content is invalid. Notify the response information generation unit 702 of the indicated check result.

The response information generation unit 702 is an example of the response information generation means according to the present invention. The response information generation unit 702 responds to the user's inquiry based on the analysis result information supplied from the request information analysis unit 701, the check result notified from the request information analysis unit 701, and the operation explanation information DB 741. Generate the answer information shown. FIG. 28 shows an example of the correspondence between the request information and the generated answer information.

The answer information transmission unit 703 transmits the answer information generated by the answer information generation unit 702 to the voice processing device 5. The response information generation unit 702 and the response information transmission unit 703 are examples of the response means according to the present invention.

FIG. 29 is a flowchart showing a procedure of voice operation support processing executed by the voice operation support device 7. When the request information transmitted from the voice processing device 5 is received by the request information receiving unit 700 (step S401; YES), the request information analysis unit 701 analyzes the request information (step S402).

The request information analysis unit 701 feeds the response information generation unit 702 the analysis result information including each data excluding the dummy data extracted from the request information and all the operation numbers corresponding to the home electric appliance 2 indicated by the request information. Supply. Further, the request information analysis unit 701 notifies the answer information generation unit 702 of the check result indicating the correctness of the inquiry content indicated by the request information.

The response information generation unit 702 generates response information based on the analysis result information supplied from the request information analysis unit 701, the check result notified from the request information analysis unit 701, and the operation explanation information DB 741 (step). S403).

The answer information transmission unit 703 transmits the answer information generated by the answer information generation unit 702 to the voice processing device 5 (step S404).

As described above, according to the voice operation support system according to the first embodiment, the voice processing device 5 is a device operation in which information related to all operations corresponding to the model is registered for each model of the home electric appliance 2. Information DB 541 is held. Therefore, regardless of the manufacturer and the operation category, the format of the operation information transmitted to the device control device 6 can be standardized, and more manufacturers can be expected to participate in the voice operation service. ..

Further, the voice operation support device 7 holds an operation explanation information DB 741 in which information for explaining to the user is registered for each operation of each model of the home electric appliance 2. Then, when the voice operation support device 7 receives an inquiry from the user about the operation of the home electric appliance 2 installed in the user's house H via the voice processing device 5, the voice operation support device 7 responds to the inquiry by voice. Generate answer information for. Therefore, the user can easily perform not only the basic operation common to the manufacturers but also the operation peculiar to the manufacturer of the home electric appliance 2. As a result, the user can fully utilize the home electric appliance 2 owned by the user. By providing a service in which the home electric appliance 2 is fully utilized in this way, it is possible to promote sales of the home electric appliance 2.

The voice processing device 5 may include the same functions as the voice operation support device 7 without the voice operation support device 7, or the device control device 6 may include the voice operation support device 7. It may include the same function as.

Further, the voice processing device 5, the device control device 6, and the voice operation support device 7 may be realized by one device.

Embodiment 2.
Subsequently, a second embodiment of the present invention will be described. In the following description, the components and the like common to those in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

FIG. 30 is a diagram showing an overall configuration of a voice operation system 1A including the voice operation support system according to the second embodiment of the present invention. The voice operation system 1A includes one or more home appliances 2, a voice input / output device 3, a home gateway 4, a voice processing device 5A, a device control device 6, and a voice operation support device 7A. Further, the voice operation support system according to the second embodiment is composed of a voice input / output device 3, a home gateway 4, a voice processing device 5A, and a voice operation support device 7A.

The hardware configuration of the voice processing device 5A is the same as that of the voice processing device 5 of the first embodiment (see FIG. 4). Functionally, as shown in FIG. 31, the voice processing device 5A has a voice data receiving unit 500, a voice data analysis unit 501A, an operation information generation unit 502, an operation information transmission unit 503, and a request information generation unit. It includes a request information transmission unit 505, a result information reception unit 506, a voice data generation unit 507, a voice data transmission unit 508, a response information reception unit 509, and an advice information reception unit 510.

The voice data analysis unit 501A executes the same processing as the voice data analysis unit 501 of the voice processing device 5 of the first embodiment. That is, the voice data analysis unit 501A analyzes the voice data received by the voice data receiving unit 500 and generates voice content data indicating the utterance content of the user. When the generated voice content data indicates an operation for the home electric appliance 2, the voice data analysis unit 501A supplies the voice content data to the operation information generation unit 502.

Further, the voice data analysis unit 501A requests the voice content data when the generated voice content data indicates an inquiry about the operation of the home appliance 2 or a request for advice on the operation of the home appliance 2. It is supplied to the information generation unit 504A.

When the voice content data supplied from the voice data analysis unit 501A indicates an inquiry about the operation of the home appliance 2, the request information generation unit 504A is the same as the request information generation unit 504 of the voice processing device 5 of the first embodiment. , The first request information for requesting the answer to the inquiry is generated based on the voice content data, the device operation information DB 541, the customer information DB 542, and the customer device information DB 543. The format of the first request information is the same as the request information generated by the request information generation unit 504 in the first embodiment (see FIG. 20).

Further, when the voice content data supplied from the voice data analysis unit 501A indicates a request for advice on the operation of the home appliance 2, the request information generation unit 504A determines that the home appliance of the user's home is based on the voice content data. Generate a second request information for requesting advice on the operation of 2. As shown in FIG. 32, the second request information includes items such as customer ID 5044, category 5045, and request number 5046. The customer ID of the user is stored in the customer ID 5044. The customer ID is an ID assigned to identify each customer in the voice operation service.

The target category name is stored in the category 5045. The request number is stored in the request number 5046. FIG. 33 shows the relationship between the request number and the request content in the present embodiment. Further, FIG. 34 shows an example of the correspondence between the utterance content of the user and the generated second request information. The request information transmission unit 505 transmits the first request information or the second request information generated by the request information generation unit 504A to the voice operation support device 7A.

The advice information receiving unit 510 receives the advice information from the voice operation support device 7A. The advice information is information generated by the voice operation support device 7A in response to the second request information transmitted earlier, and is information indicating advice regarding an operation in a category designated by the user. The advice information receiving unit 510 supplies the received advice information to the voice data generation unit 507. The voice data generation unit 507 generates voice data by a known voice synthesis process based on the advice information supplied from the advice information reception unit 510, and supplies the generated voice data to the voice data transmission unit 508. The voice data is an example of the third voice data according to the present invention. The voice data transmission unit 508 transmits the voice data supplied from the voice data generation unit 507 to the voice input / output device 3.

Subsequently, the voice operation support device 7A will be described. The hardware configuration of the voice operation support device 7A is the same as that of the voice operation support device 7 of the first embodiment (see FIG. 12). However, as shown in FIG. 35, the operation record information DB 743 is further stored in the secondary storage device 74 of the voice operation support device 7A. The operation record information DB 743 is a database in which information related to the operation record of each home electric appliance 2 installed in each customer's house is registered. The operation record information DB 743 is constructed by a DB construction unit (not shown) based on the operation explanation information DB 741 and the device information DB 742.

Specifically, as shown in FIG. 36, the operation record information DB 743 includes items such as device ID 7430, model name 7431, installation location 7432, operation number 7433, category 7434, operation name 7435, and operation record 7436 for each customer. Records including the above are registered for the number of all home appliances 2 installed in the customer's house.

The device ID 7430 stores the device ID of the home electric appliance 2. The model name 7431 stores the name of the model to which the home electric appliance 2 belongs. The installation location 7432 stores a name indicating the installation location of the home electric appliance 2. The operation number is stored in the operation number 7433. In the category 7434, the category name corresponding to the operation number stored in the operation number 7433 is stored. The operation name 7435 stores the operation name corresponding to the operation number stored in the operation number 7433.

The operation record 7436 stores the operation record. Specifically, the operation record 7436 stores the number of operations of the operation in the past year. For example, each time the voice processing device 5A transmits the operation information to the device control device 6, the voice processing device 5A transmits an operation result notification in which the device ID and the operation number included in the operation information are stored to the voice operation support device 7A. When the voice operation support device 7A receives the operation record notification from the voice processing device 5A, the voice operation support device 7A increments the number of operations stored in the operation record 7436 of the record corresponding to the operation record notification in the operation record information DB 743. Alternatively, the voice operation support device 7A may update the operation record information DB 743 by receiving the same notification as the above operation result notification from the device control device 6.

Functionally, as shown in FIG. 37, the voice operation support device 7A has a request information receiving unit 700A, a request information analysis unit 701, an answer information generation unit 702, an answer information transmission unit 703, and advice information generation. A unit 704 and an advice information transmission unit 705 are provided.

The request information receiving unit 700A receives the first request information (see FIG. 20) or the second request information (see FIG. 32) transmitted from the voice operation support device 7A. The request information receiving unit 700A supplies the received first request information to the request information analysis unit 701, and supplies the received second request information to the advice information generation unit 704.

The advice information generation unit 704 is an example of the advice information generation means according to the present invention. The advice information generation unit 704 generates advice information based on the second request information and the operation record information DB 743. Specifically, first, the advice information generation unit 704 calculates the usage tendency of the operation of the designated category name for each home electric appliance 2 installed in the customer's house by the following formula 1.

Usage tendency (%) = average number of operations for the category name ÷ number of operations for which the operation number is "1" in the basic operation (Equation 1)

FIG. 38 is an example of the usage tendency of the “energy saving operation”, and FIG. 39 is an example of the usage tendency of the “watching operation”.

The advice information generation unit 704 selects a home appliance 2 (hereinafter referred to as an advice target) to be advised from the calculated usage tendency. For example, when the category name specified by the user is "energy saving operation", the advice information generation unit 704 uses the home appliance 2 having a tendency to use "energy saving operation" less than 50% among all the home appliances 2 in the user's home. Is selected as an advice target.

Further, for example, when the category name specified by the user is "watching operation", the advice information generation unit 704 is a household appliance having a usage tendency of "watching operation" of less than 20% among all home appliances 2 in the user's home. Select device 2 as an advice target.

Then, the advice information generation unit 704 generates advice information based on the selected advice target and the category name designated by the user. FIG. 40 shows an example of the correspondence between the second request information and the generated advice information.

As described above, according to the voice operation support system according to the second embodiment, the voice operation support device 7A generates advice information for transmitting advice on the operation of the category designated by the user by voice. As a result, the user can fully utilize the home electric appliance 2 owned by the user.

In the above embodiment, the advice information generation unit 704 of the voice operation support device 7A determines the necessity of advice by managing the number of operations of each operation of each home electric appliance 2 in the past year. However, the period of the number of operations to be managed is an arbitrary design matter.

Further, when home appliances 2 of the same manufacturer and different model names are installed in the house H and at least a part of the operations in categories other than the basic operations are common, the advice information generation unit. The 704 may determine the necessity of advice in consideration of the usage status of the common operation in the other home electric appliance 2. For example, as shown in FIG. 36, the energy-saving operation is common to the air conditioner in the living room having the device ID R111 and the air conditioner in the bedroom having the device ID R112. However, although the living room air conditioner has a record of energy saving operation in the past year, the bedroom air conditioner has no record of energy saving operation in the past year.

In the above case, it is preferable that the advice information generation unit 704 does not generate the advice information regarding the energy saving operation regardless of the usage status of the energy saving operation of the air conditioner in the bedroom. This is because it can be seen that the user is aware of the energy-saving operation of the air conditioner from the usage record of the energy-saving operation of the air conditioner in the living room. By determining the necessity of advice in this way, it is possible to convey more appropriate advice to the user.

The voice processing device 5A may include the same functions as the voice operation support device 7A without the voice operation support device 7A, or the device control device 6 may include the voice operation support device 7A. It may include the same function as.

Further, the voice processing device 5A, the device control device 6, and the voice operation support device 7A may be realized by one device.

Embodiment 3.
Subsequently, the third embodiment of the present invention will be described. In the following description, the components and the like common to those in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

FIG. 41 is a diagram showing the overall configuration of the voice operation system 1B including the voice operation support system according to the third embodiment of the present invention. The voice operation system 1B includes one or more home appliances 2, a voice input / output device 3, a home gateway 4, a voice processing device 5B, a device control device 6, and a voice operation support device 7. Further, the voice operation support system according to the third embodiment is composed of a voice input / output device 3, a home gateway 4, a voice processing device 5B, and a voice operation support device 7.

The hardware configuration of the voice processing device 5B is the same as that of the voice processing device 5 of the first embodiment (see FIG. 4). However, as shown in FIG. 42, the secondary storage device 54 of the voice processing device 5B stores the customer device information DB 543A instead of the customer device information DB 543, and further stores the operation history DB 544 and the operation sequence DB 545. Has been done.

Like the customer device information DB 543, the customer device information DB 543A is a database in which information about all the home electric appliances 2 installed in each customer's house is registered. The record registered in the customer device information DB 543A includes, in addition to the same items as the customer device information DB 543, a corresponding operation sequence 5435 as shown in FIG. 43.

The corresponding operation sequence 5435 stores information about the corresponding operation sequence information. The operation sequence information is information for automatically sequentially executing operations on the other one or a plurality of home appliances 2 by voice operation on one home appliance 2, and is registered in the operation sequence DB 545 described later. Has been done. In the information stored in the corresponding operation sequence 5435, the registration number, the sequence number, etc. of the corresponding operation sequence information are stored. The registration number and sequence number will be described later. If the corresponding operation sequence information does not exist, the corresponding operation sequence 5435 stores information indicating that fact.

Returning to FIG. 42, the operation history DB 544 is a database in which the history of voice operations at each customer's home is registered. Specifically, as shown in FIGS. 44 and 45, the operation history DB 544 contains the date and time 5440, the operator 5441, the device ID 5442, the model name 5443, the installation location 5444, the operation number 5445, and the operation name 5446 for each customer. Records including items such as setting contents 5447 and continuous index 5448 are stored for a predetermined period.

The date and time 5440 stores information indicating the date and time (year / month / day / time) when the operation was performed. The name of the operator who performed the operation is stored in the operator 5441. The device ID 5442 stores the device ID of the operated home electric appliance 2. The model name 5443 stores the name of the model to which the home electric appliance 2 belongs. The installation location 5444 stores a name indicating the installation location of the home electric appliance 2. The operation number of the operation is stored in the operation number 5445. The operation name of the operation is stored in the operation name 5446. The setting contents set in the operation are stored in the setting contents 5447.

The continuous index 5448 stores the continuous index. The continuous index is an index showing continuity with the previous operation. The details of the continuous index will be described later.

Returning to FIG. 42, the operation sequence DB 545 is a database in which the above-mentioned operation sequence information is registered. The operation sequence information is registered in the operation sequence DB 545 by the user who uses the voice operation service by operating the smartphone, tablet terminal, or the like that has activated the voice operation application described above. Alternatively, the operation sequence information may be generated by the voice processing device 5B based on the operation history DB 544 of each customer and registered in the operation sequence DB 545. Alternatively, the operation sequence information may be generated by the operator of the voice operation service and registered in the operation sequence DB 545 in advance.

As shown in FIG. 46, the operation sequence information DB 545 includes registration number 5450, time zone 5451, sequence number 5452, model name 5453, installation location 5454, operation number 5455, operation name 5456, setting content 5457, description 5458, and the like. A record containing an item is registered. One operation sequence information is composed of a plurality of records. The registration number is stored in the registration number 5450. The registration number indicates the order of registration of the operation sequence information in the operation sequence information DB 545. The same registration number is assigned to the records that make up the same operation sequence information.

In the time zone 5451, information indicating a time zone in which the operation based on the operation sequence information is effective (hereinafter, referred to as time zone information) is stored. The time zone information includes information indicating each of the target month and the target time. The same time zone information is stored in the time zone 5451 of each record constituting the same sequence information.

The sequence number 5452 stores the sequence number. The sequence number indicates the order in which the operation is executed in a plurality of operations in the operation sequence information. The model name 5453 stores the name of the model of the home electric appliance 2 that is the target of the operation.

The installation location 5454 stores a name indicating the installation location of the home electric appliance 2. The operation number of the operation is stored in the operation number 5455. The operation name of the operation is stored in the operation name 5456. The setting content 5457 stores the setting content set in the operation. The description 5458 stores information (hereinafter, referred to as explanatory information) indicating a description of the operation sequence information. The same explanatory information is stored in the explanatory 5458 of each record constituting the same operation sequence information.

Functionally, as shown in FIG. 47, the voice processing device 5B has a voice data receiving unit 500, a voice data analysis unit 501B, an operation information generation unit 502A, an operation information transmission unit 503, and a request information generation unit. 504, request information transmission unit 505, result information reception unit 506, voice data generation unit 507, voice data transmission unit 508, response information reception unit 509, continuous operation detection unit 511, and similar sequence search unit 512. And a sequence setting proposal unit 513.

The voice data analysis unit 501B executes the same processing as the voice data analysis unit 501 of the voice processing device 5 of the first embodiment. That is, the voice data analysis unit 501B analyzes the voice data received by the voice data receiving unit 500 and generates voice content data indicating the utterance content of the user. When the generated voice content data indicates an operation for the home electric appliance 2, the voice data analysis unit 501B supplies the voice content data to the operation information generation unit 502A. Further, the voice data analysis unit 501B identifies the speaking user (that is, the operator) with reference to the customer information DB 542, and supplies the name of the identified operator to the operation information generation unit 502A. It is assumed that the customer information DB 542 registers data related to the voice of each user at each customer's home.

Further, when the generated audio content data indicates an answer to the operation sequence setting proposal, the audio data analysis unit 501B supplies the audio content data to the sequence setting proposal unit 513.

The operation information generation unit 502A generates operation information based on the voice content data supplied from the voice data analysis unit 501B, the device operation information DB 541, the customer information DB 542, the customer device information DB 543A, and the operation sequence DB 545. ..

Further, the operation information generation unit 502A generates a record showing the history of the operation and registers it in the operation history DB 544. At that time, the operation information generation unit 502A calculates the continuous index of the operation and stores it in the continuous index 5448. The continuous index is calculated by the following formula 2.

In each parameter of the above equation 2, [interval] is the elapsed time (minutes) from the previous operation. The [operator] is "1" when the current operator and the previous operator are the same, and "0" when they are different. [Equipment] is "1" when the home electric appliance 2 operated this time and the home electric appliance 2 operated last time are the same, and "0" when they are different. [Model] is "1" when the model of the home electric appliance 2 operated this time and the model of the home electric appliance 2 operated last time are the same, and is "0" when they are different. The [installation location] is "1" when the installation location of the home electric appliance 2 operated this time and the installation location of the home electric appliance 2 operated last time are the same, and "0" when they are different.

For example, in each operation shown in FIG. 44, the values of the above parameters are as shown in FIG.

In the above equation 2, the weight to be applied to each parameter is set in consideration of the influence of the parameter on the continuity of the operation. In this example, the weight corresponding to [Interval] is set to "10", which is the largest, and the weight corresponding to [Operator] and [Model] is set to "0.5", respectively.

The continuous index can be derived by various methods such as using a deep learning RNN (Recurrent Neural Network).

Returning to FIG. 47, the continuous operation detection unit 511 checks the operation history DB 544 at a predetermined timing (for example, at regular time intervals), and performs a plurality of operations (hereinafter referred to as continuous operations) in which continuity is recognized. To detect. Specifically, when the continuous operation detection unit 511 has an operation in which the continuous index is a predetermined reference value (for example, 10.00) or more, the continuous operation detection unit 511 detects the operation immediately before the operation and the operation as a continuous operation. When the operation whose continuous index is equal to or more than the reference value is continuous, the continuous operation includes three or more operations.

The continuous operation detection unit 511 stores information about the detected continuous operation in a continuous operation table (not shown). The continuous operation table is expanded in the RAM 53 or the secondary storage device 54. An example of the continuous operation table is shown in FIG.

Returning to FIG. 47, the similar sequence search unit 512 searches for the operation sequence information included in the continuous operation detected by the continuous operation detection unit 511. Specifically, the similar sequence search unit 512 sequentially picks up continuous operations from the continuous operation table, and searches the operation sequence DB 545 for the operation sequence information included in the picked up continuous operations.

For example, in the continuous operation table shown in FIG. 49, the operation sequence information of the registration number “1” of the operation sequence DB 545 (see FIG. 46) is added to each continuous operation of the numbers “1”, “4”, and “6”. It is included.

When the same operation sequence information appears a specified number of times (for example, three times) or more as a result of performing the above search for each continuous operation of the continuous operation table, the similar sequence search unit 512 registers the operation sequence information. Notify the sequence setting proposal unit 513 of the number.

The sequence setting proposal unit 513 is an example of the sequence setting proposal means according to the present invention. The sequence setting proposal unit 513 generates proposal information for proposing a series of operation settings (hereinafter referred to as sequence setting) indicated by the operation sequence information of the notified registration number. The proposed information includes explanatory information read from the description 5458 of the record corresponding to the registration number of the operation sequence DB 545. The sequence setting proposal unit 513 supplies the generated proposal information to the voice data generation unit 507.

The voice data generation unit 507 generates voice data by a known voice synthesis process based on the proposal information supplied from the sequence setting proposal unit 513, and supplies the generated voice data to the voice data transmission unit 508. The voice data transmission unit 508 transmits the voice data supplied from the voice data generation unit 507 to the voice input / output device 3.

When a positive answer such as "yes", "OK", "please", "do", etc. is obtained from the user in response to the above proposal, the sequence setting proposal unit 513 may perform the plurality of such proposals at the customer's home. Set the sequence for the home appliance 2. Specifically, the sequence setting proposal unit 513 indicates information (registration number, sequence number, etc.) related to the operation sequence information in the corresponding operation sequence 5435 of the record corresponding to the home appliance 2 to be the target of the sequence setting in the customer device information DB 543A. ) Is stored.

When the operation information generation unit 502A generates operation information from voice content data based on the user's voice operation and supplies it to the operation information transmission unit 503, the operation information generation unit 502A is based on the current date and time, the customer device information DB 543A, and the operation sequence DB 545. , It is determined whether or not the home appliance 2 to be operated corresponds to the first home appliance 2 in the sequence operation to be executed. For example, the current date and time is 6:00 am on August 1, and the registration number "1" and the sequence number "1" are stored in the corresponding operation sequence 5435 of the record corresponding to the home electric appliance 2 in the customer device information DB 543A. If so, the operation information generation unit 502A determines that the home electric appliance 2 corresponds to the first home electric appliance 2 in the sequence operation to be executed.

When it is determined that the home electric appliance 2 corresponds to the first home electric appliance 2 in the sequence operation to be executed, the operation information generation unit 502A refers to the home electric appliance 2 in which the sequence number after "2" of the same registration number is set. The operation information is generated in ascending order of the sequence number, and is sequentially supplied to the operation information transmission unit 503. As a result, at the user's home, for example, when the operation of the air conditioner in the bedroom is stopped by voice operation from 5 to 8 am in July to September, the operation of the air conditioner in the living room is automatically started.

As described above, according to the voice operation support system according to the third embodiment, the voice processing device 5B sets continuous operations for a plurality of home appliances 2 from the user's daily operation tendency for the home appliances 2. That is, the sequence setting is proposed to the user. Then, when the user consents, the voice processing device 5B sets the sequence for realizing the continuous operation. As a result, the user can make the plurality of home appliances 2 continuously execute desired operations by one voice operation. Therefore, the convenience of the user is enhanced.

The voice processing device 5B may propose to the user sequence settings for a plurality of home appliances 2 that are frequently continuously operated without referring to the operation sequence DB 545.

Further, the device control device 6 or the voice operation support device 7 may include the proposal and setting function of the sequence setting by the voice processing device 5B.

Further, the voice processing device 5B may include the same functions as the voice operation support device 7 without the voice operation support device 7, or the device control device 6 may include the voice operation support device 7. It may include the same function as.

Further, the voice processing device 5B, the device control device 6, and the voice operation support device 7 may be realized by one device.

Further, the proposal and setting function of the sequence setting by the voice processing device 5B may be included in the voice processing device 5A of the second embodiment.

The present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention.

In each of the above embodiments, in the voice processing devices 5, 5A, 5B, the voice processing program 540 stored in the secondary storage device 54 is executed by the CPU 51, so that the voice processing devices 5, 5A, 5B are executed. A functional unit (see FIGS. 17, 31, 47) has been realized. However, all or part of the functional parts of the voice processing devices 5, 5A and 5B may be realized by dedicated hardware. Similarly, all or part of the functional unit of the device control device 6 (see FIG. 26) may be realized by dedicated hardware, or the functional unit of the voice operation support devices 7 and 7A (FIG. 27). , All or part of FIG. 37) may be realized by dedicated hardware. The dedicated hardware is, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

The voice processing program 540, device control program 640, and voice operation support program 740 include CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), magneto-optical disc (Magneto-Optical Disc), and USB (Universal). Serial Bus) It is also possible to store and distribute in a computer-readable recording medium such as a memory, a memory card, or an HDD. Then, by installing the voice processing program 540, the device control program 640, and the voice operation support program 740 distributed in this way on a specific or general-purpose computer, the computer can be installed in the voice processing devices 5, 5A, 5B, and the device control device. 6. It is also possible to function as a voice operation support device 7, 7A.

Further, the voice processing program 540, the device control program 640, and the voice operation support program 740 are stored in a storage device of a server (not shown) on the Internet, and the voice processing devices 5, 5A, 5B, and the device control device 6 are stored from the server. , The voice processing program 540, the device control program 640, and the voice operation support program 740 may be downloaded to the voice operation support devices 7 and 7A.

The present invention is capable of various embodiments and modifications without departing from the broad spirit and scope. Further, the above-described embodiments are for explaining the present invention, and do not limit the scope of the present invention. That is, the scope of the present invention is shown not by the embodiment but by the claims. And various modifications made within the scope of the claims and within the equivalent meaning of the invention are considered to be within the scope of the present invention.

INDUSTRIAL APPLICABILITY The present invention can be suitably adopted for a system in which a home appliance installed in a house is controlled by a user's voice.

1,1A, 1B voice operation system, 2 home appliances, 3 voice input / output device, 4 home gateway, 5,5A, 5B voice processing device, 6 device control device, 7,7A voice operation support device, 30 microphone, 31A / D converter, 32 speakers, 33,50,60,70 communication interface, 34,51,61,71 CPU, 35,52,62,72 ROM, 36,53,63,73 RAM, 37,54,64, 74 Secondary storage device, 38, 55, 65, 75 bus, 300 voice input unit, 301 transmitter, 302 receiver, 303 audio output, 500 audio data receiver, 501, 501A, 501B audio data analysis unit, 502 , 502A operation information generation unit, 503 operation information transmission unit, 504, 504A request information generation unit, 505 request information transmission unit, 506 result information reception unit, 507 voice data generation unit, 508 voice data transmission unit, 509 response information reception unit. 510 Advice information receiving unit, 511 Continuous operation detection unit, 512 Similar sequence search unit, 513 Sequence setting proposal unit, 600 Operation information receiving unit, 601 Control command generation unit, 602 Control command transmission unit, 603 Response data reception unit, 604 Result information transmission unit, 700, 700A request information reception unit, 701 request information analysis unit, 702 response information generation unit, 703 response information transmission unit, 704 advice information generation unit, 705 advice information transmission unit

Claims (12)

A voice data receiving means for receiving the first voice data based on the user's voice from the voice input / output device, and
When the received first voice data indicates an inquiry about the operation of the home electric appliance, refer to the operation explanation information database associated with the information indicating the explanation about the operation peculiar to the model for each model of the home electric appliance. Then, the answer information generation means for generating the answer information for the inquiry,
A voice operation support system including a voice data transmission means for transmitting a second voice data indicating a voice based on the answer information to the voice input / output device. Further provided with an advice information generation means for generating advice information for urging the user to perform the unique operation based on the usage status of the unique operation of the home electric appliance.
The voice operation support system according to claim 1, wherein the voice data transmission means further transmits a third voice data indicating voice based on the advice information to the voice input / output device. The voice operation support system according to claim 1 or 2, further comprising a sequence setting proposing means for proposing a continuous operation setting for different home electric appliances to the user based on the operation history of the home electric appliance. The voice operation support system according to any one of claims 1 to 3 and the voice operation support system.
When the received first voice data indicates an operation for the home electric appliance, a control command generation means for generating a control command for controlling the home electric appliance based on the operation content indicated by the first voice data. When,
A voice operation system including a control command transmitting means for transmitting the control command to the home electric appliance. A voice data receiving means for receiving the first voice data based on the user's voice from the voice input / output device, and
When the received first voice data indicates an inquiry about the operation of the home electric appliance, the answer request means for requesting the voice operation support device for the answer information indicating the answer to the inquiry, and the answer request means.
A voice processing device comprising: a voice data transmission means for transmitting a second voice data indicating a voice based on the answer information obtained from the voice operation support device to the voice input / output device. For each model of home electric appliance, a storage device that stores an operation explanation information database associated with information indicating explanations about operations specific to the model, and a storage device.
A voice operation support device comprising a response means for generating response information for an inquiry from a voice processing device regarding the operation of the home electric appliance and transmitting the response information to the voice processing device with reference to the operation explanation information database. The sixth aspect of claim 6 further comprises an advice information generation means that generates advice information that prompts the user to perform the unique operation based on the usage status of the unique operation of the home electric appliance and transmits the advice information to the voice processing device. Voice operation support device. The voice data receiving means receives the first voice data based on the user's voice from the voice input / output device, and receives the first voice data.
When the received first voice data indicates an inquiry about the operation of the home electric appliance, the answer information generating means associates the information indicating the explanation with respect to the operation peculiar to the model for each model of the home electric appliance. Refer to the operation explanation information database to generate answer information for the inquiry,
A voice operation support method in which a voice data transmitting means transmits a second voice data indicating a voice based on the answer information to the voice input / output device. The advice information generation means generates advice information that prompts the user to perform the unique operation based on the usage status of the unique operation of the home electric appliance.
The voice operation support method according to claim 8, wherein the voice data transmission means further transmits a third voice data indicating voice based on the advice information to the voice input / output device. The voice operation support method according to claim 8 or 9, wherein the sequence setting proposing means proposes to the user continuous operation settings for different home appliances based on the operation history of the home appliances. For each model of home electric appliance, a computer equipped with a storage device that stores an operation explanation information database associated with information indicating explanations about operations specific to the model.
A program that refers to the operation explanation information database and functions as a response means for generating response information for an inquiry from a voice processing device regarding the operation of the home electric appliance and transmitting the response information to the voice processing device. The eleventh aspect of the present invention further functions as an advice information generation means for generating advice information for urging the user to perform the unique operation based on the usage status of the unique operation of the home electric appliance and transmitting the advice information to the voice processing device. The program described.

Images