WO2020070888A1 - 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム - Google Patents
音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラムInfo
- Publication number
- WO2020070888A1 WO2020070888A1 PCT/JP2018/037424 JP2018037424W WO2020070888A1 WO 2020070888 A1 WO2020070888 A1 WO 2020070888A1 JP 2018037424 W JP2018037424 W JP 2018037424W WO 2020070888 A1 WO2020070888 A1 WO 2020070888A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- voice
- information
- user
- voice data
- unit
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9032—Query formulation
- G06F16/90332—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
- G10L2015/228—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context
Definitions
- the present invention relates to a voice operation support system, a voice operation system, a voice processing device, a voice operation support device, a voice operation support method, and a program.
- Patent Document 1 a system for recognizing a user's voice input via a voice input device installed in a home by a server outside the home and controlling home electrical appliances in the home based on the recognition result.
- this type of system is operated by a maker that sells voice input devices and is operated by a server that performs voice recognition (hereinafter, referred to as a first server) and a maker that sells home electric appliances, and controls home electric appliances.
- a server that performs voice recognition
- a maker that sells home electric appliances and controls home electric appliances.
- a configuration for providing a voice operation service in cooperation with a server (hereinafter, referred to as a second server) is being developed.
- the first server and a plurality of second servers respectively corresponding to a plurality of different manufacturers are connected via the Internet, and the first server is connected to the plurality of manufacturers by a common voice operation independent of the difference between the manufacturers.
- similar control can be performed for different types of home electric appliances (for example, air conditioners).
- the present invention has been made in view of the above circumstances, and has as its object to provide a voice operation support system or the like that allows a user to make full use of home electric appliances owned by the user.
- a voice operation support system includes: Voice data receiving means for receiving first voice data based on the voice of the user from the voice input / output device; When the received first voice data indicates an inquiry about an operation in a home appliance held by the user, for each model of the home appliance, each of one or a plurality of unique operations of a plurality of manufacturers for the model About, referring to the operation description information database associated information indicating the description, answer information generating means for generating answer information to the inquiry, Voice data transmitting means for transmitting second voice data indicating voice based on the answer information to the voice input / output device.
- FIG. 1 is a diagram showing an overall configuration of a voice operation system including a voice operation support system according to a first embodiment of the present invention.
- FIG. 2 is a block diagram showing a hardware configuration of the audio input / output device according to the first embodiment.
- FIG. 2 is a block diagram showing a functional configuration of the voice input / output device according to the first embodiment.
- FIG. 2 is a block diagram showing a hardware configuration of the audio processing device according to the first embodiment.
- FIG. 7 is a diagram for describing a secondary storage device included in the audio processing device according to the first embodiment.
- FIG. 7 is a diagram for describing a device operation information DB stored in a secondary storage device included in the voice processing device according to the first embodiment.
- FIG. 4 is a diagram for explaining a customer device information DB stored in a secondary storage device included in the voice processing device according to the first embodiment.
- FIG. 2 is a block diagram illustrating a hardware configuration of the device control device according to the first embodiment.
- FIG. 7 is a diagram for describing a secondary storage device included in the device control device according to the first embodiment.
- FIG. 4 is a diagram for describing a device operation information DB stored in a secondary storage device included in the device control device according to the first embodiment.
- FIG. 7 is a diagram for describing a customer device information DB stored in a secondary storage device included in the device control device according to the first embodiment.
- FIG. 2 is a block diagram showing a hardware configuration of the voice operation support device according to the first embodiment.
- FIG. 4 is a diagram for explaining a secondary storage device included in the voice operation support device according to the first embodiment.
- Diagram for explaining operation explanation information DB air conditioner
- Diagram for explaining operation explanation information DB (television) stored in a secondary storage device included in the voice operation support device according to Embodiment 1.
- FIG. 4 is a diagram for describing a device information DB stored in a secondary storage device included in the voice operation support device according to the first embodiment.
- FIG. 2 is a block diagram showing a functional configuration of the audio processing device according to the first embodiment.
- FIG. 4 is a diagram showing a configuration of operation information in the first embodiment.
- FIG. 3 is a diagram showing a configuration of request information in the first embodiment. Diagram for explaining request number in the first embodiment
- FIG. 7 is a diagram showing an example of a correspondence relationship between user utterance contents and generated request information in the first embodiment.
- Flow chart showing the procedure of audio processing in Embodiment 1. Flow chart showing the procedure of operation information generation processing in the first embodiment Flowchart showing a procedure of request information generation processing in the first embodiment.
- FIG. 2 is a block diagram showing a functional configuration of the device control device according to the first embodiment.
- FIG. 2 is a block diagram showing a functional configuration of the voice operation support device according to the first embodiment.
- FIG. 7 is a diagram showing an example of a correspondence relationship between request information and generated answer information in the first embodiment.
- Flowchart showing a procedure of voice operation support processing in the first embodiment The figure which shows the whole structure of the voice operation system including the voice operation support system which concerns on Embodiment 2 of this invention.
- FIG. 6 is a block diagram showing a functional configuration of the audio processing device according to the second embodiment.
- FIG. 14 is a diagram showing a configuration of second request information in the second embodiment.
- FIG. 14 is a diagram for explaining a request number in the second embodiment.
- FIG. 9 is a diagram for explaining a secondary storage device included in the voice operation support device according to the second embodiment.
- FIG. 6 is a block diagram showing a functional configuration of the voice operation support device according to the second embodiment.
- FIG. 7 is a diagram for describing a usage tendency in the second embodiment.
- FIG. 7 is a diagram for describing a usage tendency in the second embodiment.
- FIG. 14 is a diagram illustrating an example of a correspondence relationship between second request information and generated advice information in the second embodiment.
- FIG. 10 is a diagram for describing a secondary storage device included in the audio processing device according to the third embodiment.
- FIG. 14 is a diagram for describing an operation history DB stored in a secondary storage device included in the voice processing device according to the third embodiment.
- FIG. 14 is a diagram for describing an operation history DB stored in a secondary storage device included in the voice processing device according to the third embodiment.
- FIG. 10 is a diagram for explaining an operation sequence DB stored in a secondary storage device included in the voice processing device according to the third embodiment.
- FIG. 6 is a block diagram showing a functional configuration of the audio processing device according to the third embodiment.
- FIG. 14 is a diagram for describing each parameter for calculating a continuous index in the third embodiment.
- FIG. 17 is a diagram illustrating an example of a continuous operation table in the third embodiment.
- FIG. 1 is a diagram showing an overall configuration of a voice operation system 1 including a voice operation support system according to Embodiment 1 of the present invention.
- the voice operation system 1 is a system capable of controlling the home appliances 2 installed in the house H by voice operation, and includes one or a plurality of home appliances 2, a voice input / output device 3, a home gateway 4, a voice It includes a processing device 5, a device control device 6, and a voice operation support device 7.
- the voice operation support system according to the first embodiment is a system for supporting a voice operation by a user, and includes a voice input / output device 3, a home gateway 4, a voice processing device 5, and a voice operation support device 7. It is composed of
- Each home appliance 2 is an air conditioner, a television, an induction heating (IH) cooker, or the like, and is installed in the house H (including the site).
- Each home appliance 2 is called a so-called IoT (Internet of Things) device, an information home appliance, a net home appliance, a smart home appliance, or the like, and is connected to the Internet via a home gateway 4 having a function as a broadband router.
- IoT Internet of Things
- the voice input / output device 3 is an example of the voice input / output device according to the present invention.
- the voice input / output device 3 is a device that provides various services based on voice input by a user's utterance, and is a so-called AI speaker (also referred to as a smart speaker).
- the audio input / output device 3 includes a microphone 30, an A / D converter 31, a speaker 32, a communication interface 33, a CPU (Central Processing Unit) 34, and a ROM (Read Only Memory) 35. And a RAM (Random Access Memory) 36 and a secondary storage device 37. These components are interconnected via a bus 38.
- the microphone 30 receives a voice uttered by the user, generates an analog voice signal from the input voice, and outputs the analog voice signal to the A / D converter 31.
- the A / D converter 31 removes noise from the analog audio signal output from the microphone 30, generates a digital audio signal, and supplies the digital audio signal to the CPU.
- the speaker 32 outputs a sound corresponding to the digital sound signal supplied from the CPU 34.
- the communication interface 33 is connected to the home gateway 4 by wire or wirelessly, and includes a network card for communicating with the audio processing device 5 connected to the Internet via the home gateway 4.
- the CPU 34 controls the voice input / output device 3 as a whole. Details of the functions of the voice input / output device 3 realized by the CPU 34 will be described later.
- the ROM 35 stores a plurality of firmware and data used when executing the firmware.
- the RAM 36 is used as a work area of the CPU 34.
- the ⁇ secondary storage device 37 is configured to include a readable / writable nonvolatile semiconductor memory such as an EEPROM (ElectricallyrErasable Programmable Read-Only Memory), a flash memory, or an HDD (Hard Disk Drive).
- the secondary storage device 37 stores a voice input / output program and data used when executing the voice input / output program.
- the voice input / output program is configured to input a user's voice, transmit voice data based on the input voice to the voice processing device 5, receive voice data from the voice processing device 5, and generate voice based on the received voice data. This is a program in which processing to be output is described.
- the audio input / output device 3 functionally includes an audio input unit 300, a transmission unit 301, a reception unit 302, and an audio output unit 303, as shown in FIG. These functional units are realized by the CPU 34 executing the above-described voice input / output program stored in the secondary storage device 37.
- the voice input unit 300 inputs voice uttered by the user and generates voice data based on the input voice. More specifically, the audio input unit 300 converts the analog audio signal indicating the user's voice into a digital audio signal by applying PCM (Pulse Code Modulation) to the analog audio signal, and further applies irreversible compression to the analog audio signal. Generate data.
- PCM Pulse Code Modulation
- the transmitting unit 301 transmits the audio data generated by the audio input unit 300 to the audio processing device 5 via the home gateway 4.
- the receiving unit 302 receives the audio data transmitted from the audio processing device 5 via the home gateway 4.
- the audio output unit 303 outputs audio based on the audio data received by the receiving unit 302.
- the voice processing device 5 is an example of the voice processing device according to the present invention.
- the voice processing device 5 is a server computer installed and operated by a maker, a sales company, and the like of the voice input / output device 3, and is connected to the Internet.
- the voice processing device 5 realizes voice operation on the home electric appliance 2 in cooperation with the voice input / output device 3 and a device control device 6 described later.
- the voice processing device 5 cooperates with the voice input / output device 3 and a voice operation support device 7 described later to provide a service for supporting a voice operation to the user.
- the audio processing device 5 includes a communication interface 50, a CPU 51, a ROM 52, a RAM 53, and a secondary storage device 54. These components are interconnected via a bus 55.
- the communication interface 50 is a device for connecting to the Internet and communicating with other devices such as the home gateway 4, the device control device 6, and the voice operation support device 7.
- the CPU 51 controls the voice processing device 5 as a whole. Details of the functions of the audio processing device 5 realized by the CPU 51 will be described later.
- the ROM 52 stores a plurality of firmware and data used when executing the firmware.
- the RAM 53 is used as a work area of the CPU 51.
- the secondary storage device 54 is a large-capacity storage device including a readable / writable nonvolatile semiconductor memory such as an EEPROM and a flash memory, or an HDD. As shown in FIG. 5, the secondary storage device 54 stores a voice processing program 540, a device operation information DB 541, a customer information DB 542, and a customer device information DB 543. In addition, the secondary storage device 54 stores a voice recognition DB or the like which is a database for voice recognition.
- the voice processing program 540 is a program related to voice processing for realizing a voice operation service.
- the voice processing program 540 describes voice data analysis processing, device operation-related processing, operation support-related processing, and the like.
- the voice data analysis process includes a process of analyzing voice data transmitted from the voice input / output device 3 using the above-described voice recognition DB to acquire voice content.
- the device operation-related process includes a process of generating operation information for the home electric appliance 2 based on the content of the acquired voice and transmitting the generated information to the device control device 6 and a process of inputting voice data based on the result information from the device control device 6. And a process of transmitting to the output device 3.
- the operation support-related processing is based on processing for transmitting request information for requesting a response to an inquiry from the user about the operation of the home electric appliance 2 to the voice operation support device 7 and on response information from the voice operation support device 7. And transmitting the audio data to the audio input / output device 3.
- the device operation information DB 541 is a database in which information on all operations corresponding to each model of the home electric appliance 2 is registered.
- a record including items such as an operation number 5410, a category 5411, an operation name 5412, and setting contents 5413 is registered in the device operation information DB 541 for each type of the home electric device 2.
- the model refers to a type (for example, an air conditioner, a television, an IH cooker, and the like) based on an essential function of the home electric appliance 2.
- the operation number is stored in the operation number 5410.
- the operation number is a number uniquely assigned to each operation in the model.
- the category 5411 stores a category name corresponding to the operation number stored in the operation number 5410.
- the category name is a name indicating a classification of the operation of the model in terms of the nature of the operation. For example, when the model of the home electric appliance 2 is an air conditioner, the category 5411 stores any one of “basic operation”, “energy saving operation”, and “watching operation”. Further, for example, when the model of the home electric appliance 2 is a television, one of “basic operation”, “energy saving operation”, and “dubbing operation” is stored in the category 5411.
- an operation whose category name corresponds to “basic operation” (for example, in an air conditioner, 1: start operation, 2: end operation) is common among manufacturers of home electric appliances 2 belonging to the model. It is an operation. In other words, it means that the operation corresponding to the “basic operation” can be handled by all the home electric appliances 2 belonging to the model of all manufacturers who support the voice operation service.
- the operation name 5412 stores an operation name.
- the operation name is the name of the operation corresponding to the operation number stored in the operation number 5410, and is also the content of the voice uttered by the user to perform the operation.
- the setting content 5413 stores the setting content.
- the setting contents are contents that can be set for the home appliance 2 at the time of the operation.
- the device operation information DB 541 is linked to the device operation information DB held by the device control device 6 by the audio processing device 5 and the device control device 6 appropriately cooperating with each other by OAuth 2.0, for example. As a result, the contents of the device operation information DB 541 are updated.
- the customer information DB 542 is a database in which personal information of each customer who has subscribed to the voice operation service is registered.
- the customer information DB 542 includes information such as the name, address, gender, date of birth, and IP (Internet @ Protocol) address of the voice input / output device 3 installed in the customer's home (that is, the customer's house H). Information is registered.
- the customer device information DB 543 is a database in which information on all the home appliances 2 installed in each customer home is registered. More specifically, as shown in FIG. 7, a record including items such as a device ID 5430, a model name 5431, a model name 5432, an installation location 5433, and a corresponding operation 5434 is stored in the customer device information DB 543 for each customer. Are registered for the number of all the home electric appliances 2 installed in the system.
- the device ID of the home appliance 2 is stored in the device ID 5430.
- the device ID is an ID (identification) assigned to identify each home appliance 2 in the voice operation service.
- the model name 5431 stores the name of the model to which the home appliance 2 belongs (for example, “air conditioner”, “TV”, “IH cooker”, etc.).
- the model name 5432 stores the model name (also referred to as a model number) of the home electric appliance 2.
- the installation location 5433 stores a name (for example, “living room”, “bedroom”, “kitchen”, etc.) indicating the installation location of the home appliance 2.
- the corresponding operation 5434 stores the operation numbers of all the operations that can be performed on the home appliance 2.
- the customer information DB 542 and the customer device information DB 543 are stored in the customer information DB and the customer device information DB held by the device control device 6 by the audio processing device 5 and the device control device 6 operating in cooperation with each other as appropriate according to OAuth 2.0. Linked to. As a result, the contents of the customer information DB 542 and the customer device information DB 543 are updated.
- the device control device 6 is a server computer installed and operated by a maker, a sales company, or the like of the home electric device 2 and is connected to the Internet.
- the device control device 6 realizes a voice operation on the home electric device 2 in cooperation with the voice processing device 5.
- the device control device 6 includes a communication interface 60, a CPU 61, a ROM 62, a RAM 63, and a secondary storage device 64. These components are interconnected via a bus 65.
- the communication interface 60 is a device for connecting to the Internet and communicating with other devices such as the home gateway 4 and the voice processing device 5.
- the CPU 61 controls the device control device 6 as a whole. Details of the functions of the device control device 6 realized by the CPU 61 will be described later.
- the ROM 62 stores a plurality of firmware and data used when executing the firmware.
- the RAM 63 is used as a work area of the CPU 61.
- the secondary storage device 64 is a large-capacity storage device including a readable and writable nonvolatile semiconductor memory such as an EEPROM and a flash memory, or an HDD. As shown in FIG. 9, the secondary storage device 64 stores a device control program 640, a device operation information DB 641, a customer information DB 642, and a customer device information DB 643.
- the device control program 640 is a program for realizing a voice operation on the household electrical appliance 2 in cooperation with the voice processing device 5.
- the device control program 640 includes a process for controlling the corresponding home appliance 2 based on operation information from the voice processing device 5, a process for transmitting result information indicating a control result of the home device 2 to the voice processing device 5, and the like. Is described.
- the device operation information DB 641 is a database in which information relating to operations of the home electric appliances 2 of the company is registered. Specifically, as shown in FIG. 10, a record including items such as a model name 6410, an operation number 6411, an operation name 6412, a setting content 6413, and the like is registered in the device operation information DB 641 for each model of the home electric device 2. ing.
- the model name of the home electric appliance 2 is stored in the model name 6410.
- the operation number 6411 stores an operation number.
- the operation name 6412 stores an operation name corresponding to the operation number stored in the operation number 6411.
- the setting content 6413 stores the setting content corresponding to the operation.
- the contents of each record of the device operation information DB 641 may be automatically generated from the data of the instruction manual of each type of home electric appliance 2 by a DB construction unit (not shown), and may be set in each record. .
- the device control device 6 and the audio processing device 5 cooperate appropriately according to OAuth 2.0, and thereby the device operation information DB 641 of the device control device 6 and the device operation information DB 541 of the audio processing device 5 And are linked. Thereby, the content of the device operation information DB 541 of the voice processing device 5 is updated.
- the customer information DB 642 is a database in which personal information of each customer who is a purchaser of the home appliance 2 of the company and subscribes to the voice operation service is registered. Information such as the name, address, gender, and date of birth of each customer is registered in the customer information DB 642.
- the customer device information DB 643 is a database in which information on the home electric appliances 2 installed in each customer's home is registered. Specifically, as shown in FIG. 11, the customer device information DB 643 includes, for each customer, a record including items such as a device ID 6430, a model name 6431, a model name 6432, an installation location 6433, a corresponding operation 6434, and a communication address 6435. Are registered for the number of home electric appliances 2 installed in the customer's home.
- the device ID 6430 stores the device ID assigned to the home electric device 2.
- the model name 6431 stores the name of the model to which the home appliance 2 belongs.
- the model name 6432 stores the model name of the home appliance 2.
- the installation location 6433 stores a name indicating the installation location of the home electric appliance 2.
- the corresponding operation 6434 stores the operation numbers of all the operations that can be performed on the home appliance 2.
- the communication address 6435 stores the IP address of the home appliance 2.
- a user who has purchased the home appliance 2 starts an application program (hereinafter, referred to as a voice operation application) for using a voice operation service installed in advance on a smartphone, a tablet terminal, or the like, and obtains personal information of the user and the home appliance 2.
- a predetermined procedure hereinafter, referred to as a registration procedure
- the personal information of the user and the information on the home electric appliance 2 in the user's house are transmitted from the smartphone, the tablet terminal, or the like to the device control device 6.
- the above-mentioned voice operation application can be downloaded from the device control device 6, another program distribution server, or the like to the smartphone, the tablet terminal, or the like.
- the model name, model name, and IP address are obtained from the home electric appliance 2 by communication via the home gateway 4 by a smartphone, a tablet terminal, or the like that has started the voice operation application.
- the OAuth 2.0 allows the device control device 6 and the voice processing device 5 to cooperate appropriately, and the customer information DB 642 and the customer device information DB 643 of the device control device 6 and the voice processing device 5
- the customer information DB 542 and the customer device information DB 543 are linked. Thereby, the contents of the customer information DB 542 and the customer device information DB 543 of the voice processing device 5 are updated.
- the voice operation support device 7 is an example of the voice operation support device according to the present invention.
- the voice operation support device 7 is a server computer operated by a plurality of manufacturers cooperating with the voice operation service or by a third party, and is connected to the Internet.
- the voice operation support device 7 realizes a voice operation support service in cooperation with the voice input / output device 3 and the voice processing device 5.
- the voice operation support device 7 includes a communication interface 70, a CPU 71, a ROM 72, a RAM 73, and a secondary storage device 74. These components are interconnected via a bus 75.
- the communication interface 70 is a device for connecting to the Internet and communicating with another device such as the voice processing device 5.
- the CPU 71 controls the voice operation support device 7 as a whole. Details of functions of the voice operation support device 7 realized by the CPU 71 will be described later.
- the ROM 72 stores a plurality of firmware and data used when the firmware is executed.
- the RAM 73 is used as a work area of the CPU 71.
- the secondary storage device 74 is a large-capacity storage device including a readable / writable nonvolatile semiconductor memory such as an EEPROM and a flash memory, or an HDD.
- the secondary storage device 74 stores a voice operation support program 740, an operation explanation information DB 741, and a device information DB 742, as shown in FIG.
- the secondary storage device 74 is an example of a storage device according to the present invention.
- the voice operation support program 740 is a program in which processing related to voice operation support is described.
- the operation explanation information DB 741 is a database in which information for explaining each operation for each model of the household electric appliance 2 to the user is registered. More specifically, as shown in FIGS. 14 and 15, the operation description information DB 741 includes, for each model of the home electric appliance 2, an operation number 7410, a category 7411, an operation name 7412, a schematic description 7413, a detailed description 7414, and a note 7415. A record including items such as ".” The contents of each record of the operation explanation information DB 741 may be automatically generated from the data of the instruction manual of the home electric appliance 2 of each model by a DB construction unit (not shown), and may be set in each record. .
- the operation number 7410 stores an operation number.
- the category 7411 stores a category name corresponding to the operation number stored in the operation number 7410.
- the operation name 7412 stores the operation name corresponding to the operation number.
- the brief description 7413 stores information indicating a brief description of the operation.
- the detailed description 7414 stores information indicating a detailed description of the operation.
- the notice 7415 stores information indicating a matter to be noted for the operation.
- the device information DB 742 is a database in which information on the home electric appliances 2 of all customers is registered. More specifically, as shown in FIG. 16, in the device information DB 742, records including items such as a device ID 7420 and a model name 7421, an installation location 7422, and a corresponding operation 7423 are stored in all home appliances in which each customer home is installed. The total number of devices 2 is registered.
- the device ID 7420 stores the device ID assigned to the home electric device 2.
- the model name 7421 stores the name of the model to which the home appliance 2 belongs.
- the installation location 7422 stores a name indicating the installation location of the home appliance 2.
- the corresponding operation 7423 stores the operation numbers of all the operations that can be performed on the home appliance 2.
- the device information DB 742 is linked to the customer device information DB 643 held by the device control device 6 by the voice operation support device 7 and the device control device 6 appropriately operating in cooperation with each other by OAuth 2.0, for example. Thereby, the content of the device information DB 742 is updated.
- the sound processing device 5 functionally includes a sound data receiving unit 500, a sound data analyzing unit 501, an operation information generating unit 502, an operation information transmitting unit 503, a request information generating unit 504, a request information transmitting unit 505, a result information receiving unit 506, an audio data generating unit 507, an audio data transmitting unit 508, and an answer information receiving unit 509.
- These functional units are realized by the CPU 51 executing the audio processing program 540 stored in the secondary storage device 54.
- the audio data receiving unit 500 is an example of an audio data receiving unit according to the present invention.
- the audio data receiving unit 500 receives audio data transmitted from the audio input / output device 3 via the home gateway 4.
- the audio data is an example of the first audio data according to the present invention.
- the voice data analysis unit 501 analyzes the voice data received by the voice data receiving unit 500 and generates text data (hereinafter referred to as voice content data) indicating the utterance content of the user.
- voice content data text data
- the audio data analysis unit 501 supplies the audio content data to the operation information generation unit 502.
- the audio data analysis unit 501 supplies the audio content data to the request information generation unit 504.
- the operation information generation unit 502 generates operation information based on the audio content data supplied from the audio data analysis unit 501, the device operation information DB 541, the customer information DB 542, and the customer device information DB 543. As shown in FIG. 18, the operation information includes items such as a device ID 5020, an operation number 5021, and setting contents 5022.
- the device ID of the target home appliance 2 is stored in the device ID 5020.
- the operation number 5021 stores the operation number corresponding to the target operation.
- the setting content 5022 stores the setting content specified by the user in the operation.
- the operation information generation unit 502 identifies the home electric appliance 2 to be subjected to the voice operation at the customer's house from the keywords indicating the location and the model included in the voice content data. For example, when the voice content data includes “living room” and “air conditioner”, the operation information generation unit 502 identifies the home appliance 2 with the device ID “R111” as the home appliance 2 to be subjected to the voice operation. I do.
- the operation information generation unit 502 determines the device ID of the identified home appliance 2 and keywords indicating operation names and / or setting contents included in the audio content data (for example, “operation start”, “operation end”, “power supply An operation number and / or setting contents are identified from “ON”, “power off”, “high temperature watching ON”, etc.).
- FIG. 19 shows an example of the correspondence between the utterance content of the user and the generated operation information.
- the operation information transmitting unit 503 transmits the operation information generated by the operation information generating unit 502 to the device control device 6.
- the request information generation unit 504 generates request information based on the audio content data supplied from the audio data analysis unit 501, the device operation information DB 541, the customer information DB 542, and the customer device information DB 543. As shown in FIG. 20, the request information includes items such as a device ID 5040, an operation number 5041, a category 5042, and a request number 5043.
- the device ID of the target home electric appliance 2 is stored in the device ID 5040.
- the operation number 5041 stores the operation number corresponding to the target operation. However, in the case of an inquiry that does not specify an operation, predetermined dummy data is stored in the operation number 5041.
- the category 5042 stores a target category name. However, in the case of an inquiry that does not specify a category, predetermined dummy data is stored in the category 5042.
- the request number 5043 stores a request number. The request number is a number uniquely assigned to each request as shown in FIG.
- the request information generation unit 504 identifies the home appliance 2 to be inquired at the customer's home from the keywords indicating the location and model included in the audio content data. For example, when the audio content data includes “living room” and “air conditioner”, the request information generation unit 504 identifies the home appliance 2 with the device ID “R111” as the home appliance 2 to be inquired. I do. The request information generation unit 504 stores the identified device ID in the device ID 5040.
- the request information generation unit 504 determines whether or not a keyword indicating an operation name or a category name exists in the audio content data. If neither keyword exists, the request information generation unit 504 stores “100” in the request number 5043. Further, the request information generation unit 504 stores dummy data in each of the operation number 5041 and the category 5042.
- the request information generation unit 504 stores “101” in the request number 5043.
- the request information generation unit 504 stores dummy data in the operation number 5041 and stores a keyword indicating the category name in the category 5042.
- the request information generation unit 504 identifies the operation number based on the device ID of the identified home appliance 2 and the operation name. The request information generation unit 504 stores the identified operation number in the operation number 5041. The request information generation unit 504 determines whether "detailed” or "instructions" exists in the audio content data. If neither exists, the request information generation unit 504 stores “102” in the request number 5043 and stores dummy data in the category 5042.
- the request information generation unit 504 stores “103” in the request number 5043 and stores dummy data in the category 5042. If “notes” exist in the audio content data, the request information generation unit 504 stores “104” in the request number 5043 and stores dummy data in the category 5042.
- the request information generation unit 504 generates request information according to the contents of the utterance of the user.
- FIG. 22 shows an example of the correspondence between the utterance content of the user and the generated request information.
- the request information transmission unit 505 transmits the request information generated by the request information generation unit 504 to the voice operation support device 7.
- the request information generation unit 504 and the request information transmission unit 505 are examples of a response request unit according to the present invention.
- the result information receiving unit 506 receives the result information from the device control device 6.
- the result information is information indicating whether or not the target home electric appliance 2 has been normally operated in accordance with the previously transmitted operation information.
- the result information receiving unit 506 supplies the received result information to the audio data generating unit 507.
- the voice data generating unit 507 Based on the result information supplied from the result information receiving unit 506, the voice data generating unit 507 generates voice data by a known voice synthesizing process, and supplies the generated voice data to the voice data transmitting unit 508.
- the audio data is an example of the second audio data according to the present invention.
- the audio data transmission unit 508 transmits the audio data supplied from the audio data generation unit 507 to the audio input / output device 3.
- the audio data transmission unit 508 is an example of an audio data transmission unit according to the present invention.
- the response information receiving unit 509 receives the response information from the voice operation support device 7.
- the answer information is information generated by the voice operation support device 7 in response to the previously transmitted request information, and is information indicating an answer to the user's inquiry.
- the answer information receiving unit 509 supplies the received answer information to the audio data generating unit 507.
- the voice data generation unit 507 generates voice data by a known voice synthesis process based on the response information supplied from the response information reception unit 509, and supplies the generated voice data to the voice data transmission unit 508.
- the audio data transmission unit 508 transmits the audio data supplied from the audio data generation unit 507 to the audio input / output device 3.
- FIG. 23 is a flowchart showing a procedure of voice processing performed by the voice processing device 5.
- the audio data analysis unit 501 analyzes the audio data (Step S102) and outputs the audio content. Generate data.
- Step S104 If the audio content data generated by the audio data analysis unit 501 indicates an operation on the household electrical appliance 2 (Step S103; YES), the audio processing device 5 executes an operation information generation process (Step S104). On the other hand, if the audio content data does not indicate an operation on the home appliance 2, that is, indicates an inquiry about an operation on the home appliance 2 (step S ⁇ b> 103; NO), the audio processing device 5 performs the request information generation process. Is executed (step S105).
- FIG. 24 is a flowchart showing the procedure of the operation information generation process.
- the operation information generation unit 502 determines whether the audio content data generated by the audio data analysis unit 501 lacks information necessary for generating operation information (Step S201).
- the operation information generation unit 502 lacks necessary information. It is determined that there is.
- step S201 If the necessary information is not insufficient (step S201; NO), the operation information generation unit 502 generates the operation information as described above (step S202).
- the operation information transmission unit 503 transmits the operation information generated by the operation information generation unit 502 to the device control device 6 (Step S203).
- the result information receiving unit 506 receives the result information from the device control device 6 (step S204).
- the audio data generation unit 507 generates audio data based on the result information received by the result information receiving unit 506 (Step S205).
- the operation information generation unit 502 generates inquiry information for inquiring the user about the insufficient information (step S206).
- the inquiry information includes text data such as "Is the air conditioner to be operated one of a living room air conditioner and a bedroom air conditioner?"
- the voice data generation unit 507 generates voice data based on the inquiry information generated by the operation information generation unit 502 (Step S207).
- the audio data transmission unit 508 transmits the audio data generated by the audio data generation unit 507 to the audio input / output device 3 (Step S208). Thereafter, the process returns to step S101 of the flowchart in FIG.
- the voice processing device 5 transmits the supplementary information and the previously generated voice content data. Based on the operation information, the operation information is generated and transmitted to the device control device 6.
- FIG. 25 is a flowchart showing the procedure of the request information generating process.
- the request information generation unit 504 determines whether the audio content data generated by the audio data analysis unit 501 lacks information necessary for generating request information (step S301).
- the request information generation unit 504 lacks necessary information. It is determined that there is.
- the request information generation unit 504 If the required information is not insufficient (step S301; NO), the request information generation unit 504 generates the request information as described above (step S302).
- the request information transmission unit 505 transmits the request information generated by the request information generation unit 504 to the voice operation support device 7 (Step S303).
- the answer information receiving unit 509 receives answer information from the voice operation support device 7 (step S304).
- the audio data generation unit 507 generates audio data based on the answer information received by the answer information receiving unit 509 (Step S305).
- the request information generation unit 504 generates inquiry information for inquiring the user about the insufficient information (step S306).
- the inquiry information includes text data such as "Is the air conditioner in question one of a living room air conditioner and a bedroom air conditioner?"
- the audio data generation unit 507 generates audio data based on the inquiry information generated by the request information generation unit 504 (Step S307).
- the audio data transmission unit 508 transmits the audio data generated by the audio data generation unit 507 to the audio input / output device 3 (Step S308). Thereafter, the process returns to step S101 of the flowchart in FIG.
- the voice processing device 5 transmits the supplementary information and the previously generated voice content data. Based on the request information, the request information is generated and transmitted to the voice operation support device 7.
- the device control device 6 functionally includes an operation information receiving unit 600, a control command generating unit 601, a control command transmitting unit 602, a response data receiving unit 603, and a result information transmitting unit. 604. These functional units are realized by the CPU 61 executing the device control program 640.
- the operation information receiving unit 600 receives the operation information (see FIG. 18) transmitted from the voice processing device 5.
- the control command generating unit 601 generates a control command for controlling the target home electric appliance 2 based on the operation information received by the operation information receiving unit 600, the device operation information DB 641, and the customer device information DB 643.
- the control command transmitting unit 602 transmits the control command generated by the control command generating unit 601 to the home electric appliance 2.
- the control command generator 601 is an example of a control command generator according to the present invention
- the control command transmitter 602 is an example of a control command transmitter according to the present invention.
- the response data receiving unit 603 receives the response data transmitted from the destination home electric appliance 2 after the control command transmitting unit 602 transmits the control command.
- Such response data includes information indicating whether or not an operation according to the control command has been performed.
- the response data receiving unit 603 notifies the result information transmitting unit 604 that the home electric appliance 2 has been normally operated. I do.
- the response data receiving unit 603 transmits result information indicating that the home appliance 2 is not normally operated. Notify section 604. If the response data receiving unit 603 does not receive the response data from the home appliance 2 even after a predetermined time has elapsed after the control command is transmitted to the home appliance 2, the home appliance 2 is Is notified to the result information transmitting unit 604 that no operation has been performed.
- the result information transmitting unit 604 transmits result information according to the content notified from the response data receiving unit 603 to the voice processing device 5.
- the voice operation support device 7 functionally includes a request information receiving unit 700, a request information analyzing unit 701, an answer information generating unit 702, and an answer information transmitting unit 703. These functional units are realized by the CPU 71 executing the voice operation support program 740.
- the request information receiving unit 700 receives the request information (see FIG. 20) transmitted from the voice processing device 5.
- the request information analyzer 701 analyzes the request information received by the request information receiver 700 with reference to the operation explanation information DB 741 and the device information DB 742. More specifically, the request information analysis unit 701 extracts data stored in each of the device ID 5040, operation number 5041, category 5042, and request number 5043 of the request information. Further, the request information analysis unit 701 refers to the device information DB 742 and acquires all the operation numbers corresponding to the home appliance 2 of the device ID stored in the device ID 5040.
- the request information analysis unit 701 supplies the response information generation unit 702 with analysis result information including the extracted data (excluding dummy data) and all the acquired operation numbers. Further, the request information analysis unit 701 checks whether or not the content of the inquiry indicated by the request information is correct, and notifies the answer information generation unit 702 of the check result. Specifically, when the operation number is stored in the operation number 5041, the request information analysis unit 701 checks whether the operation number corresponds to the home electric device 2. When the operation number corresponds to the home electric appliance 2, the request information analysis unit 701 notifies the response information generation unit 702 of a check result indicating that the content of the inquiry is correct. On the other hand, when the operation number does not correspond to the home electric appliance 2, the request information analysis unit 701 notifies the response information generation unit 702 of a check result indicating that the content of the inquiry is invalid.
- the request information analysis unit 701 refers to the operation explanation information DB 741 and checks whether or not the category name corresponds to the home electric appliance 2.
- the request information analysis unit 701 notifies the response information generation unit 702 of a check result indicating that the content of the inquiry is correct.
- the request information analysis unit 701 notifies the response information generation unit 702 of a check result indicating that the content of the inquiry is invalid.
- the request information analysis unit 701 checks whether or not “100” is stored in the request number 5043. If “100” is stored in the request number 5043, that is, if the request content is “all operation explanations”, the request information analysis unit 701 sends a check result indicating that the inquiry content is correct to the response information generation unit 702. Notify. On the other hand, when the request number 5043 stores a request number other than “100”, that is, when the request content is other than “all operation explanations”, the request information analysis unit 701 determines that the inquiry content is invalid. The check result shown is notified to the response information generation unit 702.
- the answer information generating unit 702 is an example of an answer information generating unit according to the present invention.
- the answer information generating unit 702 sends an answer to the user's inquiry based on the analysis result information supplied from the request information analyzing unit 701, the check result notified from the request information analyzing unit 701, and the operation explanation information DB 741. Generate answer information shown.
- FIG. 28 shows an example of the correspondence between the request information and the generated answer information.
- the response information transmitting unit 703 transmits the response information generated by the response information generating unit 702 to the voice processing device 5.
- the answer information generating unit 702 and the answer information transmitting unit 703 are examples of answer means according to the present invention.
- FIG. 29 is a flowchart showing a procedure of a voice operation support process executed by the voice operation support device 7.
- the request information analysis unit 701 analyzes the request information (Step S402).
- the request information analysis unit 701 sends to the response information generation unit 702 analysis result information including each data except the dummy data extracted from the request information and all the operation numbers corresponding to the home electric appliances 2 indicated by the request information. Supply. Further, the request information analysis unit 701 notifies the answer information generation unit 702 of a check result indicating whether the inquiry content indicated by the request information is correct.
- the response information generation unit 702 generates response information based on the analysis result information supplied from the request information analysis unit 701, the check result notified from the request information analysis unit 701, and the operation explanation information DB 741 (step S403).
- the response information transmitting unit 703 transmits the response information generated by the response information generating unit 702 to the voice processing device 5 (Step S404).
- the voice processing device 5 performs, for each model of the home electric appliance 2, a device operation in which information on all operations corresponding to the model is registered.
- the information DB 541 is held. For this reason, the format of the operation information to be transmitted to the device control device 6 can be made common regardless of the difference between manufacturers and the category of operation, and it is expected that more manufacturers will participate in the voice operation service. .
- the voice operation support device 7 holds an operation explanation information DB 741 in which information for explaining to the user about each operation for each model of the household electric appliance 2 is registered. Then, when the voice operation support device 7 receives a query about the operation of the home electric appliance 2 installed in the house H of the user from the user via the voice processing device 5, the voice operation support device 7 answers the query by voice. Generate answer information for For this reason, the user can easily perform not only the basic operation common to the makers, but also the operation unique to the manufacturer of the home electric appliance 2. As a result, the user can make full use of the home electric appliances 2 owned by the user. By providing a service that makes full use of the home electric appliance 2 in this way, sales promotion of the home electric appliance 2 can be achieved.
- the voice processing device 5 may include the same function as the voice operation support device 7 without including the voice operation support device 7, or the device control device 6 may include the voice operation support device 7. A function equivalent to the above may be included.
- the voice processing device 5, the device control device 6, and the voice operation support device 7 may be realized by one device.
- Embodiment 2 FIG. Next, a second embodiment of the present invention will be described.
- the same reference numerals are given to components and the like common to the first embodiment, and description thereof will be omitted.
- FIG. 30 is a diagram showing an overall configuration of a voice operation system 1A including a voice operation support system according to Embodiment 2 of the present invention.
- the voice operation system 1A includes one or a plurality of home appliances 2, a voice input / output device 3, a home gateway 4, a voice processing device 5A, a device control device 6, and a voice operation support device 7A.
- the voice operation support system according to the second embodiment includes a voice input / output device 3, a home gateway 4, a voice processing device 5A, and a voice operation support device 7A.
- the hardware configuration of the audio processing device 5A is the same as that of the audio processing device 5 of the first embodiment (see FIG. 4). As shown in FIG. 31, functionally, the audio processing device 5A includes an audio data receiving unit 500, an audio data analyzing unit 501A, an operation information generating unit 502, an operation information transmitting unit 503, a request information generating unit 504A, a request information transmitting unit 505, a result information receiving unit 506, an audio data generating unit 507, an audio data transmitting unit 508, an answer information receiving unit 509, and an advice information receiving unit 510.
- the audio data analysis unit 501A executes the same processing as the audio data analysis unit 501 of the audio processing device 5 according to the first embodiment. That is, the voice data analysis unit 501A analyzes the voice data received by the voice data receiving unit 500, and generates voice content data indicating the utterance content of the user. When the generated audio content data indicates an operation on the home electric appliance 2, the audio data analysis unit 501 ⁇ / b> A supplies the audio content data to the operation information generation unit 502.
- the audio data analysis unit 501A requests the audio content data when the generated audio content data indicates an inquiry about the operation of the home appliance 2 or indicates a request for advice on the operation of the home appliance 2.
- the information is supplied to the information generation unit 504A.
- the request information generation unit 504A is similar to the request information generation unit 504 of the audio processing device 5 according to the first embodiment. Based on the audio content data, the device operation information DB 541, the customer information DB 542, and the customer device information DB 543, first request information for requesting a response to the inquiry is generated.
- the format of the first request information is the same as the request information generated by the request information generation unit 504 in the first embodiment (see FIG. 20).
- the request information generation unit 504A performs, based on the audio content data, the home electric appliance of the user's home.
- the second request information for requesting advice on the second operation is generated.
- the second request information includes items such as a customer ID 5044, a category 5045, and a request number 5046, as shown in FIG.
- the customer ID 5044 stores the customer ID of the user.
- the customer ID is an ID assigned to identify each customer in the voice operation service.
- the category 5045 stores a target category name.
- the request number 5046 stores a request number.
- FIG. 33 shows the relationship between the request number and the request content in the present embodiment.
- FIG. 34 shows an example of the correspondence between the utterance content of the user and the generated second request information.
- the request information transmission unit 505 transmits the first request information or the second request information generated by the request information generation unit 504A to the voice operation support device 7A.
- the advice information receiving unit 510 receives the advice information from the voice operation support device 7A.
- the advice information is information generated by the voice operation support device 7A in response to the previously transmitted second request information, and is information indicating advice regarding an operation in a category designated by the user.
- the advice information receiving unit 510 supplies the received advice information to the audio data generation unit 507.
- the voice data generation unit 507 generates voice data by a known voice synthesis process based on the advice information supplied from the advice information reception unit 510, and supplies the generated voice data to the voice data transmission unit 508.
- the audio data is an example of the third audio data according to the present invention.
- the audio data transmission unit 508 transmits the audio data supplied from the audio data generation unit 507 to the audio input / output device 3.
- the hardware configuration of the voice operation support device 7A is the same as that of the voice operation support device 7 of the first embodiment (see FIG. 12).
- the operation result information DB 743 is further stored in the secondary storage device 74 of the voice operation support device 7A, as shown in FIG.
- the operation result information DB 743 is a database in which information relating to the operation results of each home appliance 2 installed in each customer home is registered.
- the operation result information DB 743 is constructed by a DB construction unit (not shown) based on the operation explanation information DB 741 and the device information DB 742.
- the operation result information DB 743 includes items such as a device ID 7430, a model name 7431, an installation location 7432, an operation number 7433, a category 7434, an operation name 7435, an operation result 7436, and the like for each customer. Are registered for all the home electric appliances 2 installed in the customer's home.
- the device ID 7430 stores the device ID of the home electric device 2.
- the model name 7431 stores the name of the model to which the home appliance 2 belongs.
- the installation location 7432 stores a name indicating the installation location of the home appliance 2.
- the operation number 7433 stores an operation number.
- the category 7434 stores a category name corresponding to the operation number stored in the operation number 7433.
- the operation name 7435 stores an operation name corresponding to the operation number stored in the operation number 7433.
- the operation result 7436 stores the result of the operation. Specifically, the operation result 7436 stores the number of times of the operation in the past year. For example, each time the voice processing device 5A transmits operation information to the device control device 6, the voice processing device 5A transmits an operation result notification in which the device ID and the operation number included in the operation information are stored to the voice operation support device 7A. Upon receiving the operation result notification from the voice processing device 5A, the voice operation support device 7A increments the number of operations stored in the operation result 7436 of the record corresponding to the operation result notification in the operation result information DB 743. Alternatively, the voice operation support device 7A may update the operation result information DB 743 by receiving a notification similar to the above operation result notification from the device control device 6.
- the voice operation support device 7A functionally includes a request information receiving unit 700A, a request information analysis unit 701, a response information generation unit 702, a response information transmission unit 703, and an advice information generation unit. And an advice information transmitting unit 705.
- the request information receiving unit 700A receives the first request information (see FIG. 20) or the second request information (see FIG. 32) transmitted from the voice operation support device 7A.
- the request information receiving unit 700A supplies the received first request information to the request information analyzing unit 701, and supplies the received second request information to the advice information generating unit 704.
- the advice information generation unit 704 is an example of the advice information generation unit according to the present invention.
- the advice information generation unit 704 generates advice information based on the second request information and the operation result information DB 743. More specifically, first, the advice information generation unit 704 calculates the usage tendency of the operation of the designated category name for each of the home electric appliances 2 installed in the customer's home by the following Expression 1.
- FIG. 38 shows an example of the usage tendency of the “energy saving operation”
- FIG. 39 shows an example of the usage tendency of the “watching operation”.
- the advice information generation unit 704 selects a home appliance 2 (hereinafter, referred to as an advice target) to be advised from the calculated usage tendency. For example, when the category name specified by the user is “energy saving operation”, the advice information generation unit 704 determines, among all the household appliances 2 in the user's home, the home appliance 2 whose usage tendency of “energy saving operation” is less than 50%. Is selected as the advice target.
- the advice information generation unit 704 determines, among all the home appliances 2 in the user's home, the home appliances whose usage tendency of “watching operation” is less than 20%.
- the device 2 is selected as a target for advice.
- the advice information generation unit 704 generates advice information based on the selected advice target and the category name specified by the user.
- FIG. 40 shows an example of the correspondence between the second request information and the generated advice information.
- the voice operation support device 7A generates advice information for conveying, by voice, advice on an operation in a category designated by the user. This allows the user to make full use of the home electric appliances 2 that the user has.
- the advice information generation unit 704 of the voice operation support device 7A determines the necessity of advice by managing the number of times of each operation of each home appliance 2 in the past year.
- the period of the number of operations to be managed is an arbitrary design item.
- the advice information generation unit 704 may determine whether or not the advice is necessary in consideration of the usage status of the common operation in the other home electric appliances 2. For example, as shown in FIG. 36, the air conditioner in the living room with the device ID R111 and the air conditioner in the bedroom with the device ID R112 have the same energy saving operation. However, although the air conditioner in the living room has a record of energy saving operation in the past year, the air conditioner in the bedroom has no record of the energy saving operation in the past year.
- the advice information generation unit 704 does not generate the advice information on the energy saving operation regardless of the usage state of the energy saving operation of the air conditioner in the bedroom. This is because the user is aware of the energy saving operation of the air conditioner based on the results of using the energy saving operation of the air conditioner in the living room. In this manner, by determining whether or not advice is required, more appropriate advice can be transmitted to the user.
- the voice processing device 5A may include the same function as the voice operation support device 7A without providing the voice operation support device 7A, or the device control device 6 may include the voice operation support device 7A. A function equivalent to the above may be included.
- the voice processing device 5A, the device control device 6, and the voice operation support device 7A may be realized by one device.
- Embodiment 3 FIG. Next, a third embodiment of the present invention will be described.
- the same reference numerals are given to components and the like common to the first embodiment, and description thereof will be omitted.
- FIG. 41 is a diagram showing an overall configuration of a voice operation system 1B including a voice operation support system according to Embodiment 3 of the present invention.
- the voice operation system 1B includes one or a plurality of home appliances 2, a voice input / output device 3, a home gateway 4, a voice processing device 5B, a device control device 6, and a voice operation support device 7.
- the voice operation support system according to the third embodiment includes a voice input / output device 3, a home gateway 4, a voice processing device 5B, and a voice operation support device 7.
- the hardware configuration of the audio processing device 5B is the same as that of the audio processing device 5 of the first embodiment (see FIG. 4). However, as shown in FIG. 42, the secondary storage device 54 of the voice processing device 5B stores a customer device information DB 543A instead of the customer device information DB 543, and further stores an operation history DB 544 and an operation sequence DB 545. Have been.
- the customer device information DB 543A is a database in which information on all the home appliances 2 installed in each customer's home is registered, like the customer device information DB 543.
- the record registered in the customer device information DB 543A includes, in addition to the same items as in the customer device information DB 543, a corresponding operation sequence 5435 as shown in FIG.
- the corresponding operation sequence 5435 stores information on the corresponding operation sequence information.
- the operation sequence information is information for enabling an operation for one or more other home electric appliances 2 to be automatically and sequentially executed by a voice operation for one home electric appliance 2, and is registered in an operation sequence DB 545 described later. Have been.
- the information stored in the corresponding operation sequence 5435 stores the registration number, sequence number, and the like of the corresponding operation sequence information. The registration number and sequence number will be described later. If there is no corresponding operation sequence information, the corresponding operation sequence 5435 stores information indicating that fact.
- the operation history DB 544 is a database in which a history of voice operations at each customer's home is registered. More specifically, as shown in FIGS. 44 and 45, the operation history DB 544 includes, for each customer, date and time 5440, operator 5441, device ID 5442, model name 5443, installation location 5444, operation number 5445, operation name 5446, Records including items such as the setting contents 5447 and the continuous index 5448 are stored for a predetermined period.
- the date and time 5440 stores information indicating the date and time (year / month / day / time) at which the operation was performed.
- the name of the operator who performed the operation is stored in the operator 5441.
- the device ID of the operated home appliance 2 is stored in the device ID 5442.
- the model name 5443 stores the name of the model to which the home appliance 2 belongs.
- the installation location 5444 stores a name indicating the installation location of the home appliance 2.
- the operation number 5445 stores the operation number of the operation.
- the operation name 5446 stores the operation name of the operation.
- the setting content 547 stores the setting content set in the operation.
- the continuous index is stored in the continuous index 5448.
- the continuous index is an index indicating the continuity with the immediately preceding operation. Details of the continuous index will be described later.
- the operation sequence DB 545 is a database in which the above-described operation sequence information is registered.
- the operation sequence information is registered in the operation sequence DB 545 by a user who uses the voice operation service through an operation via a smartphone, a tablet terminal, or the like that has activated the above-described voice operation application.
- the operation sequence information may be generated by the voice processing device 5B based on the operation history DB 544 of each customer and registered in the operation sequence DB 545.
- the operation sequence information may be generated by the operator of the voice operation service and registered in the operation sequence DB 545 in advance.
- the operation sequence information DB 545 includes a registration number 5450, a time zone 5451, a sequence number 5452, a model name 5453, an installation location 5454, an operation number 5455, an operation name 5456, a setting content 5457, a description 5458, and the like.
- a record containing the item is registered.
- One operation sequence information is composed of a plurality of records.
- the registration number 5450 stores a registration number.
- the registration number indicates the order in which the operation sequence information is registered in the operation sequence information DB 545.
- the same registration number is assigned to the records constituting the same operation sequence information.
- the time zone 5451 stores information indicating a time zone in which an operation based on the operation sequence information is valid (hereinafter, referred to as time zone information).
- the time zone information includes information indicating each of the target month and the target time.
- the same time zone information is stored in the time zone 5451 of each record constituting the same sequence information.
- the sequence number 5452 stores a sequence number.
- the sequence number indicates the order in which the operations are executed in a plurality of operations in the operation sequence information.
- the model name 5453 stores the name of the model of the home electric appliance 2 to be operated.
- the installation location 5454 stores a name indicating the installation location of the home appliance 2.
- the operation number 5455 stores the operation number of the operation.
- the operation name 5456 stores the operation name of the operation.
- the setting content 5457 stores the setting content set in the operation.
- the description 5458 stores information (hereinafter referred to as description information) indicating the description of the operation sequence information. The same description information is stored in the description 5458 of each record constituting the same operation sequence information.
- the audio processing device 5B functionally includes an audio data receiving unit 500, an audio data analyzing unit 501B, an operation information generating unit 502A, an operation information transmitting unit 503, a request information generating unit 504, a request information transmitting unit 505, a result information receiving unit 506, an audio data generating unit 507, an audio data transmitting unit 508, an answer information receiving unit 509, a continuous operation detecting unit 511, and a similar sequence searching unit 512. And a sequence setting suggestion unit 513.
- the audio data analysis unit 501B performs the same processing as the audio data analysis unit 501 of the audio processing device 5 according to the first embodiment. That is, the voice data analysis unit 501B analyzes the voice data received by the voice data receiving unit 500, and generates voice content data indicating the utterance content of the user. When the generated audio content data indicates an operation on the household electrical appliance 2, the audio data analysis unit 501B supplies the audio content data to the operation information generation unit 502A. Further, the voice data analysis unit 501B refers to the customer information DB 542 to identify the uttering user (that is, the operator), and also supplies the name of the identified operator to the operation information generation unit 502A. It is assumed that data relating to the voice of each user at each customer home is registered in the customer information DB 542.
- the voice data analysis unit 501B supplies the voice content data to the sequence setting proposal unit 513.
- the operation information generation unit 502A generates operation information based on the audio content data supplied from the audio data analysis unit 501B, the device operation information DB 541, the customer information DB 542, the customer device information DB 543A, and the operation sequence DB 545. .
- the operation information generation unit 502A generates a record indicating the history of the operation, and registers the record in the operation history DB 544. At this time, the operation information generation unit 502A calculates a continuous index of the operation and stores the calculated continuous index in the continuous index 5448.
- the continuous index is calculated by Equation 2 below.
- [interval] is the elapsed time (minutes) from the previous operation.
- [Operator] is “1” when the current operator and the previous operator are the same, and “0” when they are different.
- [Device] is “1” when the home appliance 2 operated this time is the same as the home appliance 2 operated last time, and becomes “0” when it is different.
- [Model] is “1” when the model of the home appliance 2 operated this time is the same as the model of the home appliance 2 operated last time, and “0” when the model is different.
- [Installation location] is “1” when the installation location of the home appliance 2 operated this time is the same as the installation location of the home appliance 2 last operated, and is “0” when the installation location is different.
- the weight for each parameter is set in consideration of the influence of the parameter on the continuity of the operation.
- the weight corresponding to [interval] is set to "10", which is the largest
- the weight corresponding to [operator] and [model] is set to "0.5", which is the smallest.
- the continuous index can be derived by various methods, for example, by using a deep learning RNN (Recurrent Neural Network).
- RNN Recurrent Neural Network
- the continuous operation detection unit 511 checks the operation history DB 544 at a predetermined timing (for example, at regular intervals), and performs a plurality of operations (hereinafter, referred to as continuous operations) for which continuity is recognized. To detect. More specifically, when there is an operation whose continuous index is equal to or more than a predetermined reference value (for example, 10.00), the continuous operation detection unit 511 detects the operation immediately before the operation and the operation as the continuous operation. In addition, when the operation whose continuous index is equal to or more than the reference value is continuous, the continuous operation includes three or more operations.
- the continuous operation detection unit 511 stores information on the detected continuous operation in a continuous operation table (not shown).
- the continuous operation table is developed in the RAM 53 or the secondary storage device 54.
- FIG. 49 shows an example of the continuous operation table.
- the similar sequence search unit 512 searches for operation sequence information included in the continuous operation detected by the continuous operation detection unit 511. Specifically, the similar sequence search unit 512 sequentially picks up continuous operations from the continuous operation table, and searches the operation sequence DB 545 for operation sequence information included in the picked up continuous operations.
- the operation sequence information of the registration number “1” of the operation sequence DB 545 is added to each of the continuous operations having the numbers “1”, “4”, and “6”. Is included.
- the similar sequence search unit 512 registers the operation sequence information. The number is notified to sequence setting proposal section 513.
- the sequence setting proposal unit 513 is an example of a sequence setting proposal unit according to the present invention.
- the sequence setting proposal unit 513 generates proposal information for proposing a series of operation settings (hereinafter, referred to as sequence settings) indicated by the operation sequence information of the notified registration number.
- the proposal information includes the description information read from the description 5458 of the record corresponding to the registration number in the operation sequence DB 545.
- the sequence setting proposal unit 513 supplies the generated proposal information to the audio data generation unit 507.
- the audio data generation unit 507 generates audio data by a known audio synthesis process based on the proposal information supplied from the sequence setting proposal unit 513, and supplies the generated audio data to the audio data transmission unit 508.
- the audio data transmission unit 508 transmits the audio data supplied from the audio data generation unit 507 to the audio input / output device 3.
- the sequence setting proposing unit 513 sets the plurality of The sequence setting is performed for the home electric appliance 2. More specifically, the sequence setting suggestion unit 513 stores information (registration number, sequence number, etc.) on the operation sequence information in the corresponding operation sequence 5435 of the record corresponding to the home appliance 2 to be set in the customer device information DB 543A. ) Is stored.
- the operation information generation unit 502A When the operation information generation unit 502A generates operation information from audio content data based on a user's voice operation and supplies the operation information to the operation information transmission unit 503, the operation information generation unit 502A performs operation based on the current date and time, the customer device information DB 543A, and the operation sequence DB 545. Then, it is determined whether or not the home electric appliance 2 to be operated corresponds to the first home electric appliance 2 in the sequence operation to be executed. For example, the current date and time is 6:00 am on August 1, and the registration number “1” and the sequence number “1” are stored in the corresponding operation sequence 5435 of the record corresponding to the home appliance 2 in the customer device information DB 543A. If the operation has been performed, the operation information generation unit 502A determines that the home appliance 2 corresponds to the first home appliance 2 in the sequence operation to be executed.
- the operation information generation unit 502A When it is determined that the home appliance 2 corresponds to the first home appliance 2 in the sequence operation to be performed, the operation information generation unit 502A performs an operation on the home appliance 2 for which the same registration number “2” or a subsequent sequence number is set.
- the operation information is generated in ascending order of the sequence number, and is sequentially supplied to the operation information transmitting unit 503.
- the operation of the air conditioner in the living room is automatically started.
- the voice processing device 5 ⁇ / b> B sets a continuous operation on a plurality of home appliances 2 based on the user's daily operation tendency on the home appliances 2. That is, the sequence setting is proposed to the user. Then, when the user consents, the voice processing device 5B performs a sequence setting for realizing the continuous operation.
- the user can cause each of the plurality of home electric appliances 2 to continuously execute a desired operation by one voice operation. Therefore, user convenience is improved.
- the voice processing device 5B may propose a sequence setting for a plurality of home appliances 2 that are frequently operated continuously without referring to the operation sequence DB 545.
- the function of proposing and setting a sequence by the voice processing device 5B may be included in the device control device 6 or the voice operation support device 7.
- the voice processing support device 5B may include the same function as the voice operation support device 7 without including the voice operation support device 7, or the device control device 6 may include the voice operation support device 7 A function equivalent to the above may be included.
- the voice processing device 5B, the device control device 6, and the voice operation support device 7 may be realized by one device.
- the sequence setting proposal and setting function of the audio processing device 5B may be included in the audio processing device 5A of the second embodiment.
- the CPU 51 executes the sound processing program 540 stored in the secondary storage device 54 in the sound processing devices 5, 5A, and 5B, so that the sound processing devices 5, 5A, and 5B A functional unit (see FIGS. 17, 31, and 47) has been realized.
- all or some of the functional units of the audio processing devices 5, 5A, 5B may be realized by dedicated hardware.
- all or a part of the functional units (see FIG. 26) of the device control device 6 may be realized by dedicated hardware, or the functional units of the voice operation support devices 7 and 7A (FIG. 27).
- FIG. 37 may be realized by dedicated hardware.
- the dedicated hardware is, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.
- the audio processing program 540, the device control program 640, and the audio operation support program 740 include a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), a magneto-optical disc (Magneto-Optical Disc), and a USB (Universal). It is also possible to store and distribute in a computer-readable recording medium such as a serial bus (memory), a memory card, and an HDD.
- a computer-readable recording medium such as a serial bus (memory), a memory card, and an HDD.
- the voice processing program 540, the device control program 640, and the voice operation support program 740 are stored in a storage device of a server (not shown) on the Internet, and the server processes the voice processing devices 5, 5A, 5B and the device control device 6 Alternatively, the voice processing program 540, the device control program 640, and the voice operation support program 740 may be downloaded to the voice operation support devices 7, 7A.
- the present invention can be suitably applied to a system for controlling home electric appliances installed in a house by user's voice.
- 1, 1A, 1B voice operation system 2 household electric appliance, 3 voice input / output device, 4 home gateway, 5, 5A, 5B voice processing device, 6 device control device, 7, 7A voice operation support device, 30 microphone, 31A / D converter, 32 speaker, 33, 50, 60, 70 communication interface, 34, 51, 61, 71 CPU, 35, 52, 62, 72 ROM, 36, 53, 63, 73 RAM, 37, 54, 64, 74 secondary storage device, 38, 55, 65, 75 bus, 300 audio input unit, 301 transmission unit, 302 reception unit, 303 audio output unit, 500 audio data reception unit, 501, 501A, 501B audio data analysis unit, 502 , 502A ⁇ operation information generation unit, 503 ⁇ operation information transmission unit, 504, 504A ⁇ request information generation unit 505 request information transmitter, 506 result information receiver, 507 audio data generator, 508 audio data transmitter, 509 answer information receiver, 510 advice information receiver, 511 continuous operation detector, 512 similar sequence searcher, 513 sequence
Abstract
Description
音声入出力装置からユーザの音声に基づく第1音声データを受信する音声データ受信手段と、
前記受信された第1音声データが、前記ユーザが保有する家電機器における操作についての問合せを示す場合、家電機器の機種毎に、当該機種に対する複数のメーカ各々の1又は複数の固有の操作のそれぞれについて、説明を示す情報を対応付けた操作説明情報データベースを参照して、前記問合せに対する回答情報を生成する回答情報生成手段と、
前記回答情報に基づく音声を示す第2音声データを前記音声入出力装置に送信する音声データ送信手段と、を備える。
図1は、本発明の実施の形態1に係る音声操作支援システムを包含した音声操作システム1の全体構成を示す図である。音声操作システム1は、家屋Hに設置された家電機器2を音声操作により制御できるようにしたシステムであり、1又は複数の家電機器2と、音声入出力装置3と、ホームゲートウェイ4と、音声処理装置5と、機器制御装置6と、音声操作支援装置7とを備える。また、実施の形態1に係る音声操作支援システムは、ユーザによる音声操作を支援するためのシステムであり、音声入出力装置3と、ホームゲートウェイ4と、音声処理装置5と、音声操作支援装置7とにより構成される。
続いて、本発明の実施の形態2について説明する。なお、以下の説明において、実施の形態1と共通する構成要素等については、同一の符号を付し、その説明を省略する。
続いて、本発明の実施の形態3について説明する。なお、以下の説明において、実施の形態1と共通する構成要素等については、同一の符号を付し、その説明を省略する。
Claims (12)
- 音声入出力装置からユーザの音声に基づく第1音声データを受信する音声データ受信手段と、
前記受信された第1音声データが、前記ユーザが保有する家電機器における操作についての問合せを示す場合、家電機器の機種毎に、当該機種に対する複数のメーカ各々の1又は複数の固有の操作のそれぞれについて、説明を示す情報を対応付けた操作説明情報データベースを参照して、前記問合せに対する回答情報を生成する回答情報生成手段と、
前記回答情報に基づく音声を示す第2音声データを前記音声入出力装置に送信する音声データ送信手段と、を備える、音声操作支援システム。 - 前記ユーザが保有する家電機器の各固有の操作の使用状況に基づいて、当該固有の操作を前記ユーザに促すアドバイス情報を生成するアドバイス情報生成手段をさらに備え、
前記音声データ送信手段は、さらに、前記アドバイス情報に基づく音声を示す第3音声データを前記音声入出力装置に送信する、請求項1に記載の音声操作支援システム。 - 前記ユーザが保有する複数の家電機器の操作履歴に基づいて、異なる家電機器に対する連続的な操作の設定を前記ユーザに提案するシーケンス設定提案手段をさらに備える、請求項1又は2に記載の音声操作支援システム。
- 請求項1から3の何れか1項に記載の音声操作支援システムと、
前記受信された第1音声データが、前記ユーザが保有する家電機器に対する操作を示す場合、前記第1音声データで示される操作内容に基づいて、前記家電機器を制御するための制御指令を生成する制御指令生成手段と、
前記制御指令を前記家電機器に送信する制御指令送信手段と、を備える、音声操作システム。 - 音声入出力装置からユーザの音声に基づく第1音声データを受信する音声データ受信手段と、
前記受信された第1音声データが、前記ユーザが保有する家電機器における操作についての問合せを示す場合、前記問合せの回答を示す回答情報を音声操作支援装置に要求する回答要求手段と、
前記音声操作支援装置から得られた前記回答情報に基づく音声を示す第2音声データを前記音声入出力装置に送信する音声データ送信手段と、を備える、音声処理装置。 - 家電機器の機種毎に、当該機種に対する複数のメーカ各々の1又は複数の固有の操作のそれぞれについて、説明を示す情報を対応付けた操作説明情報データベースを記憶する記憶装置と、
前記操作説明情報データベースを参照して、ユーザが保有する家電機器における操作についての音声処理装置からの問合せに対する回答情報を生成し、前記音声処理装置に送信する回答手段と、を備える、音声操作支援装置。 - 前記ユーザが保有する家電機器の各固有の操作の使用状況に基づいて、当該固有の操作を前記ユーザに促すアドバイス情報を生成し、前記音声処理装置に送信するアドバイス情報生成手段をさらに備える、請求項6に記載の音声操作支援装置。
- 音声データ受信手段が、音声入出力装置からユーザの音声に基づく第1音声データを受信し、
回答情報生成手段が、前記受信された第1音声データが、前記ユーザが保有する家電機器における操作についての問合せを示す場合、家電機器の機種毎に、当該機種に対する複数のメーカ各々の1又は複数の固有の操作のそれぞれについて、説明を示す情報を対応付けた操作説明情報データベースを参照して、前記問合せに対する回答情報を生成し、
音声データ送信手段が、前記回答情報に基づく音声を示す第2音声データを前記音声入出力装置に送信する、音声操作支援方法。 - アドバイス情報生成手段が、前記ユーザが保有する家電機器の各固有の操作の使用状況に基づいて、当該固有の操作を前記ユーザに促すアドバイス情報を生成し、
前記音声データ送信手段は、さらに、前記アドバイス情報に基づく音声を示す第3音声データを前記音声入出力装置に送信する、請求項8に記載の音声操作支援方法。 - シーケンス設定提案手段が、前記ユーザが保有する複数の家電機器の操作履歴に基づいて、異なる家電機器に対する連続的な操作の設定を前記ユーザに提案する、請求項8又9に記載の音声操作支援方法。
- 家電機器の機種毎に、当該機種に対する複数のメーカ各々の1又は複数の固有の操作のそれぞれについて、説明を示す情報を対応付けた操作説明情報データベースを記憶する記憶装置を備えるコンピュータを、
前記操作説明情報データベースを参照して、ユーザが保有する家電機器における操作についての音声処理装置からの問合せに対する回答情報を生成し、前記音声処理装置に送信する回答手段、として機能させる、プログラム。 - 前記ユーザが保有する家電機器の各固有の操作の使用状況に基づいて、当該固有の操作を前記ユーザに促すアドバイス情報を生成し、前記音声処理装置に送信するアドバイス情報生成手段、としてさらに機能させる、請求項11に記載のプログラム。
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020551061A JP6955115B2 (ja) | 2018-10-05 | 2018-10-05 | 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム |
DE112018008054.4T DE112018008054T5 (de) | 2018-10-05 | 2018-10-05 | Unterstützungssystem für den sprachgesteuerten betrieb, sprachbediensystem, sprachverarbeitungsvorrichtung, unterstützungsvorrichtung für den sprachgesteuerten betrieb, unterstützungsverfahren für den sprachgesteuerten betrieb und programm |
US17/270,988 US20210201910A1 (en) | 2018-10-05 | 2018-10-05 | VOICE OPERATION ASSISTANCE SYSTEM, VOICE PROCESSING DEVICE, AND VOICE OPERATION ASSISTANCE DEVICE (as amended) |
GB2104430.0A GB2591918A (en) | 2018-10-05 | 2018-10-05 | Voice operation assistance system, voice operation system, voice processing device, voice operation assistance device |
PCT/JP2018/037424 WO2020070888A1 (ja) | 2018-10-05 | 2018-10-05 | 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム |
CN201880098292.3A CN112823341A (zh) | 2018-10-05 | 2018-10-05 | 语音操作支援系统、语音操作系统、语音处理装置、语音操作支援装置、语音操作支援方法以及程序 |
JP2021160585A JP7284793B2 (ja) | 2018-10-05 | 2021-09-30 | 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2018/037424 WO2020070888A1 (ja) | 2018-10-05 | 2018-10-05 | 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020070888A1 true WO2020070888A1 (ja) | 2020-04-09 |
Family
ID=70055384
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2018/037424 WO2020070888A1 (ja) | 2018-10-05 | 2018-10-05 | 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム |
Country Status (6)
Country | Link |
---|---|
US (1) | US20210201910A1 (ja) |
JP (1) | JP6955115B2 (ja) |
CN (1) | CN112823341A (ja) |
DE (1) | DE112018008054T5 (ja) |
GB (1) | GB2591918A (ja) |
WO (1) | WO2020070888A1 (ja) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023286458A1 (ja) * | 2021-07-12 | 2023-01-19 | Tesnology株式会社 | 電子機器制御装置、通信端末および電子機器 |
JP7357819B1 (ja) | 2022-06-23 | 2023-10-06 | 三菱電機株式会社 | 操作機器管理システム、操作機器管理装置、操作機器管理方法およびプログラム |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10783883B2 (en) * | 2016-11-03 | 2020-09-22 | Google Llc | Focus session at a voice interface device |
JP7262004B2 (ja) * | 2020-10-01 | 2023-04-21 | パナソニックIpマネジメント株式会社 | 情報処理方法、情報処理装置、および、プログラム |
CN113593565B (zh) * | 2021-09-29 | 2021-12-17 | 深圳大生活家科技有限公司 | 一种智能家庭设备管控方法和系统 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002304285A (ja) * | 2001-04-06 | 2002-10-18 | Nippon Hoso Kyokai <Nhk> | 音声を用いた操作支援システム |
JP2017107078A (ja) * | 2015-12-10 | 2017-06-15 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | 音声対話方法、音声対話装置及び音声対話プログラム |
JP2018120202A (ja) * | 2016-11-02 | 2018-08-02 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | 情報処理方法及びプログラム |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2011149558A2 (en) * | 2010-05-28 | 2011-12-01 | Abelow Daniel H | Reality alternate |
JP5374574B2 (ja) * | 2011-12-21 | 2013-12-25 | パナソニック株式会社 | コンサルティング家電システム、サーバ装置、ホームサーバ装置および家電機器に対するコンサルティング情報の提供方法 |
JP6078964B2 (ja) * | 2012-03-26 | 2017-02-15 | 富士通株式会社 | 音声対話システム及びプログラム |
WO2014171144A1 (ja) | 2013-04-19 | 2014-10-23 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | 家電機器の制御方法、家電機器制御システム、及びゲートウェイ |
JP6054283B2 (ja) * | 2013-11-27 | 2016-12-27 | シャープ株式会社 | 音声認識端末、サーバ、サーバの制御方法、音声認識システム、音声認識端末の制御プログラム、サーバの制御プログラムおよび音声認識端末の制御方法 |
JP5706982B2 (ja) * | 2014-02-19 | 2015-04-22 | 株式会社日立システムズ | 音声認識制御システムおよび音声認識制御方法 |
JP2018055633A (ja) * | 2016-09-30 | 2018-04-05 | シャープ株式会社 | 会話システム、端末装置、および、プログラム |
CN108010522A (zh) * | 2016-11-02 | 2018-05-08 | 松下电器(美国)知识产权公司 | 信息处理方法以及记录介质 |
CN108010523B (zh) * | 2016-11-02 | 2023-05-09 | 松下电器(美国)知识产权公司 | 信息处理方法以及记录介质 |
US11099540B2 (en) * | 2017-09-15 | 2021-08-24 | Kohler Co. | User identity in household appliances |
US10887125B2 (en) * | 2017-09-15 | 2021-01-05 | Kohler Co. | Bathroom speaker |
US11314214B2 (en) * | 2017-09-15 | 2022-04-26 | Kohler Co. | Geographic analysis of water conditions |
KR102498007B1 (ko) * | 2018-01-08 | 2023-02-08 | 엘지전자 주식회사 | 음성인식을 이용한 세탁물 처리기기 제어시스템 및 동작방법 |
US11252237B2 (en) * | 2018-03-02 | 2022-02-15 | Panasonic Intellectual Property Management Co., Ltd. | Device management system and device management method |
JP6918215B2 (ja) * | 2018-04-17 | 2021-08-11 | 三菱電機株式会社 | 機器制御システム及び機器制御方法 |
JPWO2020195821A1 (ja) * | 2019-03-26 | 2020-10-01 | ||
US11893795B2 (en) * | 2019-12-09 | 2024-02-06 | Google Llc | Interacting with visitors of a connected home environment |
US11128636B1 (en) * | 2020-05-13 | 2021-09-21 | Science House LLC | Systems, methods, and apparatus for enhanced headsets |
-
2018
- 2018-10-05 GB GB2104430.0A patent/GB2591918A/en not_active Withdrawn
- 2018-10-05 JP JP2020551061A patent/JP6955115B2/ja active Active
- 2018-10-05 CN CN201880098292.3A patent/CN112823341A/zh active Pending
- 2018-10-05 DE DE112018008054.4T patent/DE112018008054T5/de active Pending
- 2018-10-05 WO PCT/JP2018/037424 patent/WO2020070888A1/ja active Application Filing
- 2018-10-05 US US17/270,988 patent/US20210201910A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002304285A (ja) * | 2001-04-06 | 2002-10-18 | Nippon Hoso Kyokai <Nhk> | 音声を用いた操作支援システム |
JP2017107078A (ja) * | 2015-12-10 | 2017-06-15 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | 音声対話方法、音声対話装置及び音声対話プログラム |
JP2018120202A (ja) * | 2016-11-02 | 2018-08-02 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | 情報処理方法及びプログラム |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023286458A1 (ja) * | 2021-07-12 | 2023-01-19 | Tesnology株式会社 | 電子機器制御装置、通信端末および電子機器 |
JP7357819B1 (ja) | 2022-06-23 | 2023-10-06 | 三菱電機株式会社 | 操作機器管理システム、操作機器管理装置、操作機器管理方法およびプログラム |
WO2023248441A1 (ja) * | 2022-06-23 | 2023-12-28 | 三菱電機株式会社 | 操作機器管理システム、操作機器管理装置、操作機器管理方法およびプログラム |
Also Published As
Publication number | Publication date |
---|---|
CN112823341A (zh) | 2021-05-18 |
JP6955115B2 (ja) | 2021-10-27 |
US20210201910A1 (en) | 2021-07-01 |
GB202104430D0 (en) | 2021-05-12 |
GB2591918A (en) | 2021-08-11 |
DE112018008054T5 (de) | 2021-09-16 |
JPWO2020070888A1 (ja) | 2021-02-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020070888A1 (ja) | 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム | |
US10152976B2 (en) | Device control method, display control method, and purchase settlement method | |
US10468024B2 (en) | Information processing method and non-temporary storage medium for system to control at least one device through dialog with user | |
US20130201018A1 (en) | Appliance Monitoring Systems and Methods | |
CN111758128A (zh) | 方法、控制装置以及程序 | |
CN104509079B (zh) | 基于位置标记的语音识别模型 | |
US20040002779A1 (en) | Home electrical appliance control device, control method, control program and home electrical appliance | |
US20200126549A1 (en) | Device control system | |
WO2018043113A1 (ja) | 情報処理装置、情報処理方法、プログラム | |
JP6918215B2 (ja) | 機器制御システム及び機器制御方法 | |
US20150222476A1 (en) | Server device, terminal device, maintenance service information transmission method and program | |
CN112270926B (zh) | 一种环境调节设备的售后服务方法及售后服务设备 | |
WO2016147298A1 (ja) | 推薦装置、推薦決定方法、およびコンピュータプログラム | |
CN109144971A (zh) | 设备绑定方法及匹配系统 | |
WO2019065733A1 (ja) | 音声命令システム及び音声命令方法 | |
JP6669942B2 (ja) | 連携システム、連携サーバ、および機器制御サーバ | |
JP7284793B2 (ja) | 音声操作支援システム、音声操作システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム | |
US20140173049A1 (en) | Binding an appliance to a network at point of purchase | |
US20210090431A1 (en) | Device control signal registration apparatus, device control signal registration system, device control signal registration method, and device control signal registration program storing medium | |
JP2023101590A (ja) | 音声操作支援システム、音声処理装置、音声操作支援装置、音声操作支援方法及びプログラム | |
JP6861519B2 (ja) | 故障部品特定システム、故障部品特定方法、及びプログラム | |
WO2019069596A1 (ja) | 家電システム | |
JP6997554B2 (ja) | 家電装置システム | |
CN111183416B (zh) | 家电系统 | |
US20210375278A1 (en) | System and method for providing a health care related service |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18936077 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2020551061 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 202104430 Country of ref document: GB Kind code of ref document: A Free format text: PCT FILING DATE = 20181005 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18936077 Country of ref document: EP Kind code of ref document: A1 |