EP3822966A1 - Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control - Google Patents
Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control Download PDFInfo
- Publication number
- EP3822966A1 EP3822966A1 EP20206707.0A EP20206707A EP3822966A1 EP 3822966 A1 EP3822966 A1 EP 3822966A1 EP 20206707 A EP20206707 A EP 20206707A EP 3822966 A1 EP3822966 A1 EP 3822966A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- request
- household appliance
- query
- processing means
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 94
- 238000012795 verification Methods 0.000 title claims abstract description 33
- 230000004044 response Effects 0.000 claims abstract description 93
- 230000009471 action Effects 0.000 claims abstract description 62
- 230000000007 visual effect Effects 0.000 claims abstract description 14
- 230000004913 activation Effects 0.000 claims description 27
- 230000005540 biological transmission Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 6
- 230000003213 activating effect Effects 0.000 claims description 2
- 238000005352 clarification Methods 0.000 claims description 2
- 230000005236 sound signal Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000001755 vocal effect Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F34/00—Details of control systems for washing machines, washer-dryers or laundry dryers
- D06F34/04—Signal transfer or data transmission arrangements
- D06F34/05—Signal transfer or data transmission arrangements for wireless communication between components, e.g. for remote monitoring or control
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2816—Controlling appliance services of a home automation network by calling their functionalities
- H04L12/282—Controlling appliance services of a home automation network by calling their functionalities based on user interaction within the home
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2101/00—User input for the control of domestic laundry washing machines, washer-dryers or laundry dryers
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2105/00—Systems or parameters controlled or affected by the control systems of washing machines, washer-dryers or laundry dryers
- D06F2105/58—Indications or alarms to the control system or to the user
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F2105/00—Systems or parameters controlled or affected by the control systems of washing machines, washer-dryers or laundry dryers
- D06F2105/58—Indications or alarms to the control system or to the user
- D06F2105/60—Audible signals
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F34/00—Details of control systems for washing machines, washer-dryers or laundry dryers
- D06F34/28—Arrangements for program selection, e.g. control panels therefor; Arrangements for indicating program parameters, e.g. the selected program or its progress
-
- D—TEXTILES; PAPER
- D06—TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
- D06F—LAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
- D06F34/00—Details of control systems for washing machines, washer-dryers or laundry dryers
- D06F34/28—Arrangements for program selection, e.g. control panels therefor; Arrangements for indicating program parameters, e.g. the selected program or its progress
- D06F34/32—Arrangements for program selection, e.g. control panels therefor; Arrangements for indicating program parameters, e.g. the selected program or its progress characterised by graphical features, e.g. touchscreens
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L2015/088—Word spotting
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/221—Announcement of recognition results
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
- G10L2015/228—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context
Definitions
- the present invention relates to the technical field of voice control for household appliances in general.
- the invention relates to a method and system for controlling and/or communicating with a household appliance by means of voice commands and with verification of the enabling of a remote control.
- voice communication is activated by means of a keyword or an activation word.
- the voice communication allows interacting with the household appliance, for example to provide commands, by means of voice inputs from the user, which are suitably recognized by various possible voice recognition technologies, which in themselves are known.
- the voice command may be imparted by a user also from a remote position with respect to the household appliance, there is a need to avoid commands or control actions from being imparted which are incompatible with the current state of the household appliance (which is not known to the remote user) or which could result in malfunctioning or even hazards.
- Remote control solutions are known to this end. Such a brief definition indicates (and will indicate in this description) a procedure of "enabling or disabling the possibility of performing a remote control", i.e., performing actions following remote voice commands, for example.
- the solutions with voice commands advantageously allow a significant flexibility in use, since not only do they allow the user to impart commands or set control actions requiring remote control, but also to ask questions or make requests for actions which, in themselves, do not require a verification of remote control enabling.
- Such an object is achieved by a method according to claim 1.
- a system for controlling and/or communicating with a household appliance by means of voice commands and with verification of remote control enabling which is capable of performing the aforesaid method.
- the object of the present invention is also a household appliance adapted to be controlled by means of voice commands and with verification of remote control enabling, according to the aforesaid method.
- a household appliance is defined in claim 15.
- the method firstly comprises the steps of receiving an acoustic voice signal SV, corresponding to a voice query and/or request expression uttered by a user, and generating a corresponding first electric signal SE representative of the acoustic voice signal SV, by voice receiving means 2 integrated in the household appliance 100.
- the method then provides generating a query and/or request electric signal SR based on the aforesaid received first electric signal SE, representative of the acoustic voice signal SV, by first processing means 3 integrated in the household appliance 100 operatively connected to the aforesaid voice receiving means 2, and transmitting the query and/or request electric signal SR to second processing means 4.
- the method then comprises the steps of processing the aforesaid query and/or request electric signal SR, by the second processing means 4, to identify a user query and/or request, and of recognizing, again by the second processing means 4, if the user query and/or request is a command and/or control action which requires remote control enabling.
- the method provides proceeding directly with the following steps of determining and providing the user with a response: determining a response to be provided to the user on the basis of the identified user query and/or request, by the second processing means 4; then generating a response electric signal C corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response electric signal C to the first processing means 3 of the household appliance; finally, providing the user with a visual and/or written and/or voice response, by visual display means 5 or sound reproduction means 7 comprised in the household appliance 100, on the basis of the aforesaid response electric signal C.
- the method provides performing a remote control verification and/or enabling procedure and then performing the aforesaid steps of determining and providing a response to the user only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
- the aforesaid remote control verification and/or enabling procedure (performed if there is a need for the remote control function to be enabled) firstly provides verifying whether a remote control enabling state is activated in the household appliance 100.
- the method provides determining the execution of the command and/or control action in the household appliance 100 (step which is started by the first processing means 3 of the household appliance).
- the method provides setting a standby state and asking the user for an enabling action before executing the command and/or control action.
- control status a verification on the control state (i.e., "control status" of the household appliance which extends well beyond mere verifications of the correctness of the received voice command (which are also provided).
- the step of verifying whether a remote control enabling state is activated in the household appliance 100 is performed by the first processing means 3 by means of local processing, through a connection to a local control unit 40 of the household appliance 100.
- the step of verifying whether a remote control enabling state is activated in the household appliance 100 is performed by the second processing means 4 by means of connection to storage means 6 which are in remote communication with the control unit 40 of the household appliance 100 and periodically receive and store the state S thereof.
- the generic reference "S" in Figure 2 indicates signals representative of the state of the household appliance, which comprise indications about whether the remote control is enabled or not ("remote state" ON or OFF).
- the method further comprises the steps of activating the remote control enabling state, if the user performs the enabling action, by means of the voice activation word or by means of manual command, and determining or starting the execution of the command and/or control action in the household appliance.
- the step of determining or starting the execution of the command and/or control action in the household appliance is performed, in various possible implementations, by the first processing means 3 of the household appliance, or by the control unit 40 of the household appliance, or in cooperation between the aforesaid first processing means 3 and control unit 40.
- the activation word is a generic activation word PA pronounced by the user, or a specific activation word aiming to enable the remote control.
- the method before the step of receiving an acoustic voice signal SV, the method comprises the steps of receiving an activation word PA pronounced by the user and generating a respective activation word electric signal SA, by the voice receiving means 2 integrated in the household appliance; and recognizing the activation word PA by the first processing means 3.
- the method provides carrying out the step of receiving an acoustic voice signal SV corresponding to a voice query and/or request utterance and the successive steps shown in the previously described embodiments.
- the step of generating a query and/or request electric signal SR comprises generating a respective audio stream query and/or request signal SR representative of the acoustic voice signal SV.
- the step of processing the electric query and/or request signal SR comprises recognizing whether the query and/or request is open or closed.
- a query and/or request is recognized as closed if it corresponds to a determined query and/or request, such that the second processing means 4 are capable of providing a predetermined univocal response relating to the query and/or request, on the basis of information and/or states of the household appliance which are accessible to the second processing means 4.
- a query and/or request is recognized as open in all cases in which the query and/or request is not recognized as closed.
- the step of identifying the user query and/or request comprises identifying the recognized closed query and/or request as a user query and/or request; and the step of determining a response to be provided to the user comprises determining, as a response, the aforesaid predetermined univocal response relating to the closed query and/or request.
- the step of identifying the user query and/or request comprises interpreting the open query and/or request as a partial query and/or request.
- the method comprises the further steps of determining and sending an interlocutory response to the user, comprising a request for further clarifications; and setting a standby state prepared to receive a further completion voice utterance to complete the request and/or query by the user.
- the step of recognizing whether the user query and/or request is a command and/or a control action requiring remote control enabling comprises: verifying whether the user query and/or request is closed; verifying whether the user query and/or request corresponds to a command and/or control action requiring remote control enabling, on the basis of information stored in the storage means 6 or otherwise accessible by the second processing means 4.
- the step of processing the query and/or request electric signal SR to identify a user query and/or request comprises interpreting the user query and/or request as a command and/or control action and/or question.
- the method further comprises the following steps, according to the possible alternatives.
- the method provides transmitting a command signal corresponding to the command to the first processing means 3 of the household appliance, by the second processing means 4.
- the method provides consulting, by the second processing means 4, storage means 6 in which state S of the household appliance 100 is periodically stored to define the action to be implemented and the execution mode.
- the method provides generating such a univocal response to the question, by the second processing means 4.
- the method provides generating an interlocutory response for a request for further information, by the second processing means 4.
- the step of recognizing the activation word, by first processing means 3, and the step of processing the query and/or request electric signal SR, by the second processing means 4, comprise executing at least one voice recognition algorithm and/or procedure.
- the voice recognition algorithm and/or procedure is selected between various possible types of algorithms and/or vocal recognition procedure in themselves known.
- procedures based on machine learning computational techniques are used.
- the step of generating a response electric signal C corresponding to the determined response comprises generating a first string of text characters corresponding to the response to be provided to the user.
- the aforesaid step further comprises generating a second string of text characters corresponding to the question of the user, as it is interpreted on the basis of at least one algorithm and/or procedure for transforming into text a voice command that can be executed by the second processing means 4.
- procedures based on machine learning computational techniques or other algorithms or procedures in themselves known are used to transform a voice command into text.
- the step of generating a response electric signal C corresponding to the determined response comprises, after the step of determining a response to the command and/or control action, the further steps of generating a response audio signal SAR, corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response audio signal SAR to the first processing means 3 of the household appliance; finally reproducing the response audio signal SAR to generate a voice response output, by voice output generation means 7 comprised in the household appliance.
- the step of providing the user with a response comprises providing the user with a written response, by means of display means 5 of the household appliance.
- the written response comprises displaying the aforesaid first string of text characters and/or second string of text characters as pop-up on a screen 5 comprised in the household appliance.
- the step of providing the user with a response comprises providing the user with visual information.
- the step of providing the user with a response comprises providing the user with a voice response, by means of voice output generation means 7 comprised in the household appliance.
- the vocal response comprises the sound reproduction of the aforesaid response audio signal SAR, which is generated by the second processing means 4 and transmitted to the first processing means 3.
- Figure 3 shows a flow chart of the actions performed in an embodiment of the method, showing certain exchanges of relevant information, certain actions and verifications which are performed, and the related reference to the logic level and/or physical entity (belonging to the household appliance, in local or at a remote cloud) in which the various actions, verifications and exchanges of information occur.
- the chart in Figure 3 shows the controls related to the enabling or non-enabling of the remote control and the subsequent actions.
- Figure 3 also mentions a "parking storage” (which may, for example, be implemented in the remote storage means 6 or may belong to the second processing means 4), which is configured to save a command or request by the user which can only be executed if the remote control is enabled, pending the verification and/or enabling of the activation state of the remote control.
- a "parking storage” (which may, for example, be implemented in the remote storage means 6 or may belong to the second processing means 4), which is configured to save a command or request by the user which can only be executed if the remote control is enabled, pending the verification and/or enabling of the activation state of the remote control.
- the stored command or request is retrieved by the second processing means 4 and suitably processed for the successive actions only when the remote control is enabled: this occurs if it has been verified that the state of the household appliance already provides "remote control ON" or, in the contrary case, after the user has possibly enabled the remote control by means of voice commands or other command.
- the household appliance 100 in this embodiment comprises both the control unit 40 (which is also connected to the remote storage means 6 to provide information on the state of the household appliance) and the first processing means 3.
- the second processing means 4 are implemented, in this example, in a distributed manner in a cloud and comprise, for example, a voice recognition software module.
- the cloud further comprises the storage means 6.
- Such a system comprises voice reception means 2 and first processing means 3 integrated in the household appliance 100, second processing means 4 (typically, remote) and a user interface 5 comprised in the household appliance 100.
- the aforesaid voice receiving means 2 integrated in the household appliance 100 are configured to receive an acoustic voice signal SV corresponding to a voice query and/or request utterance pronounced by the user and to transform it into a corresponding first electric signal SE representative of the acoustic voice signal SV.
- the first processing means 3 integrated in the household appliance 100 are operatively connected to the aforesaid voice receiving means 2 to receive said first electric signal SE and are configured to generate a query and/or request electric signal SR based on said first electric signal SE, representative of the acoustic voice signal SV.
- the second processing means 4 are operatively connected to the aforesaid first processing means 3 and are configured to carry out the following steps: receiving the query and/or request electric signal SR from the first processing means 3; processing such a query and/or request electric signal SR to identify a user query and/or request; and recognizing whether the user query and/or request is a command and/or control action which requires remote control enabling.
- the second processing means 4 are configured to proceed directly with the following steps of determining and providing the user with a response, i.e. to perform the following actions: determining a response to be provided to the user, on the basis of the identified user query and/or request, generating a response electric signal C corresponding to the determined response, and transmitting such a response electric signal C to the first processing means 3 of the household appliance.
- the second processing means 4 are configured to perform a remote control verification and/or enabling procedure and then perform the aforesaid steps of determining and providing a response to the user, which are executed only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
- the aforesaid remote control verification and/or enabling procedure comprises verifying whether a remote control enabling state is activated in the household appliance 100. If it is verified that a remote control enabling state is activated in the household appliance 100, the first processing means 3 of the household appliance are configured to determine the execution and/or the starting of the command and/or control action in the household appliance 100. If it is verified that a remote control enabling state is not activated in the household appliance 100, a standby state is set and the user is asked for an enabling action before executing the command and/or control action.
- the user interface 5 of the household appliance 100 is operatively connected to the first processing means 3 and is configured to provide a visual and/or written and/or voice response to the user on the basis of the aforesaid response electric signal C.
- the first processing means 3 and the second processing means 4 are also configured to carry out a method according to any one of the embodiments described above.
- the system 1 further comprises a control unit 40 of the household appliance and further comprises storage means 6 which typically are located remotely with respect to the household appliance.
- the control unit 40 of the household appliance is operatively connected to the first processing means 3 and configured to cooperate with the first processing means 3 in order to execute commands and/or control actions of the household appliance 100.
- control unit 40 is configured to control and/or monitor information relating to state S of the household appliance, comprising information about the enabling or disabling of the remote control state of the household appliance.
- the storage means 6 are operatively connected both to the second processing means 4 and to the control unit 40 of the household appliance and are configured to perform the following actions: receiving, from the aforesaid control unit 40, and storing the aforesaid information relating to state S of the household appliance, and providing the second processing means 4 with updated information on the enabling or disabling of the remote control state of the household appliance.
- the visual interface 5 comprises a screen 5 integrated in the household appliance and configured to display a string C of text characters corresponding to the response provided by the second processing means 4.
- the aforesaid screen 5 comprises a liquid crystal display or other type of display in itself known, either already provided in the household appliance or added to it to provide the visual interface function.
- the second processing means 4 are further configured to generate a response audio stream signal SAR corresponding to the determined response, and to transmit such a response audio stream signal SAR to the first processing means 2.
- system 1 further comprises voice output generation means 7 integrated in the household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output.
- the voice receiving means 2 comprise one or more microphones; the first processing means 3 comprise one or more electronic processing circuits and/or one or more electronic processors; and the system 1 further comprises transmission means 8 configured to transmit the aforesaid electric query or request signal SR.
- the aforesaid voice receiving means 2, first processing means 3 and transmission means 8 are integrated in a voice recognition module 30 incorporated in the household appliance 100.
- the second processing means 4 comprise one or more electronic processing circuits and/or one or more electronic processors, arranged remotely with respect to the household appliance 100.
- system 1 further comprises telecommunication means 9 configured to connect the transmission means 8 of the household appliance to the second remote processing means 4.
- the second processing means 4 are implemented in a cloud infrastructure 20 and the telecommunication means 9 comprise wireless communication means, a router and an infrastructure for telecommunicating to the cloud infrastructure 20.
- the voice receiving means 2 are further configured to receive a first activation word PA pronounced by a user and to generate a respective activation word electric signal SA.
- the first processing means 3 are further configured to recognize the activation word, and in case of the activation word I successfully recognized, to execute the aforesaid step of generating a query and/or request electric signal SR.
- the voice receiving means 2 are further configured to receive a second activation word PA pronounced by a user (provided to enable the remote control state) and to generate a respective second activation word electric signal.
- the first processing means 3 are further configured to recognize the second activation word, and in case the second activation word is successfully recognized, to enable the remote control of the household appliance.
- the aforesaid first and second activation words coincide and the enabling effect of the remote control occurs if the activation word is pronounced in response to a request by the system for the user to request remote control enabling.
- the first processing means 3 are further configured to generate a query and/or request audio stream signal SR representative of the acoustic voice signal SV, as the query and/or request electric signal SR.
- system 1 further comprises voice output generation means 7 integrated in the household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output.
- the voice output generation means 7 comprise a speaker or other sound reproduction means 7 adapted to provide the audio response.
- the household appliance 100 comprises a first portion 10 of a system 1 comprising at least the aforesaid voice receiving means 2, first processing means 3 and visual interface 5 (according to any one of the implementation options described above).
- the household appliance 100 is operatively connected to a second portion 20 of a system 1 comprising at least the aforesaid second processing means 4 (according to any one of the implementation options described above).
- the household appliance is a washing machine.
- Figure 4 shows the visual interface, i.e., screen 5, the voice recognition module 30 and the sound reproduction means 7.
- the above-described method and the system for controlling and/or communicating with a household appliance are capable of interpreting the user queries and/or requests and of discriminating the user queries and/or requests requiring remote control enabling from the user queries and/or requests requiring remote control which do not require remote control enabling.
- the required verification and/or remote control enabling procedures are performed for all and only those user queries and/or requests that actually require remote control enabling (thereby ensuring the adequate safety levels).
- the user queries and/or requests that do not require remote control enabling are processed and performed without verification procedures and/or remote control enabling procedures (which are not required in these cases), thus ensuring response and execution speed and effectiveness.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Human Computer Interaction (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Textile Engineering (AREA)
- Automation & Control Theory (AREA)
- Signal Processing (AREA)
- Selective Calling Equipment (AREA)
Abstract
Description
- The present invention relates to the technical field of voice control for household appliances in general.
- In particular, the invention relates to a method and system for controlling and/or communicating with a household appliance by means of voice commands and with verification of the enabling of a remote control.
- In the technical field of household appliances equipped with an advanced user interface, several solutions which allow controlling the household appliance and/or communicating therewith by means of voice commands are known.
- Typically, voice communication is activated by means of a keyword or an activation word.
- Once activated, the voice communication allows interacting with the household appliance, for example to provide commands, by means of voice inputs from the user, which are suitably recognized by various possible voice recognition technologies, which in themselves are known.
- In such a context, since the voice command may be imparted by a user also from a remote position with respect to the household appliance, there is a need to avoid commands or control actions from being imparted which are incompatible with the current state of the household appliance (which is not known to the remote user) or which could result in malfunctioning or even hazards.
- "Remote control" solutions are known to this end. Such a brief definition indicates (and will indicate in this description) a procedure of "enabling or disabling the possibility of performing a remote control", i.e., performing actions following remote voice commands, for example.
- On the other hand, the solutions with voice commands advantageously allow a significant flexibility in use, since not only do they allow the user to impart commands or set control actions requiring remote control, but also to ask questions or make requests for actions which, in themselves, do not require a verification of remote control enabling.
- In this context, the rigid and indiscriminate application of "remote control" procedures (which are anyway necessary and unavoidable, to a certain extent) may be disadvantageous for the user in terms of less effectiveness and speed and increased complexity.
- In this context, there emerges the need (not completely met by the solutions known to date) to devise methods and systems for controlling and/or communicating with a household appliance by means of voice commands which improve the interactions with the verification procedures of the remote control enabling in order to provide the user with effective, quick, simple, and simultaneously safe, voice command solutions.
- It is the object of the present invention to provide a method for controlling and/or communicating with a household appliance by means of voice commands and with verification of the remote control enabling, which allows the above-mentioned drawbacks, indicated with reference to the known art, to be at least partially obviated and the above-mentioned needs, which are particularly felt in the considered technical field, to be met. Such an object is achieved by a method according to claim 1.
- Further embodiments of such a method are defined by
claims 2 to 9. - It also in an object of the present invention to provide a system for controlling and/or communicating with a household appliance by means of voice commands and with verification of remote control enabling, which is capable of performing the aforesaid method. Such a system is defined in
claim 10. - Further embodiments of such a system are defined by claims 11 to 14.
- The object of the present invention is also a household appliance adapted to be controlled by means of voice commands and with verification of remote control enabling, according to the aforesaid method. Such a household appliance is defined in claim 15.
- Further features and advantages of the method and system according to the invention will be apparent from the following description which illustrates preferred embodiments, given by way of indicative, non-limiting examples, with reference to the accompanying drawing, in which:
-
Figures 1 and2 show, by means of respective simplified block diagrams, two respective embodiments of a system for controlling and/or communicating with a household appliance by means of voice commands, according to the present invention; -
Figure 3 is a flow chart which describes exchanges of information, verifications and actions which occur in an embodiment of the method according to the invention; -
Figure 4 shows an embodiment of a household appliance according to the invention. - With reference to
Figure 1 , a method for controlling and/or communicating with ahousehold appliance 100 by means of voice commands and with verification of the enabling of a remote control, is described. - The method firstly comprises the steps of receiving an acoustic voice signal SV, corresponding to a voice query and/or request expression uttered by a user, and generating a corresponding first electric signal SE representative of the acoustic voice signal SV, by
voice receiving means 2 integrated in thehousehold appliance 100. - The method then provides generating a query and/or request electric signal SR based on the aforesaid received first electric signal SE, representative of the acoustic voice signal SV, by first processing means 3 integrated in the
household appliance 100 operatively connected to the aforesaidvoice receiving means 2, and transmitting the query and/or request electric signal SR to second processing means 4. - The method then comprises the steps of processing the aforesaid query and/or request electric signal SR, by the second processing means 4, to identify a user query and/or request, and of recognizing, again by the second processing means 4, if the user query and/or request is a command and/or control action which requires remote control enabling.
- If the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, the method provides proceeding directly with the following steps of determining and providing the user with a response: determining a response to be provided to the user on the basis of the identified user query and/or request, by the second processing means 4; then generating a response electric signal C corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response electric signal C to the first processing means 3 of the household appliance; finally, providing the user with a visual and/or written and/or voice response, by visual display means 5 or sound reproduction means 7 comprised in the
household appliance 100, on the basis of the aforesaid response electric signal C. - If the user query and/or request is recognized as a command and/or control action which requires remote control enabling, the method provides performing a remote control verification and/or enabling procedure and then performing the aforesaid steps of determining and providing a response to the user only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
- The aforesaid remote control verification and/or enabling procedure (performed if there is a need for the remote control function to be enabled) firstly provides verifying whether a remote control enabling state is activated in the
household appliance 100. - If it is verified that a remote control enabling state is activated in the
household appliance 100, the method provides determining the execution of the command and/or control action in the household appliance 100 (step which is started by the first processing means 3 of the household appliance). - If it is verified that a remote control enabling state is not activated in the
household appliance 100, the method provides setting a standby state and asking the user for an enabling action before executing the command and/or control action. - It should be noted that the above-described method is specifically directed to an application in the context of household appliances and provides verifying whether a possibility of remote control of the household appliance is enabled, i.e., a verification on the control state (i.e., "control status") of the household appliance which extends well beyond mere verifications of the correctness of the received voice command (which are also provided).
- Further embodiments of the method are herein described with reference to
Figure 2 . - According to an embodiment of the method, the step of verifying whether a remote control enabling state is activated in the
household appliance 100 is performed by the first processing means 3 by means of local processing, through a connection to alocal control unit 40 of thehousehold appliance 100. - According to another embodiment of the method, the step of verifying whether a remote control enabling state is activated in the
household appliance 100 is performed by the second processing means 4 by means of connection to storage means 6 which are in remote communication with thecontrol unit 40 of thehousehold appliance 100 and periodically receive and store the state S thereof. - The generic reference "S", in
Figure 2 , indicates signals representative of the state of the household appliance, which comprise indications about whether the remote control is enabled or not ("remote state" ON or OFF). - According to an embodiment, if the user query and/or request is interpreted as a command and/or control action which requires remote control enabling, and it is verified that a remote control enabling state is not activated in the
household appliance 100, the method further comprises the steps of activating the remote control enabling state, if the user performs the enabling action, by means of the voice activation word or by means of manual command, and determining or starting the execution of the command and/or control action in the household appliance. - The step of determining or starting the execution of the command and/or control action in the household appliance is performed, in various possible implementations, by the first processing means 3 of the household appliance, or by the
control unit 40 of the household appliance, or in cooperation between the aforesaid first processing means 3 andcontrol unit 40. - According to various implementations, the activation word is a generic activation word PA pronounced by the user, or a specific activation word aiming to enable the remote control.
- According to an embodiment, before the step of receiving an acoustic voice signal SV, the method comprises the steps of receiving an activation word PA pronounced by the user and generating a respective activation word electric signal SA, by the
voice receiving means 2 integrated in the household appliance; and recognizing the activation word PA by the first processing means 3. - In the case the recognition of the activation word is successful, the method provides carrying out the step of receiving an acoustic voice signal SV corresponding to a voice query and/or request utterance and the successive steps shown in the previously described embodiments.
- According to an implementation option, the step of generating a query and/or request electric signal SR comprises generating a respective audio stream query and/or request signal SR representative of the acoustic voice signal SV.
- According to one embodiment of the method, the step of processing the electric query and/or request signal SR comprises recognizing whether the query and/or request is open or closed.
- A query and/or request is recognized as closed if it corresponds to a determined query and/or request, such that the second processing means 4 are capable of providing a predetermined univocal response relating to the query and/or request, on the basis of information and/or states of the household appliance which are accessible to the second processing means 4.
- A query and/or request is recognized as open in all cases in which the query and/or request is not recognized as closed.
- According to an implementation option of the method, if the query and/or request is closed, the step of identifying the user query and/or request comprises identifying the recognized closed query and/or request as a user query and/or request; and the step of determining a response to be provided to the user comprises determining, as a response, the aforesaid predetermined univocal response relating to the closed query and/or request.
- If the query and/or request is open, the step of identifying the user query and/or request comprises interpreting the open query and/or request as a partial query and/or request.
- In this case, the method comprises the further steps of determining and sending an interlocutory response to the user, comprising a request for further clarifications; and setting a standby state prepared to receive a further completion voice utterance to complete the request and/or query by the user.
- According to an implementation option of the method, the step of recognizing whether the user query and/or request is a command and/or a control action requiring remote control enabling, comprises: verifying whether the user query and/or request is closed; verifying whether the user query and/or request corresponds to a command and/or control action requiring remote control enabling, on the basis of information stored in the storage means 6 or otherwise accessible by the second processing means 4.
- According to an implementation option of the method, the step of processing the query and/or request electric signal SR to identify a user query and/or request comprises interpreting the user query and/or request as a command and/or control action and/or question.
- In this case, the method further comprises the following steps, according to the possible alternatives.
- If the user query and/or request is interpreted as a command, the method provides transmitting a command signal corresponding to the command to the first processing means 3 of the household appliance, by the second processing means 4.
- If the user query and/or request is interpreted as a control action, the method provides consulting, by the second processing means 4, storage means 6 in which state S of the
household appliance 100 is periodically stored to define the action to be implemented and the execution mode. - If the user query and/or request is interpreted as a pertinent question, such that the second processing means 4 are capable of providing a univocal response, the method provides generating such a univocal response to the question, by the second processing means 4.
- If the user query and/or request is interpreted as a non-pertinent question, such that the second processing means 4 are not capable of providing a univocal response, the method provides generating an interlocutory response for a request for further information, by the second processing means 4.
- According to an embodiment of the method, the step of recognizing the activation word, by first processing means 3, and the step of processing the query and/or request electric signal SR, by the second processing means 4, comprise executing at least one voice recognition algorithm and/or procedure.
- According to various possible implementation options, the voice recognition algorithm and/or procedure is selected between various possible types of algorithms and/or vocal recognition procedure in themselves known.
- According to an implementation option, procedures based on machine learning computational techniques are used.
- According to one embodiment of the method, the step of generating a response electric signal C corresponding to the determined response comprises generating a first string of text characters corresponding to the response to be provided to the user.
- According to a particular implementation option, the aforesaid step further comprises generating a second string of text characters corresponding to the question of the user, as it is interpreted on the basis of at least one algorithm and/or procedure for transforming into text a voice command that can be executed by the second processing means 4.
- According to an implementation option, procedures based on machine learning computational techniques or other algorithms or procedures in themselves known are used to transform a voice command into text.
- According to anther embodiment of the method, the step of generating a response electric signal C corresponding to the determined response comprises, after the step of determining a response to the command and/or control action, the further steps of generating a response audio signal SAR, corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response audio signal SAR to the first processing means 3 of the household appliance; finally reproducing the response audio signal SAR to generate a voice response output, by voice output generation means 7 comprised in the household appliance.
- With reference to the step of providing the user with a response, various embodiments are possible, consistently with the respective options for generating an above-mentioned electric response signal.
- According to one embodiment, the step of providing the user with a response comprises providing the user with a written response, by means of display means 5 of the household appliance.
- According to an implementation option, the written response comprises displaying the aforesaid first string of text characters and/or second string of text characters as pop-up on a
screen 5 comprised in the household appliance. - According to another embodiment, the step of providing the user with a response comprises providing the user with visual information.
- According to one embodiment, the step of providing the user with a response comprises providing the user with a voice response, by means of voice output generation means 7 comprised in the household appliance.
- According to an implementation option, the vocal response comprises the sound reproduction of the aforesaid response audio signal SAR, which is generated by the second processing means 4 and transmitted to the first processing means 3.
- With reference to
Figure 3 , certain further details are provided below, by mere way of example of an embodiment of the method of the invention. -
Figure 3 shows a flow chart of the actions performed in an embodiment of the method, showing certain exchanges of relevant information, certain actions and verifications which are performed, and the related reference to the logic level and/or physical entity (belonging to the household appliance, in local or at a remote cloud) in which the various actions, verifications and exchanges of information occur. - In particular, the chart in
Figure 3 shows the controls related to the enabling or non-enabling of the remote control and the subsequent actions. -
Figure 3 also mentions a "parking storage" (which may, for example, be implemented in the remote storage means 6 or may belong to the second processing means 4), which is configured to save a command or request by the user which can only be executed if the remote control is enabled, pending the verification and/or enabling of the activation state of the remote control. - In this case, the stored command or request is retrieved by the second processing means 4 and suitably processed for the successive actions only when the remote control is enabled: this occurs if it has been verified that the state of the household appliance already provides "remote control ON" or, in the contrary case, after the user has possibly enabled the remote control by means of voice commands or other command.
- With regards to the entities indicated on the left-hand part of the chart in
Figure 3 , it should be noted that thehousehold appliance 100 in this embodiment comprises both the control unit 40 (which is also connected to the remote storage means 6 to provide information on the state of the household appliance) and the first processing means 3. - The second processing means 4 are implemented, in this example, in a distributed manner in a cloud and comprise, for example, a voice recognition software module.
- The cloud further comprises the storage means 6.
- With reference to
Figure 1 , a system for controlling and/or interacting with ahousehold appliance 100 by means of voice commands and with verification of the enabling of a remote control is described below. - Such a system comprises voice reception means 2 and first processing means 3 integrated in the
household appliance 100, second processing means 4 (typically, remote) and auser interface 5 comprised in thehousehold appliance 100. - The aforesaid voice receiving means 2 integrated in the
household appliance 100 are configured to receive an acoustic voice signal SV corresponding to a voice query and/or request utterance pronounced by the user and to transform it into a corresponding first electric signal SE representative of the acoustic voice signal SV. - The first processing means 3 integrated in the
household appliance 100 are operatively connected to the aforesaid voice receiving means 2 to receive said first electric signal SE and are configured to generate a query and/or request electric signal SR based on said first electric signal SE, representative of the acoustic voice signal SV. - The second processing means 4 are operatively connected to the aforesaid first processing means 3 and are configured to carry out the following steps: receiving the query and/or request electric signal SR from the first processing means 3; processing such a query and/or request electric signal SR to identify a user query and/or request; and recognizing whether the user query and/or request is a command and/or control action which requires remote control enabling.
- If the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, the second processing means 4 are configured to proceed directly with the following steps of determining and providing the user with a response, i.e. to perform the following actions: determining a response to be provided to the user, on the basis of the identified user query and/or request, generating a response electric signal C corresponding to the determined response, and transmitting such a response electric signal C to the first processing means 3 of the household appliance.
- If the user query and/or request is recognized as a command and/or control action which requires remote control enabling, the second processing means 4 are configured to perform a remote control verification and/or enabling procedure and then perform the aforesaid steps of determining and providing a response to the user, which are executed only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
- The aforesaid remote control verification and/or enabling procedure comprises verifying whether a remote control enabling state is activated in the
household appliance 100. If it is verified that a remote control enabling state is activated in thehousehold appliance 100, the first processing means 3 of the household appliance are configured to determine the execution and/or the starting of the command and/or control action in thehousehold appliance 100. If it is verified that a remote control enabling state is not activated in thehousehold appliance 100, a standby state is set and the user is asked for an enabling action before executing the command and/or control action. - The
user interface 5 of thehousehold appliance 100 is operatively connected to the first processing means 3 and is configured to provide a visual and/or written and/or voice response to the user on the basis of the aforesaid response electric signal C. - With reference to
Figures 1 and2 , further embodiments of the system 1 are described. - According to various possible embodiments of the system 1, the first processing means 3 and the second processing means 4 are also configured to carry out a method according to any one of the embodiments described above.
- According to one embodiment, the system 1 further comprises a
control unit 40 of the household appliance and further comprises storage means 6 which typically are located remotely with respect to the household appliance. - The
control unit 40 of the household appliance is operatively connected to the first processing means 3 and configured to cooperate with the first processing means 3 in order to execute commands and/or control actions of thehousehold appliance 100. - Moreover, the
control unit 40 is configured to control and/or monitor information relating to state S of the household appliance, comprising information about the enabling or disabling of the remote control state of the household appliance. - The storage means 6 are operatively connected both to the second processing means 4 and to the
control unit 40 of the household appliance and are configured to perform the following actions: receiving, from theaforesaid control unit 40, and storing the aforesaid information relating to state S of the household appliance, and providing the second processing means 4 with updated information on the enabling or disabling of the remote control state of the household appliance. - According to an implementation option of the system, the
visual interface 5 comprises ascreen 5 integrated in the household appliance and configured to display a string C of text characters corresponding to the response provided by the second processing means 4. - According to an implementation example, the
aforesaid screen 5 comprises a liquid crystal display or other type of display in itself known, either already provided in the household appliance or added to it to provide the visual interface function. - According to another implementation option of the system, the second processing means 4 are further configured to generate a response audio stream signal SAR corresponding to the determined response, and to transmit such a response audio stream signal SAR to the first processing means 2.
- In this case, system 1 further comprises voice output generation means 7 integrated in the
household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output. - According to a particular implementation option of the system, the voice receiving means 2 comprise one or more microphones; the first processing means 3 comprise one or more electronic processing circuits and/or one or more electronic processors; and the system 1 further comprises transmission means 8 configured to transmit the aforesaid electric query or request signal SR.
- According to an embodiment, the aforesaid voice receiving means 2, first processing means 3 and transmission means 8 are integrated in a
voice recognition module 30 incorporated in thehousehold appliance 100. - According to a particular implementation option of the system, the second processing means 4 comprise one or more electronic processing circuits and/or one or more electronic processors, arranged remotely with respect to the
household appliance 100. In this case, system 1 further comprises telecommunication means 9 configured to connect the transmission means 8 of the household appliance to the second remote processing means 4. - According to an implementation example of the system, the second processing means 4 are implemented in a
cloud infrastructure 20 and the telecommunication means 9 comprise wireless communication means, a router and an infrastructure for telecommunicating to thecloud infrastructure 20. - According to an embodiment of the system, the voice receiving means 2 are further configured to receive a first activation word PA pronounced by a user and to generate a respective activation word electric signal SA. The first processing means 3 are further configured to recognize the activation word, and in case of the activation word I successfully recognized, to execute the aforesaid step of generating a query and/or request electric signal SR.
- According to another embodiment of the system, the voice receiving means 2 are further configured to receive a second activation word PA pronounced by a user (provided to enable the remote control state) and to generate a respective second activation word electric signal. The first processing means 3 are further configured to recognize the second activation word, and in case the second activation word is successfully recognized, to enable the remote control of the household appliance.
- According to an implementation option, the aforesaid first and second activation words coincide and the enabling effect of the remote control occurs if the activation word is pronounced in response to a request by the system for the user to request remote control enabling.
- According to an embodiment of the system, the first processing means 3 are further configured to generate a query and/or request audio stream signal SR representative of the acoustic voice signal SV, as the query and/or request electric signal SR.
- According to an embodiment, system 1 further comprises voice output generation means 7 integrated in the
household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output. - According to an implementation option, the voice output generation means 7 comprise a speaker or other sound reproduction means 7 adapted to provide the audio response.
- With reference to
Figures 1 ,2 and4 , ahousehold appliance 100 according to the invention is now described. - The
household appliance 100 comprises afirst portion 10 of a system 1 comprising at least the aforesaid voice receiving means 2, first processing means 3 and visual interface 5 (according to any one of the implementation options described above). - The
household appliance 100 is operatively connected to asecond portion 20 of a system 1 comprising at least the aforesaid second processing means 4 (according to any one of the implementation options described above). - In the example shown in
Figure 4 , the household appliance is a washing machine.Figure 4 shows the visual interface, i.e.,screen 5, thevoice recognition module 30 and the sound reproduction means 7. - It is worth noting that the object of the present invention is fully achieved by the method and system illustrated above by virtue of the functional and structural features thereof.
- Indeed, the above-described method and the system for controlling and/or communicating with a household appliance are capable of interpreting the user queries and/or requests and of discriminating the user queries and/or requests requiring remote control enabling from the user queries and/or requests requiring remote control which do not require remote control enabling.
- Once such a discrimination has been performed, the required verification and/or remote control enabling procedures are performed for all and only those user queries and/or requests that actually require remote control enabling (thereby ensuring the adequate safety levels).
- On the contrary, the user queries and/or requests that do not require remote control enabling are processed and performed without verification procedures and/or remote control enabling procedures (which are not required in these cases), thus ensuring response and execution speed and effectiveness.
- A person skilled in the art may make changes and adaptations to the embodiments of the method and system described above or can replace elements with others which are functionally equivalent to satisfy contingent needs without departing from the scope of protection of the appended claims. All the features described above as belonging to one possible embodiment may be implemented independently from the other embodiments described.
Claims (15)
- A method for controlling and/or communicating with a household appliance (100) by means of voice commands and with verification of the enabling of a remote control, comprising the steps of:- receiving an acoustic voice signal (SV), corresponding to a voice query and/or request expression uttered by a user, and generating a corresponding first electric signal (SE) representative of the acoustic voice signal (SV), by voice receiving means (2) integrated in the household appliance (100);- generating a query and/or request electric signal (SR), based on said received first electric signal (SE), representative of the acoustic voice signal (SV), by first processing means (3) integrated in the household appliance (100) operatively connected to said voice receiving means (2);- transmitting said query and/or request electric signal (SR) to second processing means (4);- processing said query and/or request electric signal (SR), by the second processing means (4), to identify a user query and/or request;- recognizing, by the second processing means (4), whether the user query and/or request is a command and/or control action which requires remote control enabling;- if the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, proceeding directly with the following steps:- determining a response to be provided to the user on the basis of the identified user query and/or request, by the second processing means (4);- generating an response electric signal (C) corresponding to the determined response, by the second processing means (4);- transmitting said response electric signal (C) to the first processing means (3) of the household appliance;- providing the user with a visual and/or written and/or voice response, by visual display means (5) or sound reproduction means (7) comprised in the household appliance (100), based on said response electric signal (C);- if the user query and/or request is recognized as a command and/or control action which requires remote control enabling, performing a remote control verification and/or enabling procedure and then performing said steps of determining and providing a response to the user;wherein said remote control verification and/or enabling procedure comprises:- verifying whether a remote control enabling state is activated in the household appliance (100);- if it is verified that a remote control enabling state is activated in the household appliance (100), determining the execution of the command and/or control action in the household appliance (100);- if it is verified that a remote control enabling state is not activated in the household appliance (100), setting a standby state and asking the user for an enabling action before executing the command and/or control action.
- A method according to claim 1, wherein the step of verifying whether a remote control enabling state is activated in the household appliance (100) is performed by the first processing means (3) by means of local processing, through a connection to a local control unit (40) of the household appliance (100), or is performed by the second processing means (4) by means of a connection to storage means (6) which are in remote communication with the control unit (40) of the household appliance (100) and periodically receive and store the state (S) thereof.
- A method according to any one of the preceding claims, wherein if the user query and/or request is interpreted as a command and/or control action which requires remote control enabling, and it is verified that a remote control enabling state is not activated in the household appliance (100), the method further comprises the step of:- if the user performs the enabling action by means of voice activation word or by manual command, activating the remote control enabling state and either determining or starting the execution of the command and/or control action in the household appliance, by the first processing means (3) of the household appliance.
- A method according to any one of the preceding claims, comprising, before said step of receiving an acoustic voice signal (SV), the steps of:- receiving an activation word (PA) uttered by the user and generating a respective activation word electric signal (SA), by said voice receiving means (2) integrated in the household appliance;- carrying out a recognition of the activation word (PA), by said first processing means (3);- in the case the recognition of the activation word is successful, proceeding to carry out said step of receiving an acoustic voice signal (SV) corresponding to a voice query and/or request utterance and the subsequent steps as defined in claim 1.
- A method according to any one of the preceding claims, wherein the step of generating a query and/or request electric signal (SR) comprises generating a respective audio stream query and/or request signal (SR) representative of the acoustic voice signal (SV),
and/or wherein the step of recognizing the activation word by first processing means (3) and the step of processing the query and/or request electric signal (SR), by the second processing means (4), comprise executing at least one voice recognition algorithm and/or procedure. - A method according to any one of the preceding claims, wherein said step of processing the query and/or request electric signal (SR) comprises recognizing whether the query and/or request is open or closed,
wherein a query and/or request is recognized as closed if it corresponds to a specific query and/or request, such that the second processing means (4) are capable of providing a predetermined univocal response relating to the query and/or request, based on information and/or states of the household appliance which are accessible to the second processing means (4),
and wherein a query and/or request is recognized as open in all cases in which said query and/or request is not recognized as closed. - A method according to claim 6, wherein:if the query and/or request is closed:- the step of identifying the user query and/or request comprises identifying the recognized closed query and/or request as a user query and/or request;- the step of determining a response to be provided to the user comprises determining, as a response, said predetermined univocal response relating to the closed query and/or request;if the query and/or request is open:and the method comprises the further steps of:- the step of identifying the user query and/or request comprises interpreting the open query and/or request as a partial query and/or request,- determining and sending an interlocutory response to the user, comprising a request for further clarifications;- setting a standby state prepared to receive a further completion voice utterance to complete the request and/or query by the user.
- A method according to any one of claims 6 or 7, wherein the step of recognizing whether the user query and/or request is a command and/or control action which requires remote control enabling, comprises:- verifying whether the user query and/or request is closed;- if the user query and/or request is closed, checking, on the basis of information stored in the storage means (6) or otherwise accessible by the second processing means (4), whether the user query and/or request corresponds to a command and/or control action requiring remote control enabling.
- A method according to any one of the preceding claims, wherein the step of processing the query and/or request electric signal (SR) to identify a user query and/or request, comprises:- interpreting the user query and/or request as a command and/or a control action and/or a question;
and wherein the method further comprises the steps of:if the user query and/or request is interpreted as a command, transmitting a command signal corresponding to the command to the first processing means (3) of the household appliance, by the second processing means (4);if the user query and/or request is interpreted as a control action, consulting, by the second processing means (4), storage means (6) in which the state of the household appliance (100) is periodically stored, to define the action to be implemented and the execution mode;if the user query and/or request is interpreted as a pertinent question, such that the second processing means (4) are capable of providing a univocal response, generating said univocal response to the question, by the second processing means (4);if the user query and/or request is interpreted as a non-pertinent question, such that the second processing means (4) are not capable of providing a univocal response, generating an interlocutory response for a request for further information, by the second processing means (4). - A system (1) for controlling and/or interacting with a household appliance (100) by means of voice commands and with verification of the enabling of a remote control, comprising:- voice receiving means (2) integrated in the household appliance (100), configured to receive an acoustic voice signal (SV), corresponding to a query and/or request voice expression pronounced by the user, and to transform it into a corresponding first electric signal (SE) representative of the acoustic voice signal (SV);- first processing means (3) integrated in the household appliance (100) operatively connected to said voice receiving means (2) to receive said first electric signal (SE), said first processing means (3) being configured to generate a query and/or request electric signal (SR) based on said first electric signal (SE), representative of the acoustic voice signal (SV);- second processing means (4) operatively connected to said first processing means (3), said second processing means (4) being configured to:receive said query and/or request electric signal (SR) from the first processing means (3);process said electric query and/or request signal (SR) to identify a user query and/or request;recognize whether the user query and/or request is a command and/or control action which requires remote control enabling;- if the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, proceeding directly with the steps of:
determining a response to be provided to the user, based on the identified user query and/or request, generating a response electric signal (C) corresponding to the determined response, and transmitting said response electric signal (C) to the first processing means (3) of the household appliance;- if the user query and/or request is recognized as a command and/or control action which requires remote control enabling, performing a remote control verification and/or enabling procedure and then performing said steps of determining and providing a response to the user;
wherein said remote control verification and/or enabling procedure comprises:- verifying whether a remote control enabling state is activated in the household appliance (100);- if it is verified that a remote control enabling state is activated in the household appliance (100), determining the executing and/or the starting the command and/or control action in the household appliance (100);- if it is verified that a remote control enabling state is not activated in the household appliance (100), setting a standby state and asking the user for an enabling action before executing the command and/or control action;- a user interface (5) comprised in the household appliance (100), operatively connected to said first processing means (3), said user interface (5) being configured to provide a visual and/or written and/or voice response to the user based on said response electric signal (C). - A system (1) according to claim 10, further comprising:- a control unit (40) of the household appliance operatively connected to the first processing means (3) and configured to cooperate with said first processing means (3) in order to execute commands and/or control actions of the household appliance (100), and further configured to control and/or monitor information relating to the state (S) of the household appliance, comprising information on the enabling or disabling of the remote control state of the household appliance;- storage means (6) operatively connected to the second processing means (4) and to said control unit (40) of the household appliance, said storage means (6) being configured to:- receive, from said control unit (40), and store said information relating to the state (S) of the household appliance;- provide the second processing means (4) with said updated information on the enabling or disabling of the remote control state of the household appliance.
- A system (1) according to any one of claims 10 or 11, wherein the visual interface (5) comprises a screen (5) integrated in the household appliance configured to display a string (C) of text characters corresponding to the response provided by the second processing means (4),
and/or wherein the second processing means (4) are further configured to generate a response audio stream signal (SAR) corresponding to the determined response, and to transmit said response audio stream signal (SAR) to the first processing means (2);
wherein the system (1) further comprises voice output generation means (7) integrated in the household appliance (100) and configured to reproduce the response audio stream signal (SAR) to generate a voice response output. - A system according to any one of claims 10 to 12, wherein:- the voice receiving means (2) comprise one or more microphones;- the first processing means (3) comprise one or more electronic processing circuits and/or one or more electronic processors;and wherein the system (1) further comprises transmission means (8) configured to transmit said query or request electric signal (SR),
and/or wherein said voice receiving means (2), said first processing means (3) and said transmission means (8) are integrated in a voice recognition module (30) incorporated in the household appliance (100). - A system (1) according to any one of claims 11 to 13, wherein the second processing means (4) comprise one or more electronic processing circuits and/or one or more electronic processors, arranged remotely with respect to the household appliance (100),
and wherein the system (1) further comprises telecommunication means (9) configured to connect the transmission means (8) of the household appliance to the second processing means (4),
and wherein the second processing means (4) are implemented in a cloud infrastructure (20) and the telecommunication means (9) comprise wireless communication means, a router and an infrastructure for telecommunicating to the cloud infrastructure (20). - A household appliance (100) comprising a first portion (10) of a system (1) according to any one of claims 10 to 14, comprising at least said voice receiving means (2), first processing means (3) and visual interface (5),
said household appliance (100) being operatively connected to a second portion (20) of a system (1) according to any one of claims 10 to 14, comprising at least said second processing means (4).
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
IT102019000020943A IT201900020943A1 (en) | 2019-11-12 | 2019-11-12 | Method and system for controlling and / or communicating with an appliance using voice commands with verification of the enabling of a remote control |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3822966A1 true EP3822966A1 (en) | 2021-05-19 |
EP3822966B1 EP3822966B1 (en) | 2023-09-13 |
Family
ID=69903840
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20206707.0A Active EP3822966B1 (en) | 2019-11-12 | 2020-11-10 | Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP3822966B1 (en) |
ES (1) | ES2965450T3 (en) |
IT (1) | IT201900020943A1 (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150302856A1 (en) * | 2014-04-17 | 2015-10-22 | Qualcomm Incorporated | Method and apparatus for performing function by speech input |
EP3413304A2 (en) * | 2017-05-19 | 2018-12-12 | LG Electronics Inc. | Method for operating home appliance and voice recognition server system |
US20190228780A1 (en) * | 2018-01-24 | 2019-07-25 | Comcast Cable Communications, Llc | Verification of user identity for voice enabled devices |
-
2019
- 2019-11-12 IT IT102019000020943A patent/IT201900020943A1/en unknown
-
2020
- 2020-11-10 EP EP20206707.0A patent/EP3822966B1/en active Active
- 2020-11-10 ES ES20206707T patent/ES2965450T3/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150302856A1 (en) * | 2014-04-17 | 2015-10-22 | Qualcomm Incorporated | Method and apparatus for performing function by speech input |
EP3413304A2 (en) * | 2017-05-19 | 2018-12-12 | LG Electronics Inc. | Method for operating home appliance and voice recognition server system |
US20190228780A1 (en) * | 2018-01-24 | 2019-07-25 | Comcast Cable Communications, Llc | Verification of user identity for voice enabled devices |
Also Published As
Publication number | Publication date |
---|---|
EP3822966B1 (en) | 2023-09-13 |
ES2965450T3 (en) | 2024-04-15 |
IT201900020943A1 (en) | 2021-05-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5208246B2 (en) | Voice input device having a period of interest | |
US20210241775A1 (en) | Hybrid speech interface device | |
JP6480568B2 (en) | Voice application architecture | |
CN109671426B (en) | Voice control method and device, storage medium and air conditioner | |
CN105793923A (en) | Local and remote speech processing | |
CN109144458B (en) | Electronic device for performing operation corresponding to voice input | |
CN109584883A (en) | The method and system of mobile terminal, long-range vocal print control vehicle device | |
JP2016018078A (en) | Voice recognition device and voice recognition system | |
CN106218557B (en) | Vehicle-mounted microphone with voice recognition control function | |
CN106601242A (en) | Executing method and device of operation event and terminal | |
CN112399222A (en) | Voice instruction learning method and device for smart television, smart television and medium | |
EP3822966B1 (en) | Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control | |
CN109979445A (en) | It is a kind of for understanding the system and method for standard language and dialect | |
EP3826003A1 (en) | Method and system for controlling and/or interacting with a household appliance by means of voice commands and with recognition of the language of the voice commands, and corresponding household appliance | |
KR20200057501A (en) | ELECTRONIC APPARATUS AND WiFi CONNECTING METHOD THEREOF | |
CN112203902B (en) | Device control apparatus and control method for controlling device | |
CN110648664A (en) | Household appliance control method and device with storage function | |
CN109102808A (en) | A kind of local speech recognition system and its method based on display interaction | |
EP3796310B1 (en) | Method and system for controlling and/or communicating with a domestic appliance by means of voice commands and text displays | |
CN109361820B (en) | Electronic equipment control method and system and electronic equipment | |
CN113284488A (en) | Control method and device for household appliance, voice box and storage medium | |
KR102417899B1 (en) | Apparatus and method for recognizing voice of vehicle | |
US20200342876A1 (en) | Method, device and computer-readable storage medium having instructions for processing a speech input, transportation vehicle, and user terminal with speech processing | |
KR100423495B1 (en) | Operation control system by speech recognition for portable device and a method using the same | |
EP3839719B1 (en) | Computing device and method of operating the same |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20211028 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: D06F 105/60 20200101ALN20230314BHEP Ipc: D06F 105/58 20200101ALN20230314BHEP Ipc: D06F 101/00 20200101ALN20230314BHEP Ipc: D06F 34/32 20200101ALN20230314BHEP Ipc: D06F 34/28 20200101ALN20230314BHEP Ipc: G10L 15/08 20060101ALN20230314BHEP Ipc: H04L 12/28 20060101ALI20230314BHEP Ipc: D06F 34/05 20200101ALI20230314BHEP Ipc: G06F 3/16 20060101ALI20230314BHEP Ipc: G10L 15/22 20060101AFI20230314BHEP |
|
INTG | Intention to grant announced |
Effective date: 20230404 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602020017553 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20230913 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231214 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20231201 Year of fee payment: 4 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231213 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231214 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: TR Payment date: 20231211 Year of fee payment: 4 Ref country code: IT Payment date: 20231130 Year of fee payment: 4 Ref country code: FR Payment date: 20231130 Year of fee payment: 4 Ref country code: DE Payment date: 20231121 Year of fee payment: 4 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1612126 Country of ref document: AT Kind code of ref document: T Effective date: 20230913 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20240116 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240113 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2965450 Country of ref document: ES Kind code of ref document: T3 Effective date: 20240415 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240113 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240115 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230913 |