EP3822966A1 - Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control - Google Patents

Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control Download PDF

Info

Publication number
EP3822966A1
EP3822966A1 EP20206707.0A EP20206707A EP3822966A1 EP 3822966 A1 EP3822966 A1 EP 3822966A1 EP 20206707 A EP20206707 A EP 20206707A EP 3822966 A1 EP3822966 A1 EP 3822966A1
Authority
EP
European Patent Office
Prior art keywords
request
household appliance
query
processing means
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP20206707.0A
Other languages
German (de)
French (fr)
Other versions
EP3822966B1 (en
Inventor
Leonardo Sergi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Candy SpA
Original Assignee
Candy SpA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Candy SpA filed Critical Candy SpA
Publication of EP3822966A1 publication Critical patent/EP3822966A1/en
Application granted granted Critical
Publication of EP3822966B1 publication Critical patent/EP3822966B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F34/00Details of control systems for washing machines, washer-dryers or laundry dryers
    • D06F34/04Signal transfer or data transmission arrangements
    • D06F34/05Signal transfer or data transmission arrangements for wireless communication between components, e.g. for remote monitoring or control
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • G06F3/167Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2816Controlling appliance services of a home automation network by calling their functionalities
    • H04L12/282Controlling appliance services of a home automation network by calling their functionalities based on user interaction within the home
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2101/00User input for the control of domestic laundry washing machines, washer-dryers or laundry dryers
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2105/00Systems or parameters controlled or affected by the control systems of washing machines, washer-dryers or laundry dryers
    • D06F2105/58Indications or alarms to the control system or to the user
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F2105/00Systems or parameters controlled or affected by the control systems of washing machines, washer-dryers or laundry dryers
    • D06F2105/58Indications or alarms to the control system or to the user
    • D06F2105/60Audible signals
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F34/00Details of control systems for washing machines, washer-dryers or laundry dryers
    • D06F34/28Arrangements for program selection, e.g. control panels therefor; Arrangements for indicating program parameters, e.g. the selected program or its progress
    • DTEXTILES; PAPER
    • D06TREATMENT OF TEXTILES OR THE LIKE; LAUNDERING; FLEXIBLE MATERIALS NOT OTHERWISE PROVIDED FOR
    • D06FLAUNDERING, DRYING, IRONING, PRESSING OR FOLDING TEXTILE ARTICLES
    • D06F34/00Details of control systems for washing machines, washer-dryers or laundry dryers
    • D06F34/28Arrangements for program selection, e.g. control panels therefor; Arrangements for indicating program parameters, e.g. the selected program or its progress
    • D06F34/32Arrangements for program selection, e.g. control panels therefor; Arrangements for indicating program parameters, e.g. the selected program or its progress characterised by graphical features, e.g. touchscreens
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/08Speech classification or search
    • G10L2015/088Word spotting
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/221Announcement of recognition results
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223Execution procedure of a spoken command
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/226Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
    • G10L2015/228Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context

Definitions

  • the present invention relates to the technical field of voice control for household appliances in general.
  • the invention relates to a method and system for controlling and/or communicating with a household appliance by means of voice commands and with verification of the enabling of a remote control.
  • voice communication is activated by means of a keyword or an activation word.
  • the voice communication allows interacting with the household appliance, for example to provide commands, by means of voice inputs from the user, which are suitably recognized by various possible voice recognition technologies, which in themselves are known.
  • the voice command may be imparted by a user also from a remote position with respect to the household appliance, there is a need to avoid commands or control actions from being imparted which are incompatible with the current state of the household appliance (which is not known to the remote user) or which could result in malfunctioning or even hazards.
  • Remote control solutions are known to this end. Such a brief definition indicates (and will indicate in this description) a procedure of "enabling or disabling the possibility of performing a remote control", i.e., performing actions following remote voice commands, for example.
  • the solutions with voice commands advantageously allow a significant flexibility in use, since not only do they allow the user to impart commands or set control actions requiring remote control, but also to ask questions or make requests for actions which, in themselves, do not require a verification of remote control enabling.
  • Such an object is achieved by a method according to claim 1.
  • a system for controlling and/or communicating with a household appliance by means of voice commands and with verification of remote control enabling which is capable of performing the aforesaid method.
  • the object of the present invention is also a household appliance adapted to be controlled by means of voice commands and with verification of remote control enabling, according to the aforesaid method.
  • a household appliance is defined in claim 15.
  • the method firstly comprises the steps of receiving an acoustic voice signal SV, corresponding to a voice query and/or request expression uttered by a user, and generating a corresponding first electric signal SE representative of the acoustic voice signal SV, by voice receiving means 2 integrated in the household appliance 100.
  • the method then provides generating a query and/or request electric signal SR based on the aforesaid received first electric signal SE, representative of the acoustic voice signal SV, by first processing means 3 integrated in the household appliance 100 operatively connected to the aforesaid voice receiving means 2, and transmitting the query and/or request electric signal SR to second processing means 4.
  • the method then comprises the steps of processing the aforesaid query and/or request electric signal SR, by the second processing means 4, to identify a user query and/or request, and of recognizing, again by the second processing means 4, if the user query and/or request is a command and/or control action which requires remote control enabling.
  • the method provides proceeding directly with the following steps of determining and providing the user with a response: determining a response to be provided to the user on the basis of the identified user query and/or request, by the second processing means 4; then generating a response electric signal C corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response electric signal C to the first processing means 3 of the household appliance; finally, providing the user with a visual and/or written and/or voice response, by visual display means 5 or sound reproduction means 7 comprised in the household appliance 100, on the basis of the aforesaid response electric signal C.
  • the method provides performing a remote control verification and/or enabling procedure and then performing the aforesaid steps of determining and providing a response to the user only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
  • the aforesaid remote control verification and/or enabling procedure (performed if there is a need for the remote control function to be enabled) firstly provides verifying whether a remote control enabling state is activated in the household appliance 100.
  • the method provides determining the execution of the command and/or control action in the household appliance 100 (step which is started by the first processing means 3 of the household appliance).
  • the method provides setting a standby state and asking the user for an enabling action before executing the command and/or control action.
  • control status a verification on the control state (i.e., "control status" of the household appliance which extends well beyond mere verifications of the correctness of the received voice command (which are also provided).
  • the step of verifying whether a remote control enabling state is activated in the household appliance 100 is performed by the first processing means 3 by means of local processing, through a connection to a local control unit 40 of the household appliance 100.
  • the step of verifying whether a remote control enabling state is activated in the household appliance 100 is performed by the second processing means 4 by means of connection to storage means 6 which are in remote communication with the control unit 40 of the household appliance 100 and periodically receive and store the state S thereof.
  • the generic reference "S" in Figure 2 indicates signals representative of the state of the household appliance, which comprise indications about whether the remote control is enabled or not ("remote state" ON or OFF).
  • the method further comprises the steps of activating the remote control enabling state, if the user performs the enabling action, by means of the voice activation word or by means of manual command, and determining or starting the execution of the command and/or control action in the household appliance.
  • the step of determining or starting the execution of the command and/or control action in the household appliance is performed, in various possible implementations, by the first processing means 3 of the household appliance, or by the control unit 40 of the household appliance, or in cooperation between the aforesaid first processing means 3 and control unit 40.
  • the activation word is a generic activation word PA pronounced by the user, or a specific activation word aiming to enable the remote control.
  • the method before the step of receiving an acoustic voice signal SV, the method comprises the steps of receiving an activation word PA pronounced by the user and generating a respective activation word electric signal SA, by the voice receiving means 2 integrated in the household appliance; and recognizing the activation word PA by the first processing means 3.
  • the method provides carrying out the step of receiving an acoustic voice signal SV corresponding to a voice query and/or request utterance and the successive steps shown in the previously described embodiments.
  • the step of generating a query and/or request electric signal SR comprises generating a respective audio stream query and/or request signal SR representative of the acoustic voice signal SV.
  • the step of processing the electric query and/or request signal SR comprises recognizing whether the query and/or request is open or closed.
  • a query and/or request is recognized as closed if it corresponds to a determined query and/or request, such that the second processing means 4 are capable of providing a predetermined univocal response relating to the query and/or request, on the basis of information and/or states of the household appliance which are accessible to the second processing means 4.
  • a query and/or request is recognized as open in all cases in which the query and/or request is not recognized as closed.
  • the step of identifying the user query and/or request comprises identifying the recognized closed query and/or request as a user query and/or request; and the step of determining a response to be provided to the user comprises determining, as a response, the aforesaid predetermined univocal response relating to the closed query and/or request.
  • the step of identifying the user query and/or request comprises interpreting the open query and/or request as a partial query and/or request.
  • the method comprises the further steps of determining and sending an interlocutory response to the user, comprising a request for further clarifications; and setting a standby state prepared to receive a further completion voice utterance to complete the request and/or query by the user.
  • the step of recognizing whether the user query and/or request is a command and/or a control action requiring remote control enabling comprises: verifying whether the user query and/or request is closed; verifying whether the user query and/or request corresponds to a command and/or control action requiring remote control enabling, on the basis of information stored in the storage means 6 or otherwise accessible by the second processing means 4.
  • the step of processing the query and/or request electric signal SR to identify a user query and/or request comprises interpreting the user query and/or request as a command and/or control action and/or question.
  • the method further comprises the following steps, according to the possible alternatives.
  • the method provides transmitting a command signal corresponding to the command to the first processing means 3 of the household appliance, by the second processing means 4.
  • the method provides consulting, by the second processing means 4, storage means 6 in which state S of the household appliance 100 is periodically stored to define the action to be implemented and the execution mode.
  • the method provides generating such a univocal response to the question, by the second processing means 4.
  • the method provides generating an interlocutory response for a request for further information, by the second processing means 4.
  • the step of recognizing the activation word, by first processing means 3, and the step of processing the query and/or request electric signal SR, by the second processing means 4, comprise executing at least one voice recognition algorithm and/or procedure.
  • the voice recognition algorithm and/or procedure is selected between various possible types of algorithms and/or vocal recognition procedure in themselves known.
  • procedures based on machine learning computational techniques are used.
  • the step of generating a response electric signal C corresponding to the determined response comprises generating a first string of text characters corresponding to the response to be provided to the user.
  • the aforesaid step further comprises generating a second string of text characters corresponding to the question of the user, as it is interpreted on the basis of at least one algorithm and/or procedure for transforming into text a voice command that can be executed by the second processing means 4.
  • procedures based on machine learning computational techniques or other algorithms or procedures in themselves known are used to transform a voice command into text.
  • the step of generating a response electric signal C corresponding to the determined response comprises, after the step of determining a response to the command and/or control action, the further steps of generating a response audio signal SAR, corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response audio signal SAR to the first processing means 3 of the household appliance; finally reproducing the response audio signal SAR to generate a voice response output, by voice output generation means 7 comprised in the household appliance.
  • the step of providing the user with a response comprises providing the user with a written response, by means of display means 5 of the household appliance.
  • the written response comprises displaying the aforesaid first string of text characters and/or second string of text characters as pop-up on a screen 5 comprised in the household appliance.
  • the step of providing the user with a response comprises providing the user with visual information.
  • the step of providing the user with a response comprises providing the user with a voice response, by means of voice output generation means 7 comprised in the household appliance.
  • the vocal response comprises the sound reproduction of the aforesaid response audio signal SAR, which is generated by the second processing means 4 and transmitted to the first processing means 3.
  • Figure 3 shows a flow chart of the actions performed in an embodiment of the method, showing certain exchanges of relevant information, certain actions and verifications which are performed, and the related reference to the logic level and/or physical entity (belonging to the household appliance, in local or at a remote cloud) in which the various actions, verifications and exchanges of information occur.
  • the chart in Figure 3 shows the controls related to the enabling or non-enabling of the remote control and the subsequent actions.
  • Figure 3 also mentions a "parking storage” (which may, for example, be implemented in the remote storage means 6 or may belong to the second processing means 4), which is configured to save a command or request by the user which can only be executed if the remote control is enabled, pending the verification and/or enabling of the activation state of the remote control.
  • a "parking storage” (which may, for example, be implemented in the remote storage means 6 or may belong to the second processing means 4), which is configured to save a command or request by the user which can only be executed if the remote control is enabled, pending the verification and/or enabling of the activation state of the remote control.
  • the stored command or request is retrieved by the second processing means 4 and suitably processed for the successive actions only when the remote control is enabled: this occurs if it has been verified that the state of the household appliance already provides "remote control ON" or, in the contrary case, after the user has possibly enabled the remote control by means of voice commands or other command.
  • the household appliance 100 in this embodiment comprises both the control unit 40 (which is also connected to the remote storage means 6 to provide information on the state of the household appliance) and the first processing means 3.
  • the second processing means 4 are implemented, in this example, in a distributed manner in a cloud and comprise, for example, a voice recognition software module.
  • the cloud further comprises the storage means 6.
  • Such a system comprises voice reception means 2 and first processing means 3 integrated in the household appliance 100, second processing means 4 (typically, remote) and a user interface 5 comprised in the household appliance 100.
  • the aforesaid voice receiving means 2 integrated in the household appliance 100 are configured to receive an acoustic voice signal SV corresponding to a voice query and/or request utterance pronounced by the user and to transform it into a corresponding first electric signal SE representative of the acoustic voice signal SV.
  • the first processing means 3 integrated in the household appliance 100 are operatively connected to the aforesaid voice receiving means 2 to receive said first electric signal SE and are configured to generate a query and/or request electric signal SR based on said first electric signal SE, representative of the acoustic voice signal SV.
  • the second processing means 4 are operatively connected to the aforesaid first processing means 3 and are configured to carry out the following steps: receiving the query and/or request electric signal SR from the first processing means 3; processing such a query and/or request electric signal SR to identify a user query and/or request; and recognizing whether the user query and/or request is a command and/or control action which requires remote control enabling.
  • the second processing means 4 are configured to proceed directly with the following steps of determining and providing the user with a response, i.e. to perform the following actions: determining a response to be provided to the user, on the basis of the identified user query and/or request, generating a response electric signal C corresponding to the determined response, and transmitting such a response electric signal C to the first processing means 3 of the household appliance.
  • the second processing means 4 are configured to perform a remote control verification and/or enabling procedure and then perform the aforesaid steps of determining and providing a response to the user, which are executed only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
  • the aforesaid remote control verification and/or enabling procedure comprises verifying whether a remote control enabling state is activated in the household appliance 100. If it is verified that a remote control enabling state is activated in the household appliance 100, the first processing means 3 of the household appliance are configured to determine the execution and/or the starting of the command and/or control action in the household appliance 100. If it is verified that a remote control enabling state is not activated in the household appliance 100, a standby state is set and the user is asked for an enabling action before executing the command and/or control action.
  • the user interface 5 of the household appliance 100 is operatively connected to the first processing means 3 and is configured to provide a visual and/or written and/or voice response to the user on the basis of the aforesaid response electric signal C.
  • the first processing means 3 and the second processing means 4 are also configured to carry out a method according to any one of the embodiments described above.
  • the system 1 further comprises a control unit 40 of the household appliance and further comprises storage means 6 which typically are located remotely with respect to the household appliance.
  • the control unit 40 of the household appliance is operatively connected to the first processing means 3 and configured to cooperate with the first processing means 3 in order to execute commands and/or control actions of the household appliance 100.
  • control unit 40 is configured to control and/or monitor information relating to state S of the household appliance, comprising information about the enabling or disabling of the remote control state of the household appliance.
  • the storage means 6 are operatively connected both to the second processing means 4 and to the control unit 40 of the household appliance and are configured to perform the following actions: receiving, from the aforesaid control unit 40, and storing the aforesaid information relating to state S of the household appliance, and providing the second processing means 4 with updated information on the enabling or disabling of the remote control state of the household appliance.
  • the visual interface 5 comprises a screen 5 integrated in the household appliance and configured to display a string C of text characters corresponding to the response provided by the second processing means 4.
  • the aforesaid screen 5 comprises a liquid crystal display or other type of display in itself known, either already provided in the household appliance or added to it to provide the visual interface function.
  • the second processing means 4 are further configured to generate a response audio stream signal SAR corresponding to the determined response, and to transmit such a response audio stream signal SAR to the first processing means 2.
  • system 1 further comprises voice output generation means 7 integrated in the household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output.
  • the voice receiving means 2 comprise one or more microphones; the first processing means 3 comprise one or more electronic processing circuits and/or one or more electronic processors; and the system 1 further comprises transmission means 8 configured to transmit the aforesaid electric query or request signal SR.
  • the aforesaid voice receiving means 2, first processing means 3 and transmission means 8 are integrated in a voice recognition module 30 incorporated in the household appliance 100.
  • the second processing means 4 comprise one or more electronic processing circuits and/or one or more electronic processors, arranged remotely with respect to the household appliance 100.
  • system 1 further comprises telecommunication means 9 configured to connect the transmission means 8 of the household appliance to the second remote processing means 4.
  • the second processing means 4 are implemented in a cloud infrastructure 20 and the telecommunication means 9 comprise wireless communication means, a router and an infrastructure for telecommunicating to the cloud infrastructure 20.
  • the voice receiving means 2 are further configured to receive a first activation word PA pronounced by a user and to generate a respective activation word electric signal SA.
  • the first processing means 3 are further configured to recognize the activation word, and in case of the activation word I successfully recognized, to execute the aforesaid step of generating a query and/or request electric signal SR.
  • the voice receiving means 2 are further configured to receive a second activation word PA pronounced by a user (provided to enable the remote control state) and to generate a respective second activation word electric signal.
  • the first processing means 3 are further configured to recognize the second activation word, and in case the second activation word is successfully recognized, to enable the remote control of the household appliance.
  • the aforesaid first and second activation words coincide and the enabling effect of the remote control occurs if the activation word is pronounced in response to a request by the system for the user to request remote control enabling.
  • the first processing means 3 are further configured to generate a query and/or request audio stream signal SR representative of the acoustic voice signal SV, as the query and/or request electric signal SR.
  • system 1 further comprises voice output generation means 7 integrated in the household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output.
  • the voice output generation means 7 comprise a speaker or other sound reproduction means 7 adapted to provide the audio response.
  • the household appliance 100 comprises a first portion 10 of a system 1 comprising at least the aforesaid voice receiving means 2, first processing means 3 and visual interface 5 (according to any one of the implementation options described above).
  • the household appliance 100 is operatively connected to a second portion 20 of a system 1 comprising at least the aforesaid second processing means 4 (according to any one of the implementation options described above).
  • the household appliance is a washing machine.
  • Figure 4 shows the visual interface, i.e., screen 5, the voice recognition module 30 and the sound reproduction means 7.
  • the above-described method and the system for controlling and/or communicating with a household appliance are capable of interpreting the user queries and/or requests and of discriminating the user queries and/or requests requiring remote control enabling from the user queries and/or requests requiring remote control which do not require remote control enabling.
  • the required verification and/or remote control enabling procedures are performed for all and only those user queries and/or requests that actually require remote control enabling (thereby ensuring the adequate safety levels).
  • the user queries and/or requests that do not require remote control enabling are processed and performed without verification procedures and/or remote control enabling procedures (which are not required in these cases), thus ensuring response and execution speed and effectiveness.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Textile Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Signal Processing (AREA)
  • Selective Calling Equipment (AREA)

Abstract

A method for controlling and/or communicating with a household appliance 100 by means of voice commands and with verification of remote control enabling is described.The method comprises the steps of receiving an acoustic voice signal SV, corresponding to a voice query and/or request expression pronounced by a user, and generating a corresponding first electric signal SE representative of the acoustic voice signal SV, by voice receiving means 2 integrated in the household appliance 100.The method then provides generating a query and/or request electric signal SR, on the basis of the received first electric signal SE, by first processing means 3 integrated in the household appliance 100 operatively connected to said voice reception means 2; and transmitting the query and/or request electric signal SR to second processing means 4.The method then comprises the steps of processing, performed by the second processing means 4, the query and/or request electric signal SR to identify a user query and/or request; and of recognizing if the user query and/or request is a command and/or control action which requires remote control enabling.If the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, the method proceeds directly with the steps of determining a response to be provided to the user on the basis of the identified user query and/or request and generating a response electric signal C corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response electric signal C to the first processing means 3 of the household appliance; finally, providing the user with a visual and/or written and/or voice response, by display means 5 or sound reproduction means 7 comprised in the household appliance 100, on the basis of the aforesaid response electric signal C.If the user query and/or request is recognized as a command and/or control action which requires remote control enabling, the method provides performing a remote control verification and/or enabling procedure and then performing the aforesaid steps of determining and providing a response to the user, depending on the outcome of the remote control verification procedure.The aforesaid remote control verification and/or enabling procedure firstly provides verifying whether a remote control enabling state is activated in the household appliance 100.If it is verified that a remote control enabling state is activated in the household appliance 100, the method provides executing the command and/or control action in the household appliance 100.If it is verified that a remote control enabling state is not activated in the household appliance 100, the method provides setting a standby state and asking the user for an enabling action before executing the command and/or control action.

Description

    TECHNOLOGICAL BACKGROUND OF THE INVENTION Field of application.
  • The present invention relates to the technical field of voice control for household appliances in general.
  • In particular, the invention relates to a method and system for controlling and/or communicating with a household appliance by means of voice commands and with verification of the enabling of a remote control.
  • Description of the prior art.
  • In the technical field of household appliances equipped with an advanced user interface, several solutions which allow controlling the household appliance and/or communicating therewith by means of voice commands are known.
  • Typically, voice communication is activated by means of a keyword or an activation word.
  • Once activated, the voice communication allows interacting with the household appliance, for example to provide commands, by means of voice inputs from the user, which are suitably recognized by various possible voice recognition technologies, which in themselves are known.
  • In such a context, since the voice command may be imparted by a user also from a remote position with respect to the household appliance, there is a need to avoid commands or control actions from being imparted which are incompatible with the current state of the household appliance (which is not known to the remote user) or which could result in malfunctioning or even hazards.
  • "Remote control" solutions are known to this end. Such a brief definition indicates (and will indicate in this description) a procedure of "enabling or disabling the possibility of performing a remote control", i.e., performing actions following remote voice commands, for example.
  • On the other hand, the solutions with voice commands advantageously allow a significant flexibility in use, since not only do they allow the user to impart commands or set control actions requiring remote control, but also to ask questions or make requests for actions which, in themselves, do not require a verification of remote control enabling.
  • In this context, the rigid and indiscriminate application of "remote control" procedures (which are anyway necessary and unavoidable, to a certain extent) may be disadvantageous for the user in terms of less effectiveness and speed and increased complexity.
  • In this context, there emerges the need (not completely met by the solutions known to date) to devise methods and systems for controlling and/or communicating with a household appliance by means of voice commands which improve the interactions with the verification procedures of the remote control enabling in order to provide the user with effective, quick, simple, and simultaneously safe, voice command solutions.
  • SUMMARY OF THE INVENTION
  • It is the object of the present invention to provide a method for controlling and/or communicating with a household appliance by means of voice commands and with verification of the remote control enabling, which allows the above-mentioned drawbacks, indicated with reference to the known art, to be at least partially obviated and the above-mentioned needs, which are particularly felt in the considered technical field, to be met. Such an object is achieved by a method according to claim 1.
  • Further embodiments of such a method are defined by claims 2 to 9.
  • It also in an object of the present invention to provide a system for controlling and/or communicating with a household appliance by means of voice commands and with verification of remote control enabling, which is capable of performing the aforesaid method. Such a system is defined in claim 10.
  • Further embodiments of such a system are defined by claims 11 to 14.
  • The object of the present invention is also a household appliance adapted to be controlled by means of voice commands and with verification of remote control enabling, according to the aforesaid method. Such a household appliance is defined in claim 15.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Further features and advantages of the method and system according to the invention will be apparent from the following description which illustrates preferred embodiments, given by way of indicative, non-limiting examples, with reference to the accompanying drawing, in which:
    • Figures 1 and 2 show, by means of respective simplified block diagrams, two respective embodiments of a system for controlling and/or communicating with a household appliance by means of voice commands, according to the present invention;
    • Figure 3 is a flow chart which describes exchanges of information, verifications and actions which occur in an embodiment of the method according to the invention;
    • Figure 4 shows an embodiment of a household appliance according to the invention.
    DETAILED DESCRIPTION
  • With reference to Figure 1, a method for controlling and/or communicating with a household appliance 100 by means of voice commands and with verification of the enabling of a remote control, is described.
  • The method firstly comprises the steps of receiving an acoustic voice signal SV, corresponding to a voice query and/or request expression uttered by a user, and generating a corresponding first electric signal SE representative of the acoustic voice signal SV, by voice receiving means 2 integrated in the household appliance 100.
  • The method then provides generating a query and/or request electric signal SR based on the aforesaid received first electric signal SE, representative of the acoustic voice signal SV, by first processing means 3 integrated in the household appliance 100 operatively connected to the aforesaid voice receiving means 2, and transmitting the query and/or request electric signal SR to second processing means 4.
  • The method then comprises the steps of processing the aforesaid query and/or request electric signal SR, by the second processing means 4, to identify a user query and/or request, and of recognizing, again by the second processing means 4, if the user query and/or request is a command and/or control action which requires remote control enabling.
  • If the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, the method provides proceeding directly with the following steps of determining and providing the user with a response: determining a response to be provided to the user on the basis of the identified user query and/or request, by the second processing means 4; then generating a response electric signal C corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response electric signal C to the first processing means 3 of the household appliance; finally, providing the user with a visual and/or written and/or voice response, by visual display means 5 or sound reproduction means 7 comprised in the household appliance 100, on the basis of the aforesaid response electric signal C.
  • If the user query and/or request is recognized as a command and/or control action which requires remote control enabling, the method provides performing a remote control verification and/or enabling procedure and then performing the aforesaid steps of determining and providing a response to the user only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
  • The aforesaid remote control verification and/or enabling procedure (performed if there is a need for the remote control function to be enabled) firstly provides verifying whether a remote control enabling state is activated in the household appliance 100.
  • If it is verified that a remote control enabling state is activated in the household appliance 100, the method provides determining the execution of the command and/or control action in the household appliance 100 (step which is started by the first processing means 3 of the household appliance).
  • If it is verified that a remote control enabling state is not activated in the household appliance 100, the method provides setting a standby state and asking the user for an enabling action before executing the command and/or control action.
  • It should be noted that the above-described method is specifically directed to an application in the context of household appliances and provides verifying whether a possibility of remote control of the household appliance is enabled, i.e., a verification on the control state (i.e., "control status") of the household appliance which extends well beyond mere verifications of the correctness of the received voice command (which are also provided).
  • Further embodiments of the method are herein described with reference to Figure 2.
  • According to an embodiment of the method, the step of verifying whether a remote control enabling state is activated in the household appliance 100 is performed by the first processing means 3 by means of local processing, through a connection to a local control unit 40 of the household appliance 100.
  • According to another embodiment of the method, the step of verifying whether a remote control enabling state is activated in the household appliance 100 is performed by the second processing means 4 by means of connection to storage means 6 which are in remote communication with the control unit 40 of the household appliance 100 and periodically receive and store the state S thereof.
  • The generic reference "S", in Figure 2, indicates signals representative of the state of the household appliance, which comprise indications about whether the remote control is enabled or not ("remote state" ON or OFF).
  • According to an embodiment, if the user query and/or request is interpreted as a command and/or control action which requires remote control enabling, and it is verified that a remote control enabling state is not activated in the household appliance 100, the method further comprises the steps of activating the remote control enabling state, if the user performs the enabling action, by means of the voice activation word or by means of manual command, and determining or starting the execution of the command and/or control action in the household appliance.
  • The step of determining or starting the execution of the command and/or control action in the household appliance is performed, in various possible implementations, by the first processing means 3 of the household appliance, or by the control unit 40 of the household appliance, or in cooperation between the aforesaid first processing means 3 and control unit 40.
  • According to various implementations, the activation word is a generic activation word PA pronounced by the user, or a specific activation word aiming to enable the remote control.
  • According to an embodiment, before the step of receiving an acoustic voice signal SV, the method comprises the steps of receiving an activation word PA pronounced by the user and generating a respective activation word electric signal SA, by the voice receiving means 2 integrated in the household appliance; and recognizing the activation word PA by the first processing means 3.
  • In the case the recognition of the activation word is successful, the method provides carrying out the step of receiving an acoustic voice signal SV corresponding to a voice query and/or request utterance and the successive steps shown in the previously described embodiments.
  • According to an implementation option, the step of generating a query and/or request electric signal SR comprises generating a respective audio stream query and/or request signal SR representative of the acoustic voice signal SV.
  • According to one embodiment of the method, the step of processing the electric query and/or request signal SR comprises recognizing whether the query and/or request is open or closed.
  • A query and/or request is recognized as closed if it corresponds to a determined query and/or request, such that the second processing means 4 are capable of providing a predetermined univocal response relating to the query and/or request, on the basis of information and/or states of the household appliance which are accessible to the second processing means 4.
  • A query and/or request is recognized as open in all cases in which the query and/or request is not recognized as closed.
  • According to an implementation option of the method, if the query and/or request is closed, the step of identifying the user query and/or request comprises identifying the recognized closed query and/or request as a user query and/or request; and the step of determining a response to be provided to the user comprises determining, as a response, the aforesaid predetermined univocal response relating to the closed query and/or request.
  • If the query and/or request is open, the step of identifying the user query and/or request comprises interpreting the open query and/or request as a partial query and/or request.
  • In this case, the method comprises the further steps of determining and sending an interlocutory response to the user, comprising a request for further clarifications; and setting a standby state prepared to receive a further completion voice utterance to complete the request and/or query by the user.
  • According to an implementation option of the method, the step of recognizing whether the user query and/or request is a command and/or a control action requiring remote control enabling, comprises: verifying whether the user query and/or request is closed; verifying whether the user query and/or request corresponds to a command and/or control action requiring remote control enabling, on the basis of information stored in the storage means 6 or otherwise accessible by the second processing means 4.
  • According to an implementation option of the method, the step of processing the query and/or request electric signal SR to identify a user query and/or request comprises interpreting the user query and/or request as a command and/or control action and/or question.
  • In this case, the method further comprises the following steps, according to the possible alternatives.
  • If the user query and/or request is interpreted as a command, the method provides transmitting a command signal corresponding to the command to the first processing means 3 of the household appliance, by the second processing means 4.
  • If the user query and/or request is interpreted as a control action, the method provides consulting, by the second processing means 4, storage means 6 in which state S of the household appliance 100 is periodically stored to define the action to be implemented and the execution mode.
  • If the user query and/or request is interpreted as a pertinent question, such that the second processing means 4 are capable of providing a univocal response, the method provides generating such a univocal response to the question, by the second processing means 4.
  • If the user query and/or request is interpreted as a non-pertinent question, such that the second processing means 4 are not capable of providing a univocal response, the method provides generating an interlocutory response for a request for further information, by the second processing means 4.
  • According to an embodiment of the method, the step of recognizing the activation word, by first processing means 3, and the step of processing the query and/or request electric signal SR, by the second processing means 4, comprise executing at least one voice recognition algorithm and/or procedure.
  • According to various possible implementation options, the voice recognition algorithm and/or procedure is selected between various possible types of algorithms and/or vocal recognition procedure in themselves known.
  • According to an implementation option, procedures based on machine learning computational techniques are used.
  • According to one embodiment of the method, the step of generating a response electric signal C corresponding to the determined response comprises generating a first string of text characters corresponding to the response to be provided to the user.
  • According to a particular implementation option, the aforesaid step further comprises generating a second string of text characters corresponding to the question of the user, as it is interpreted on the basis of at least one algorithm and/or procedure for transforming into text a voice command that can be executed by the second processing means 4.
  • According to an implementation option, procedures based on machine learning computational techniques or other algorithms or procedures in themselves known are used to transform a voice command into text.
  • According to anther embodiment of the method, the step of generating a response electric signal C corresponding to the determined response comprises, after the step of determining a response to the command and/or control action, the further steps of generating a response audio signal SAR, corresponding to the determined response, by the second processing means 4; then transmitting the aforesaid response audio signal SAR to the first processing means 3 of the household appliance; finally reproducing the response audio signal SAR to generate a voice response output, by voice output generation means 7 comprised in the household appliance.
  • With reference to the step of providing the user with a response, various embodiments are possible, consistently with the respective options for generating an above-mentioned electric response signal.
  • According to one embodiment, the step of providing the user with a response comprises providing the user with a written response, by means of display means 5 of the household appliance.
  • According to an implementation option, the written response comprises displaying the aforesaid first string of text characters and/or second string of text characters as pop-up on a screen 5 comprised in the household appliance.
  • According to another embodiment, the step of providing the user with a response comprises providing the user with visual information.
  • According to one embodiment, the step of providing the user with a response comprises providing the user with a voice response, by means of voice output generation means 7 comprised in the household appliance.
  • According to an implementation option, the vocal response comprises the sound reproduction of the aforesaid response audio signal SAR, which is generated by the second processing means 4 and transmitted to the first processing means 3.
  • With reference to Figure 3, certain further details are provided below, by mere way of example of an embodiment of the method of the invention.
  • Figure 3 shows a flow chart of the actions performed in an embodiment of the method, showing certain exchanges of relevant information, certain actions and verifications which are performed, and the related reference to the logic level and/or physical entity (belonging to the household appliance, in local or at a remote cloud) in which the various actions, verifications and exchanges of information occur.
  • In particular, the chart in Figure 3 shows the controls related to the enabling or non-enabling of the remote control and the subsequent actions.
  • Figure 3 also mentions a "parking storage" (which may, for example, be implemented in the remote storage means 6 or may belong to the second processing means 4), which is configured to save a command or request by the user which can only be executed if the remote control is enabled, pending the verification and/or enabling of the activation state of the remote control.
  • In this case, the stored command or request is retrieved by the second processing means 4 and suitably processed for the successive actions only when the remote control is enabled: this occurs if it has been verified that the state of the household appliance already provides "remote control ON" or, in the contrary case, after the user has possibly enabled the remote control by means of voice commands or other command.
  • With regards to the entities indicated on the left-hand part of the chart in Figure 3, it should be noted that the household appliance 100 in this embodiment comprises both the control unit 40 (which is also connected to the remote storage means 6 to provide information on the state of the household appliance) and the first processing means 3.
  • The second processing means 4 are implemented, in this example, in a distributed manner in a cloud and comprise, for example, a voice recognition software module.
  • The cloud further comprises the storage means 6.
  • With reference to Figure 1, a system for controlling and/or interacting with a household appliance 100 by means of voice commands and with verification of the enabling of a remote control is described below.
  • Such a system comprises voice reception means 2 and first processing means 3 integrated in the household appliance 100, second processing means 4 (typically, remote) and a user interface 5 comprised in the household appliance 100.
  • The aforesaid voice receiving means 2 integrated in the household appliance 100 are configured to receive an acoustic voice signal SV corresponding to a voice query and/or request utterance pronounced by the user and to transform it into a corresponding first electric signal SE representative of the acoustic voice signal SV.
  • The first processing means 3 integrated in the household appliance 100 are operatively connected to the aforesaid voice receiving means 2 to receive said first electric signal SE and are configured to generate a query and/or request electric signal SR based on said first electric signal SE, representative of the acoustic voice signal SV.
  • The second processing means 4 are operatively connected to the aforesaid first processing means 3 and are configured to carry out the following steps: receiving the query and/or request electric signal SR from the first processing means 3; processing such a query and/or request electric signal SR to identify a user query and/or request; and recognizing whether the user query and/or request is a command and/or control action which requires remote control enabling.
  • If the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, the second processing means 4 are configured to proceed directly with the following steps of determining and providing the user with a response, i.e. to perform the following actions: determining a response to be provided to the user, on the basis of the identified user query and/or request, generating a response electric signal C corresponding to the determined response, and transmitting such a response electric signal C to the first processing means 3 of the household appliance.
  • If the user query and/or request is recognized as a command and/or control action which requires remote control enabling, the second processing means 4 are configured to perform a remote control verification and/or enabling procedure and then perform the aforesaid steps of determining and providing a response to the user, which are executed only if and when the remote control verification and/or enabling procedure has been completed in a circumstance in which the remote control is enabled (i.e., in other words, only if and when the state of the household appliance is such as to ensure the electronic control is enabled).
  • The aforesaid remote control verification and/or enabling procedure comprises verifying whether a remote control enabling state is activated in the household appliance 100. If it is verified that a remote control enabling state is activated in the household appliance 100, the first processing means 3 of the household appliance are configured to determine the execution and/or the starting of the command and/or control action in the household appliance 100. If it is verified that a remote control enabling state is not activated in the household appliance 100, a standby state is set and the user is asked for an enabling action before executing the command and/or control action.
  • The user interface 5 of the household appliance 100 is operatively connected to the first processing means 3 and is configured to provide a visual and/or written and/or voice response to the user on the basis of the aforesaid response electric signal C.
  • With reference to Figures 1 and 2, further embodiments of the system 1 are described.
  • According to various possible embodiments of the system 1, the first processing means 3 and the second processing means 4 are also configured to carry out a method according to any one of the embodiments described above.
  • According to one embodiment, the system 1 further comprises a control unit 40 of the household appliance and further comprises storage means 6 which typically are located remotely with respect to the household appliance.
  • The control unit 40 of the household appliance is operatively connected to the first processing means 3 and configured to cooperate with the first processing means 3 in order to execute commands and/or control actions of the household appliance 100.
  • Moreover, the control unit 40 is configured to control and/or monitor information relating to state S of the household appliance, comprising information about the enabling or disabling of the remote control state of the household appliance.
  • The storage means 6 are operatively connected both to the second processing means 4 and to the control unit 40 of the household appliance and are configured to perform the following actions: receiving, from the aforesaid control unit 40, and storing the aforesaid information relating to state S of the household appliance, and providing the second processing means 4 with updated information on the enabling or disabling of the remote control state of the household appliance.
  • According to an implementation option of the system, the visual interface 5 comprises a screen 5 integrated in the household appliance and configured to display a string C of text characters corresponding to the response provided by the second processing means 4.
  • According to an implementation example, the aforesaid screen 5 comprises a liquid crystal display or other type of display in itself known, either already provided in the household appliance or added to it to provide the visual interface function.
  • According to another implementation option of the system, the second processing means 4 are further configured to generate a response audio stream signal SAR corresponding to the determined response, and to transmit such a response audio stream signal SAR to the first processing means 2.
  • In this case, system 1 further comprises voice output generation means 7 integrated in the household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output.
  • According to a particular implementation option of the system, the voice receiving means 2 comprise one or more microphones; the first processing means 3 comprise one or more electronic processing circuits and/or one or more electronic processors; and the system 1 further comprises transmission means 8 configured to transmit the aforesaid electric query or request signal SR.
  • According to an embodiment, the aforesaid voice receiving means 2, first processing means 3 and transmission means 8 are integrated in a voice recognition module 30 incorporated in the household appliance 100.
  • According to a particular implementation option of the system, the second processing means 4 comprise one or more electronic processing circuits and/or one or more electronic processors, arranged remotely with respect to the household appliance 100. In this case, system 1 further comprises telecommunication means 9 configured to connect the transmission means 8 of the household appliance to the second remote processing means 4.
  • According to an implementation example of the system, the second processing means 4 are implemented in a cloud infrastructure 20 and the telecommunication means 9 comprise wireless communication means, a router and an infrastructure for telecommunicating to the cloud infrastructure 20.
  • According to an embodiment of the system, the voice receiving means 2 are further configured to receive a first activation word PA pronounced by a user and to generate a respective activation word electric signal SA. The first processing means 3 are further configured to recognize the activation word, and in case of the activation word I successfully recognized, to execute the aforesaid step of generating a query and/or request electric signal SR.
  • According to another embodiment of the system, the voice receiving means 2 are further configured to receive a second activation word PA pronounced by a user (provided to enable the remote control state) and to generate a respective second activation word electric signal. The first processing means 3 are further configured to recognize the second activation word, and in case the second activation word is successfully recognized, to enable the remote control of the household appliance.
  • According to an implementation option, the aforesaid first and second activation words coincide and the enabling effect of the remote control occurs if the activation word is pronounced in response to a request by the system for the user to request remote control enabling.
  • According to an embodiment of the system, the first processing means 3 are further configured to generate a query and/or request audio stream signal SR representative of the acoustic voice signal SV, as the query and/or request electric signal SR.
  • According to an embodiment, system 1 further comprises voice output generation means 7 integrated in the household appliance 100 and configured to reproduce the response audio stream signal SAR to generate a voice response output.
  • According to an implementation option, the voice output generation means 7 comprise a speaker or other sound reproduction means 7 adapted to provide the audio response.
  • With reference to Figures 1, 2 and 4, a household appliance 100 according to the invention is now described.
  • The household appliance 100 comprises a first portion 10 of a system 1 comprising at least the aforesaid voice receiving means 2, first processing means 3 and visual interface 5 (according to any one of the implementation options described above).
  • The household appliance 100 is operatively connected to a second portion 20 of a system 1 comprising at least the aforesaid second processing means 4 (according to any one of the implementation options described above).
  • In the example shown in Figure 4, the household appliance is a washing machine. Figure 4 shows the visual interface, i.e., screen 5, the voice recognition module 30 and the sound reproduction means 7.
  • It is worth noting that the object of the present invention is fully achieved by the method and system illustrated above by virtue of the functional and structural features thereof.
  • Indeed, the above-described method and the system for controlling and/or communicating with a household appliance are capable of interpreting the user queries and/or requests and of discriminating the user queries and/or requests requiring remote control enabling from the user queries and/or requests requiring remote control which do not require remote control enabling.
  • Once such a discrimination has been performed, the required verification and/or remote control enabling procedures are performed for all and only those user queries and/or requests that actually require remote control enabling (thereby ensuring the adequate safety levels).
  • On the contrary, the user queries and/or requests that do not require remote control enabling are processed and performed without verification procedures and/or remote control enabling procedures (which are not required in these cases), thus ensuring response and execution speed and effectiveness.
  • A person skilled in the art may make changes and adaptations to the embodiments of the method and system described above or can replace elements with others which are functionally equivalent to satisfy contingent needs without departing from the scope of protection of the appended claims. All the features described above as belonging to one possible embodiment may be implemented independently from the other embodiments described.

Claims (15)

  1. A method for controlling and/or communicating with a household appliance (100) by means of voice commands and with verification of the enabling of a remote control, comprising the steps of:
    - receiving an acoustic voice signal (SV), corresponding to a voice query and/or request expression uttered by a user, and generating a corresponding first electric signal (SE) representative of the acoustic voice signal (SV), by voice receiving means (2) integrated in the household appliance (100);
    - generating a query and/or request electric signal (SR), based on said received first electric signal (SE), representative of the acoustic voice signal (SV), by first processing means (3) integrated in the household appliance (100) operatively connected to said voice receiving means (2);
    - transmitting said query and/or request electric signal (SR) to second processing means (4);
    - processing said query and/or request electric signal (SR), by the second processing means (4), to identify a user query and/or request;
    - recognizing, by the second processing means (4), whether the user query and/or request is a command and/or control action which requires remote control enabling;
    - if the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, proceeding directly with the following steps:
    - determining a response to be provided to the user on the basis of the identified user query and/or request, by the second processing means (4);
    - generating an response electric signal (C) corresponding to the determined response, by the second processing means (4);
    - transmitting said response electric signal (C) to the first processing means (3) of the household appliance;
    - providing the user with a visual and/or written and/or voice response, by visual display means (5) or sound reproduction means (7) comprised in the household appliance (100), based on said response electric signal (C);
    - if the user query and/or request is recognized as a command and/or control action which requires remote control enabling, performing a remote control verification and/or enabling procedure and then performing said steps of determining and providing a response to the user;
    wherein said remote control verification and/or enabling procedure comprises:
    - verifying whether a remote control enabling state is activated in the household appliance (100);
    - if it is verified that a remote control enabling state is activated in the household appliance (100), determining the execution of the command and/or control action in the household appliance (100);
    - if it is verified that a remote control enabling state is not activated in the household appliance (100), setting a standby state and asking the user for an enabling action before executing the command and/or control action.
  2. A method according to claim 1, wherein the step of verifying whether a remote control enabling state is activated in the household appliance (100) is performed by the first processing means (3) by means of local processing, through a connection to a local control unit (40) of the household appliance (100), or is performed by the second processing means (4) by means of a connection to storage means (6) which are in remote communication with the control unit (40) of the household appliance (100) and periodically receive and store the state (S) thereof.
  3. A method according to any one of the preceding claims, wherein if the user query and/or request is interpreted as a command and/or control action which requires remote control enabling, and it is verified that a remote control enabling state is not activated in the household appliance (100), the method further comprises the step of:
    - if the user performs the enabling action by means of voice activation word or by manual command, activating the remote control enabling state and either determining or starting the execution of the command and/or control action in the household appliance, by the first processing means (3) of the household appliance.
  4. A method according to any one of the preceding claims, comprising, before said step of receiving an acoustic voice signal (SV), the steps of:
    - receiving an activation word (PA) uttered by the user and generating a respective activation word electric signal (SA), by said voice receiving means (2) integrated in the household appliance;
    - carrying out a recognition of the activation word (PA), by said first processing means (3);
    - in the case the recognition of the activation word is successful, proceeding to carry out said step of receiving an acoustic voice signal (SV) corresponding to a voice query and/or request utterance and the subsequent steps as defined in claim 1.
  5. A method according to any one of the preceding claims, wherein the step of generating a query and/or request electric signal (SR) comprises generating a respective audio stream query and/or request signal (SR) representative of the acoustic voice signal (SV),
    and/or wherein the step of recognizing the activation word by first processing means (3) and the step of processing the query and/or request electric signal (SR), by the second processing means (4), comprise executing at least one voice recognition algorithm and/or procedure.
  6. A method according to any one of the preceding claims, wherein said step of processing the query and/or request electric signal (SR) comprises recognizing whether the query and/or request is open or closed,
    wherein a query and/or request is recognized as closed if it corresponds to a specific query and/or request, such that the second processing means (4) are capable of providing a predetermined univocal response relating to the query and/or request, based on information and/or states of the household appliance which are accessible to the second processing means (4),
    and wherein a query and/or request is recognized as open in all cases in which said query and/or request is not recognized as closed.
  7. A method according to claim 6, wherein:
    if the query and/or request is closed:
    - the step of identifying the user query and/or request comprises identifying the recognized closed query and/or request as a user query and/or request;
    - the step of determining a response to be provided to the user comprises determining, as a response, said predetermined univocal response relating to the closed query and/or request;
    if the query and/or request is open:
    - the step of identifying the user query and/or request comprises interpreting the open query and/or request as a partial query and/or request,
    and the method comprises the further steps of:
    - determining and sending an interlocutory response to the user, comprising a request for further clarifications;
    - setting a standby state prepared to receive a further completion voice utterance to complete the request and/or query by the user.
  8. A method according to any one of claims 6 or 7, wherein the step of recognizing whether the user query and/or request is a command and/or control action which requires remote control enabling, comprises:
    - verifying whether the user query and/or request is closed;
    - if the user query and/or request is closed, checking, on the basis of information stored in the storage means (6) or otherwise accessible by the second processing means (4), whether the user query and/or request corresponds to a command and/or control action requiring remote control enabling.
  9. A method according to any one of the preceding claims, wherein the step of processing the query and/or request electric signal (SR) to identify a user query and/or request, comprises:
    - interpreting the user query and/or request as a command and/or a control action and/or a question;
    and wherein the method further comprises the steps of:
    if the user query and/or request is interpreted as a command, transmitting a command signal corresponding to the command to the first processing means (3) of the household appliance, by the second processing means (4);
    if the user query and/or request is interpreted as a control action, consulting, by the second processing means (4), storage means (6) in which the state of the household appliance (100) is periodically stored, to define the action to be implemented and the execution mode;
    if the user query and/or request is interpreted as a pertinent question, such that the second processing means (4) are capable of providing a univocal response, generating said univocal response to the question, by the second processing means (4);
    if the user query and/or request is interpreted as a non-pertinent question, such that the second processing means (4) are not capable of providing a univocal response, generating an interlocutory response for a request for further information, by the second processing means (4).
  10. A system (1) for controlling and/or interacting with a household appliance (100) by means of voice commands and with verification of the enabling of a remote control, comprising:
    - voice receiving means (2) integrated in the household appliance (100), configured to receive an acoustic voice signal (SV), corresponding to a query and/or request voice expression pronounced by the user, and to transform it into a corresponding first electric signal (SE) representative of the acoustic voice signal (SV);
    - first processing means (3) integrated in the household appliance (100) operatively connected to said voice receiving means (2) to receive said first electric signal (SE), said first processing means (3) being configured to generate a query and/or request electric signal (SR) based on said first electric signal (SE), representative of the acoustic voice signal (SV);
    - second processing means (4) operatively connected to said first processing means (3), said second processing means (4) being configured to:
    receive said query and/or request electric signal (SR) from the first processing means (3);
    process said electric query and/or request signal (SR) to identify a user query and/or request;
    recognize whether the user query and/or request is a command and/or control action which requires remote control enabling;
    - if the user query and/or request is recognized as a command and/or control action which does not require remote control enabling, proceeding directly with the steps of:
    determining a response to be provided to the user, based on the identified user query and/or request, generating a response electric signal (C) corresponding to the determined response, and transmitting said response electric signal (C) to the first processing means (3) of the household appliance;
    - if the user query and/or request is recognized as a command and/or control action which requires remote control enabling, performing a remote control verification and/or enabling procedure and then performing said steps of determining and providing a response to the user;
    wherein said remote control verification and/or enabling procedure comprises:
    - verifying whether a remote control enabling state is activated in the household appliance (100);
    - if it is verified that a remote control enabling state is activated in the household appliance (100), determining the executing and/or the starting the command and/or control action in the household appliance (100);
    - if it is verified that a remote control enabling state is not activated in the household appliance (100), setting a standby state and asking the user for an enabling action before executing the command and/or control action;
    - a user interface (5) comprised in the household appliance (100), operatively connected to said first processing means (3), said user interface (5) being configured to provide a visual and/or written and/or voice response to the user based on said response electric signal (C).
  11. A system (1) according to claim 10, further comprising:
    - a control unit (40) of the household appliance operatively connected to the first processing means (3) and configured to cooperate with said first processing means (3) in order to execute commands and/or control actions of the household appliance (100), and further configured to control and/or monitor information relating to the state (S) of the household appliance, comprising information on the enabling or disabling of the remote control state of the household appliance;
    - storage means (6) operatively connected to the second processing means (4) and to said control unit (40) of the household appliance, said storage means (6) being configured to:
    - receive, from said control unit (40), and store said information relating to the state (S) of the household appliance;
    - provide the second processing means (4) with said updated information on the enabling or disabling of the remote control state of the household appliance.
  12. A system (1) according to any one of claims 10 or 11, wherein the visual interface (5) comprises a screen (5) integrated in the household appliance configured to display a string (C) of text characters corresponding to the response provided by the second processing means (4),
    and/or wherein the second processing means (4) are further configured to generate a response audio stream signal (SAR) corresponding to the determined response, and to transmit said response audio stream signal (SAR) to the first processing means (2);
    wherein the system (1) further comprises voice output generation means (7) integrated in the household appliance (100) and configured to reproduce the response audio stream signal (SAR) to generate a voice response output.
  13. A system according to any one of claims 10 to 12, wherein:
    - the voice receiving means (2) comprise one or more microphones;
    - the first processing means (3) comprise one or more electronic processing circuits and/or one or more electronic processors;
    and wherein the system (1) further comprises transmission means (8) configured to transmit said query or request electric signal (SR),
    and/or wherein said voice receiving means (2), said first processing means (3) and said transmission means (8) are integrated in a voice recognition module (30) incorporated in the household appliance (100).
  14. A system (1) according to any one of claims 11 to 13, wherein the second processing means (4) comprise one or more electronic processing circuits and/or one or more electronic processors, arranged remotely with respect to the household appliance (100),
    and wherein the system (1) further comprises telecommunication means (9) configured to connect the transmission means (8) of the household appliance to the second processing means (4),
    and wherein the second processing means (4) are implemented in a cloud infrastructure (20) and the telecommunication means (9) comprise wireless communication means, a router and an infrastructure for telecommunicating to the cloud infrastructure (20).
  15. A household appliance (100) comprising a first portion (10) of a system (1) according to any one of claims 10 to 14, comprising at least said voice receiving means (2), first processing means (3) and visual interface (5),
    said household appliance (100) being operatively connected to a second portion (20) of a system (1) according to any one of claims 10 to 14, comprising at least said second processing means (4).
EP20206707.0A 2019-11-12 2020-11-10 Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control Active EP3822966B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
IT102019000020943A IT201900020943A1 (en) 2019-11-12 2019-11-12 Method and system for controlling and / or communicating with an appliance using voice commands with verification of the enabling of a remote control

Publications (2)

Publication Number Publication Date
EP3822966A1 true EP3822966A1 (en) 2021-05-19
EP3822966B1 EP3822966B1 (en) 2023-09-13

Family

ID=69903840

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20206707.0A Active EP3822966B1 (en) 2019-11-12 2020-11-10 Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control

Country Status (3)

Country Link
EP (1) EP3822966B1 (en)
ES (1) ES2965450T3 (en)
IT (1) IT201900020943A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150302856A1 (en) * 2014-04-17 2015-10-22 Qualcomm Incorporated Method and apparatus for performing function by speech input
EP3413304A2 (en) * 2017-05-19 2018-12-12 LG Electronics Inc. Method for operating home appliance and voice recognition server system
US20190228780A1 (en) * 2018-01-24 2019-07-25 Comcast Cable Communications, Llc Verification of user identity for voice enabled devices

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150302856A1 (en) * 2014-04-17 2015-10-22 Qualcomm Incorporated Method and apparatus for performing function by speech input
EP3413304A2 (en) * 2017-05-19 2018-12-12 LG Electronics Inc. Method for operating home appliance and voice recognition server system
US20190228780A1 (en) * 2018-01-24 2019-07-25 Comcast Cable Communications, Llc Verification of user identity for voice enabled devices

Also Published As

Publication number Publication date
EP3822966B1 (en) 2023-09-13
ES2965450T3 (en) 2024-04-15
IT201900020943A1 (en) 2021-05-12

Similar Documents

Publication Publication Date Title
JP5208246B2 (en) Voice input device having a period of interest
US20210241775A1 (en) Hybrid speech interface device
JP6480568B2 (en) Voice application architecture
CN109671426B (en) Voice control method and device, storage medium and air conditioner
CN105793923A (en) Local and remote speech processing
CN109144458B (en) Electronic device for performing operation corresponding to voice input
CN109584883A (en) The method and system of mobile terminal, long-range vocal print control vehicle device
JP2016018078A (en) Voice recognition device and voice recognition system
CN106218557B (en) Vehicle-mounted microphone with voice recognition control function
CN106601242A (en) Executing method and device of operation event and terminal
CN112399222A (en) Voice instruction learning method and device for smart television, smart television and medium
EP3822966B1 (en) Method and system for controlling and/or communicating with a household appliance by means of voice commands with verification of the enabling of a remote control
CN109979445A (en) It is a kind of for understanding the system and method for standard language and dialect
EP3826003A1 (en) Method and system for controlling and/or interacting with a household appliance by means of voice commands and with recognition of the language of the voice commands, and corresponding household appliance
KR20200057501A (en) ELECTRONIC APPARATUS AND WiFi CONNECTING METHOD THEREOF
CN112203902B (en) Device control apparatus and control method for controlling device
CN110648664A (en) Household appliance control method and device with storage function
CN109102808A (en) A kind of local speech recognition system and its method based on display interaction
EP3796310B1 (en) Method and system for controlling and/or communicating with a domestic appliance by means of voice commands and text displays
CN109361820B (en) Electronic equipment control method and system and electronic equipment
CN113284488A (en) Control method and device for household appliance, voice box and storage medium
KR102417899B1 (en) Apparatus and method for recognizing voice of vehicle
US20200342876A1 (en) Method, device and computer-readable storage medium having instructions for processing a speech input, transportation vehicle, and user terminal with speech processing
KR100423495B1 (en) Operation control system by speech recognition for portable device and a method using the same
EP3839719B1 (en) Computing device and method of operating the same

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20211028

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: D06F 105/60 20200101ALN20230314BHEP

Ipc: D06F 105/58 20200101ALN20230314BHEP

Ipc: D06F 101/00 20200101ALN20230314BHEP

Ipc: D06F 34/32 20200101ALN20230314BHEP

Ipc: D06F 34/28 20200101ALN20230314BHEP

Ipc: G10L 15/08 20060101ALN20230314BHEP

Ipc: H04L 12/28 20060101ALI20230314BHEP

Ipc: D06F 34/05 20200101ALI20230314BHEP

Ipc: G06F 3/16 20060101ALI20230314BHEP

Ipc: G10L 15/22 20060101AFI20230314BHEP

INTG Intention to grant announced

Effective date: 20230404

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602020017553

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231214

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20231201

Year of fee payment: 4

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231213

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231214

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20231211

Year of fee payment: 4

Ref country code: IT

Payment date: 20231130

Year of fee payment: 4

Ref country code: FR

Payment date: 20231130

Year of fee payment: 4

Ref country code: DE

Payment date: 20231121

Year of fee payment: 4

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1612126

Country of ref document: AT

Kind code of ref document: T

Effective date: 20230913

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20240116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240113

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2965450

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20240415

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240113

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240115

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913