WO2020138844A1 - Appareil ménager et procédé de reconnaissance vocale associé - Google Patents

Appareil ménager et procédé de reconnaissance vocale associé Download PDF

Info

Publication number
WO2020138844A1
WO2020138844A1 PCT/KR2019/018113 KR2019018113W WO2020138844A1 WO 2020138844 A1 WO2020138844 A1 WO 2020138844A1 KR 2019018113 W KR2019018113 W KR 2019018113W WO 2020138844 A1 WO2020138844 A1 WO 2020138844A1
Authority
WO
WIPO (PCT)
Prior art keywords
home appliance
noise
voice data
data
voice
Prior art date
Application number
PCT/KR2019/018113
Other languages
English (en)
Inventor
Nokhaeng LEE
Original Assignee
Samsung Electronics Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co., Ltd. filed Critical Samsung Electronics Co., Ltd.
Priority to AU2019416316A priority Critical patent/AU2019416316A1/en
Priority to EP19904108.8A priority patent/EP3844750A4/fr
Priority to CN201980086395.2A priority patent/CN113287168A/zh
Publication of WO2020138844A1 publication Critical patent/WO2020138844A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/20—Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/26—Pre-filtering or post-filtering
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L21/0232—Processing in the frequency domain
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G10L25/84—Detection of presence or absence of voice signals for discriminating voice from noise
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165—Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal

Definitions

  • Apparatuses and methods consistent with the disclosure relate to a home appliance and a method for voice recognition thereof, and more particularly, to a home appliance which estimates noise using information on noise that occurs in another home appliance and reduces the estimated noise to increase a voice recognition rate, and a method for voice recognition thereof.
  • Home appliances may perform various functions according to control commands from users. Recently, home appliances employ a voice recognition function to receive a control command through a user voice, as well as receiving a control command via an input device such as a keypad, a remote controller, and the like.
  • home appliances to which a method of activating a voice recognition function based on a specific start command (e.g., Bixby) when a user speaks the specific start command has been increasingly adopted.
  • a specific start command e.g., Bixby
  • Embodiments of the disclosure overcome the above disadvantages and other disadvantages not described above. Also, the disclosure is not required to overcome the disadvantages described above, and an embodiment of the disclosure may not overcome any of the problems described above.
  • the disclosure provides a home appliance which estimates noise using information on noise produced in another home appliance and reduces the estimated noise to increase a voice recognition rate, and a method for voice recognition thereof.
  • a home appliance includes: a communication device configured to communicate with another home appliance; a microphone configured to receive a voice from a user; and a processor configured to perform signal processing on first voice data obtained from the microphone and perform voice recognition using the signal-processed first voice data, wherein the processor generates noise data using second voice data received from the other home appliance and performs the signal processing on the first voice data using the generated noise data.
  • a method for voice recognition of a home appliance includes: obtaining first voice data from a microphone; performing signal processing on the obtained first voice data; and performing voice recognition using the signal-processed first voice data, wherein the performing of signal processing includes: receiving second voice data from another home appliance; generating noise data using the received second voice data; and performing signal processing on the first voice data using the generated noise data.
  • a recording medium storing a program for performing a method for voice recognition, in which the method includes: requesting voice data from a first home appliance and a second home appliance; receiving first voice data and second voice data respectively obtained from the first home appliance and the second home appliance based on the request; generating noise data using the received second voice data; performing signal processing on the first voice data using the generated noise data; and transmitting the signal-processed first voice data to the first home appliance.
  • FIG. 1 is a block diagram of a voice recognition system according to an embodiment of the disclosure
  • FIG. 2 is a block diagram illustrating a simple configuration of a first home appliance according to an embodiment of the disclosure
  • FIG. 3 is a block diagram illustrating a specific configuration of a first home appliance according to an embodiment of the disclosure
  • FIG. 4 is a block diagram illustrating a simple configuration of a second home appliance according to an embodiment of the disclosure
  • FIG. 5 is a block diagram illustrating a simple configuration of a server according to an embodiment of the disclosure.
  • FIG. 6 is a diagram illustrating a noise removing method according to a first embodiment
  • FIG. 7 is a diagram illustrating a noise removing method according to the first embodiment
  • FIG. 8 is a diagram illustrating a noise removing method according to a second embodiment
  • FIG. 9 is a diagram illustrating a noise removing method according to the second embodiment.
  • FIG. 10 is a diagram illustrating a noise removing method according to the second embodiment
  • FIG. 11 is a diagram illustrating a noise removing method according to a third embodiment
  • FIG. 12 is a diagram illustrating a noise removing method according to the third embodiment.
  • FIG. 13 is a diagram illustrating a noise removing method according to the third embodiment.
  • FIG. 14 is a sequence diagram illustrating a voice recognition method according to the first embodiment
  • FIG. 15 is a sequence diagram illustrating a voice recognition method according to the first embodiment
  • FIG. 16 is a sequence diagram illustrating a voice recognition method according to the first embodiment
  • FIG. 17 is a flowchart illustrating a voice recognition method according to the second embodiment
  • FIG. 18 is a flowchart illustrating a voice recognition method according to the second embodiment
  • FIG. 19 is a flowchart illustrating a voice recognition method of a first home appliance according to an embodiment of the disclosure.
  • FIG. 20 is a flowchart illustrating a voice recognition method of a second home appliance according to an embodiment of the disclosure.
  • FIG. 21 is a flowchart illustrating a voice recognition method of a server according to an embodiment of the disclosure.
  • various functions described below can be implemented or supported by one or more computer programs, each of which is formed from computer readable program code and embodied in a computer readable medium.
  • application and “program” refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, related data, or a portion thereof adapted for implementation in a suitable computer readable program code.
  • computer readable program code includes any type of computer code, including source code, object code, and executable code.
  • computer readable medium includes any type of medium capable of being accessed by a computer, such as read only memory (ROM), random access memory (RAM), a hard disk drive, a compact disc (CD), a digital video disc (DVD), or any other type of memory.
  • ROM read only memory
  • RAM random access memory
  • CD compact disc
  • DVD digital video disc
  • a "non-transitory” computer readable medium excludes wired, wireless, optical, or other communication links that transport transitory electrical or other signals.
  • a non-transitory computer readable medium includes media where data can be permanently stored and media where data can be stored and later overwritten, such as a rewritable optical disc or an erasable memory device.
  • FIGS. 1 through 21, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged system or device.
  • FIG. 1 is a diagram illustrating a voice recognition system according to an embodiment of the disclosure.
  • a voice recognition system 1000 includes a first home appliance 100, a second home appliance 200, and a server 300.
  • the first home appliance 100 and the second home appliance 200 are electric machine tools used in a home, and include a robot cleaner, a vacuum cleaner, an electric range, a gas range, an electric wave oven, a range hood, a washing machine, a dryer, a refrigerator, a dishwasher, an air conditioner, and the like.
  • the first home appliance 100 and the second home appliance 200 may perform a voice recognition function and perform a function according to a voice recognition result.
  • voice recognition refers to a technique of converting an acoustic signal of an input speech into a word or sentence.
  • the first home appliance 100 and the second home appliance 200 may detect a user speaking voice and perform voice recognition on the detected voice. Specifically, when a wake up word (WUW) which is a trigger voice command for activating the voice recognition function is detected, the first home appliance 100 and the second home appliance 200 activate the voice recognition function and perform voice recognition using voice data of a voice input thereafter.
  • WUW wake up word
  • the second home appliance 200 may be a home appliance that produces large magnitude noise.
  • the second home appliance 200 may produce large magnitude noise due to a motor, a fan, or the like included therein.
  • the voice recognition function of the first home appliance 100 may not operate properly. Therefore, the first home appliance 100 may estimate the noise using information on the large magnitude noise produced in the second home appliance 200 and reduce the estimated noise to increase a voice recognition rate.
  • the first home appliance 100 may transmit a recording request signal to the second home appliance 200.
  • the first home appliance 100 may receive second voice data obtained based on the recording request signal from the second home appliance 200.
  • the second voice data is voice data regarding noise produced in the second home appliance 200.
  • the first home appliance 100 may obtain first voice data based on the recording request signal.
  • the first home appliance 100 may obtain first voice data and second voice data of the voice detected at the same time.
  • the first home appliance 100 may generate noise data using the second voice data.
  • the noise data refers to sound data corresponding to pure noise of a noise source.
  • the first home appliance 100 may generate the noise data by applying the second voice data to a noise path estimation filter. A specific operation of generating noise data will be described later with reference to FIGS. 6 through 10.
  • the first home appliance 100 may signal-process the first voice data using the generated noise data. Specifically, the first home appliance 100 may remove noise included in the first voice data using the generated noise data.
  • the first home appliance 100 may perform voice recognition using the first signal-processed voice data.
  • the second home appliance 200 may be connected to the first home appliance 100 through wired or wireless communication.
  • the second home appliance 200 may receive a recording request signal from the first home appliance 100.
  • the second home appliance 200 may obtain second voice data based on the received recording request signal.
  • the obtained second voice data may be transmitted to the first home appliance 100.
  • the recording request signal transmission operation and the signal processing operation on the first voice data of the first home appliance 100 may also be performed by the server 300.
  • the server 300 may be connected to the first home appliance 100 and the second home appliance 200 through wired or wireless communication.
  • the server 300 may transmit a recording request signal to the first home appliance 100 and the second home appliance 200.
  • the server 300 may receive first voice data obtained based on the recording request signal from the first home appliance 100.
  • the server 300 may receive second voice data obtained based on the recording request signal from the second home appliance 200.
  • the server 300 may generate noise data using the second voice data. Specifically, the server 300 may generate the noise data by applying the second voice data to the noise path estimation filter.
  • the server 300 may signal-process the first voice data using the generated noise data.
  • the server 300 may transmit the signal-processed first voice data to the first home appliance 100.
  • the first home appliance 100 is connected to one second home appliance 200, but it may also be implemented such that the first home appliance 100 is connected to a plurality of home appliances and perform signal processing using information on noise produced in the plurality of home appliances.
  • the server 300 is connected to two home appliances, but it may also be implemented such that the server 300 is connected to three or more home appliances.
  • FIG. 1 it is illustrated and described that the server 300 is included in the voice recognition system 1000, but only the first home appliance 100 and the second home appliance 200 may be implemented without the server 300.
  • FIG. 2 is a block diagram illustrating a simple configuration of a first home appliance according to an embodiment of the disclosure.
  • the first home appliance 100 may include a microphone 110, a communication device 120, and a processor 130.
  • the microphone 110 may output a user's spoken voice and a surrounding sound as voice data.
  • the microphone 110 may deliver the output voice data to the processor 130.
  • the microphone 110 may be disposed on a surface of the housing. Specifically, the microphone 110 may be disposed on a surface of the housing to generate first voice data corresponding to a sound near the first home appliance 100 in order to collect the user's spoken voice.
  • an arrangement position of the microphone 110 is not limited to the example described above.
  • the first home appliance 100 includes one microphone, but the first home appliance 100 may include two or more microphones when implemented.
  • the communication device 120 may be connected to an external device and may receive various data from the external device.
  • the communication device 120 may be connected to an external device through a local area network (LAN) and the Internet, as well as through a universal serial bus (USB) port or a wireless communication (e.g., Wi-Fi 802.11a/b/g/n, NFC, Bluetooth).
  • LAN local area network
  • USB universal serial bus
  • the external device may be a PC, a notebook, a smartphone, a server, or the like.
  • the communication device 120 may be connected to the second home appliance 200 or the server 300 in the voice recognition system 1000.
  • the communication device 120 may communicate with the second home appliance 200 in the voice recognition system 1000 through Bluetooth communication and communicate with the server 300 in the voice recognition system 1000 through Wi-Fi communication.
  • the method for the first home appliance 100 to communicate with the second home appliance 200 or the server 300 is not limited to the example described above.
  • the processor 130 controls the first home appliance 100. Specifically, the processor 130 may control each component of the first home appliance 100 according to a user's control command. For example, when the first home appliance 100 is a refrigerator, the processor 130 may control an operation of a compressor to provide cooled air to a cooling chamber containing food based on a cooling command being received.
  • the processor 130 may perform voice recognition using first voice data obtained through the microphone 110.
  • the first voice data may include noise as well as a user's voice, and a voice recognition rate may be lowered by the contained noise.
  • the processor 130 may perform preprocessing on the first voice data.
  • the preprocessing is a series of signal processing performed before voice recognition and may remove noise included in the voice data.
  • the processor 130 may perform voice recognition using the preprocessed first voice data.
  • the noise included in the first voice data may be due to the second home appliance 200 disposed near the first home appliance 100 and producing large magnitude noise. Therefore, the processor 130 obtains information on the noise produced in the second home appliance 200 from the second home appliance 200 and preprocesses the first voice data using the obtained information on the noise.
  • the processor 130 may control the communication device 120 to detect another connectable home appliance. Specifically, the processor 130 may control the communication device 120 to detect the second home appliance connectable through short-range wireless communication or the second home appliance 200 connected to an access point (AP) to which the first home appliance 100 is connected.
  • AP access point
  • the second home appliance 200 available for short-range wireless communication with the first home appliance 100 or connected to the AP to which the first home appliance 100 is connected is expected to be disposed near the first home appliance 100. Therefore, it may be inferred that the noise produced in the second home appliance 200 may affect the voice recognition function of the first home appliance 100.
  • the processor 130 may control the communication device 120 to request transmission of voice data from the second home appliance 200. Specifically, the processor 130 may control the communication device 120 to transmit a recording request signal for requesting generation and transmission of second voice data, which is voice data of the noise produced in the second home appliance 200, to the second home appliance 200.
  • the recording request signal may include information on a predetermined time.
  • the second home appliance 200 may detect noise using a microphone and generate second voice data at a predetermined time based on the received recording request signal. For example, when information on the predetermined time included in the recording request signal is 3:00:30 p.m., the second home appliance 200 may detect noise using the microphone at 3:00:30 p.m. and generate second voice data.
  • the information on the predetermined time may be implemented to include absolute time information as in the example described above and may be implemented to include relative time information such as "x seconds after the recording request signal is received".
  • the recording request signal may include information on the predetermined time length of the second voice data.
  • the second home appliance 200 may generate second voice data having a predetermined time length based on the received recording request signal. For example, when the information on the predetermined time length included in the recording request signal is 3 seconds, the second home appliance 200 may generate second voice data having a length of 3 seconds.
  • the information that may be included in the recording request signal is not limited to the example described above.
  • the processor 130 may receive second voice data generated based on the request transmitted to the second home appliance 200 from the second home appliance 200. Specifically, the processor 130 may receive the second voice data generated based on the predetermined time information and the predetermined time length information included in the recording request signal from the second home appliance 200.
  • the processor 130 may receive the second voice data regarding the voice detected through the microphone of the second home appliance 200 from 3:00:30 p.m. to 3:00:32 p.m.
  • the processor 130 may obtain first voice data based on the request transmitted to the second home appliance 200. Specifically, the processor 130 may obtain first voice data based on the predetermined time information and predetermined time length information included in the recording request signal transmitted to the second home appliance 200.
  • the processor 130 may obtain the first voice data regarding the voice detected through the microphone 110 from 3:00:30 p.m. to 3:00:32 p.m.
  • the first voice data and the second voice data being obtained are generated based on the recording request signal, and thus correspond to voice data detected by the first home appliance 100 and the second home appliance 200 at the same time, respectively.
  • the second voice data is voice data for noise directly detected in the second home appliance 200, and thus may include a noise having a magnitude larger than that included in the first voice data.
  • the noise included in the first voice data may be more accurately extracted than when only the first voice data is used.
  • the processor 130 may generate noise data which is sound data corresponding to pure noise of a noise source in the second home appliance 200 using the second voice data.
  • the processor 130 may use a noise path estimation filter to generate noise data.
  • the noise path estimation filter refers to a filter for filtering components other than the noise of the noise source included in the second voice data.
  • the noise path estimation filter may be referred to as a filtering algorithm.
  • a user's speech component included in the second voice data may be filtered to output noise data.
  • a specific operation of generating noise data using the noise path estimation filter will be described later with reference to FIGS. 6 through 10.
  • the processor 130 may use a pre-stored noise path estimation filter or receive a noise path estimation filter from an external device, and generate noise data using the received noise path estimation filter. Meanwhile, a specific operation of receiving the noise path estimation filter from outside and generating noise data will be described later with reference to FIGS. 11 through 13.
  • the processor 130 may perform preprocessing on the first voice data using the noise data. Specifically, the processor 130 may perform preprocessing on the first voice data by removing a component corresponding to noise of the noise source from the first voice data using the noise data generated through the noise path estimation filter.
  • the operations of the processor 130 may be repeatedly performed according to a predetermined period. Specifically, the processor 130 may perform a sequential operation of requesting voice data from the second home appliance 200, receiving second voice data obtained based on the request from the second home appliance 200, obtaining first voice data corresponding to the request, generating noise data using the received second voice data, and performing preprocessing on the first voice data using the generated noise data, according to a predetermined period.
  • the processor 130 may change the preprocessing method for the first voice data according to whether noise is produced in the second home appliance 200 at present.
  • the processor 130 may change the preprocessing method for the first voice data according to whether the motor of the second home appliance 200 is driven.
  • the processor 130 may control the communication device 120 to determine whether the motor of the second home appliance 200 is driven. If the motor of the second home appliance 200 is driven, the noise produced in the second home appliance 200 is large in magnitude, and thus, preprocessing may be performed on the first voice data using the second voice data according to the method described above.
  • voice recognition may be immediately performed using the first voice data without using the second voice data or preprocessing may be performed on the first voice data according to the method of the related art.
  • the processor 130 may change the preprocessing method for the first voice data according to whether the noise source is driven.
  • the processor 130 may perform voice recognition using the preprocessed first voice data.
  • the first home appliance performs the operation of transmitting the recording request signal to the second home appliance 200, receiving the second voice data from the second home appliance 200, and performing preprocessing on the first voice data, but it may be implemented such that the server 300 instead of the first home appliance 100 performs the operation described above and the first home appliance 100 simply receives preprocessed first voice data from the server 300 and performs voice recognition. A specific operation thereof will be described later with reference to FIG. 5.
  • FIG. 3 is a block diagram illustrating a specific configuration of a first home appliance according to an embodiment of the disclosure.
  • the first home appliance 100 may include the microphone 110, the communication device 120, the processor 130, an input device 140, a memory 150, and a display 160.
  • the microphone 110 and the communication device 120 perform the same functions as those of FIG. 2, and thus redundant descriptions thereof will be omitted. Also, the processor 130 has been described above with reference to FIG. 2, and thus, the descriptions of FIG. 2 are not repeated and only the contents related to the components added to FIG. 3 will be described below.
  • the input device 140 may include a plurality of function keys for the user to set or select various functions supported by the first home appliance 100. Through this, the user may input various control commands for the first home appliance 100. For example, when the first home appliance 100 is a washing machine, the user may input a spin-dry command of the washing machine through the input device 140.
  • the memory 150 stores various data for an overall operation of the first home appliance 100, such as a program for processing or controlling the processor 130. Specifically, the memory 150 may store a plurality of application programs run in the first home appliance 100 and data and instructions for operating the first home appliance 100.
  • the memory 150 may be accessed by the processor 130, and data reading/writing/modifying/deleting/updating may be performed by the processor 130.
  • the memory 150 may be implemented not only as a storage medium in the first home appliance 100 but also as an external storage medium, a removable disk including a USB memory, a web server through a network, and the like.
  • the memory 150 may store a noise path estimation filter necessary to generate noise data.
  • the display 160 may display various types of information provided by the first home appliance 100. Specifically, the display 160 may display an operation state of the first home appliance 100 or may display a user interface window for selecting a function and an option selected by the user.
  • the display 160 may display that the washing machine is performing a spin-dry operation or display an interface window allowing the user to select how many minutes the spin-dry function is to be operated.
  • the display 160 may display a result of performing the voice recognition function or may display an interface window so that the user may change a setting for the voice recognition function.
  • a beamforming technology of obtaining a plurality of voice signals including a voice and noise through a plurality of microphones and separating the voice and the noise using features that the voice and noise are incident in different directions and frequency spectrums thereof are different is used to remove noise.
  • the first home appliance obtains second voice data regarding the corresponding noise from another home appliance that produces large magnitude noise, and performs preprocessing using the obtained second voice data, whereby noise included in the voice data may be accurately removed even though a magnitude of the noise produced from outside is large.
  • FIG. 4 is a block diagram illustrating a simple configuration of a second home appliance according to an embodiment of the disclosure.
  • the second home appliance 200 may include a microphone 210, a communication device 220, a processor 240, a motor 230, an accelerometer 250, an input device 260, and a memory 270.
  • the microphone 210 is a device that converts a sound into a sound signal and may output a user's spoken voice and surrounding sounds as voice data.
  • the microphone 210 may delivery the output voice data to the processor 240.
  • the microphone 210 may be disposed on a surface of the housing. Specifically, in order to collect the user's spoken voice, the microphone 210 may be disposed on the surface of the housing to generate second voice data corresponding to a sound around the second home appliance 200.
  • the microphone 210 may be disposed in the housing. Specifically, to collect a noise sound produced in the second home appliance 200 itself, the microphone 210 may be disposed inside the housing (specifically, near a noise source that produces noise such as a motor) to generate second voice data corresponding to the sound generated by the second home appliance 200.
  • an arrangement position of the microphone 210 is not limited to the example described above.
  • the second home appliance 200 includes one microphone but the second home appliance 200 may include two or more microphones when implemented.
  • the communication device 220 may be connected to an external device and receive various data from the external device. Specifically, the communication device 220 may be connected to an external device through a local area network (LAN) and the Internet, as well as through a universal serial bus (USB) port or a wireless communication Wi-Fi 802.11a/b/g/n, NFC, Bluetooth) port.
  • the external device may be a PC, a notebook, a smartphone, a server, or the like.
  • the communication device 220 may be connected to the first home appliance 100 or the server 300 in the voice recognition system 1000.
  • the communication device 220 may communicate with the first home appliance 100 in the voice recognition system 1000 through Bluetooth communication and communicate with the server 300 in the voice recognition system 1000 through Wi-Fi communication.
  • the method for the second home appliance 200 to communicate with the first home appliance 100 or the server 300 is not limited to the example described above.
  • the motor 230 is disposed in the second home appliance 200 to drive a component related to performing of a function of the second home appliance 200.
  • the motor 230 may rotate a drum containing laundry at a high speed for spin-dry of the laundry.
  • vibration and noise may occur while the motor 230 starts the drum.
  • the motor 230 may start a refrigerant compressor that generates a refrigerant.
  • vibration and noise may be generated while the motor 230 starts the refrigerant compressor.
  • the motor 230 may be a noise source when the second home appliance 200 produces large magnitude noise by itself. Therefore, the microphone 210 may be disposed in the vicinity of the motor 230 and detect a sound produced by the motor 230 and generate second voice data corresponding thereto.
  • the processor 240 controls the second home appliance 200. Specifically, the processor 240 may control each component of the second home appliance 200 according to a control command of the user. For example, when the second home appliance 200 is a washing machine, the processor 240 may control the operation of the motor to provide a rotational force to the drum containing the laundry when a spin-dry command is received.
  • the processor 240 may receive a request for transmission of second voice data. Specifically, the processor 240 may receive a recording request signal requesting generation and transmission of the second voice data which is voice data regarding noise produced in the second home appliance 200 from the first home appliance 100 or the server 300 connected to the second home appliance 200.
  • the processor 240 may obtain second voice data based on the received request. Specifically, the processor 240 may detect noise using the microphone 210 based on the received recording request signal and obtain second voice data. More specifically, the processor 240 may obtain the second voice data based on information on a predetermined time and predetermined time length information included in the received recording request signal.
  • the processor 240 may control the communication device 220 to transmit the obtained second voice data. Specifically, the processor 240 may transmit the obtained second voice data to the first home appliance 100 or the server 300 connected to the second home appliance 200.
  • the second voice data transmitted to the first home appliance 100 or the server 300 is voice data regarding noise produced in the second home appliance 200, and thus, may be used to perform preprocessing to remove the noise included in the first voice data obtained in the first home appliance 100.
  • the first home appliance 100 or the server 300 may generate noise data which is pure motor noise data regarding the motor of the second home appliance 200 and perform preprocessing to remove the motor noise from the first voice data using the generated motor noise data.
  • the processor 240 may obtain the second voice data at each predetermined period when the recording request signal is received, and transmit the obtained second voice data as described above.
  • the processor 240 may also provide other types of reference data regarding the noise to the first home appliance 100 or the server 300.
  • the reference data may be data including information of the noise source.
  • the information of the noise source may include a magnitude and phase of vibration produced from the noise source, a magnitude and phase of noise produced from the noise source, main frequency information, and the like.
  • the processor 240 may generate noise data by obtaining reference data through the accelerometer 250 or by obtaining reference data through a control command input through the input device 260. A specific operation thereof will be described below.
  • the accelerometer 250 is a device for measuring acceleration of an object.
  • the accelerometer 250 may be disposed in the vicinity of the motor 230 to measure acceleration of the motor 230 and generate information on the measured acceleration.
  • the processor 240 may extract an operating frequency of the motor 230 from the obtained acceleration information and generate reference data using the extracted operating frequency.
  • the processor 240 may generate reference data represented by a trigonometric function having a specific size and phase using 50 Hz.
  • the input device 260 may include a plurality of function keys allowing the user to set or select various functions supported in the second home appliance 200. Through this, the user may input various control commands regarding the second home appliance 200.
  • a control command input through the input device 260 may be related to driving of the motor 230.
  • an operating frequency of the motor 230 corresponding to the control command input through the input device 260 may be checked.
  • the motor 230 may rotate the drum of the washing machine to perform a spin-dry function. In this case, it may be checked that the operating frequency of the motor 230 corresponding to the spin-dry command is 50 Hz.
  • the processor 240 may generate reference data using an operating frequency of the motor 230 corresponding to the control command.
  • This may be applied in the same manner to a control command generated by the processor 240 itself according to a situation determination.
  • the processor 240 may transmit at least one of second voice data obtained through the microphone 210 and reference data generated using a driving frequency identified from the control command or the acceleration information of the accelerometer 250 to the first home appliance 100 or the server 300.
  • the first home appliance 100 or the server 300 may generate noise data which is sound data corresponding to pure noise of the noise source in the second home appliance 200 using the received reference data.
  • the processor 240 may not transmit the above-described second voice data and reference data to the first home appliance 100 or the server 300, and instead, the processor 240 may generate noise data using the second voice data and the reference data and transmit the generated noise data to the first home appliance 100 or the server 300.
  • a noise path estimation filter may be previously stored in the second home appliance 200.
  • the first home appliance 100 or the server 300 may receive noise data and perform preprocessing to remove noise included in the first voice data using the received noise data.
  • the memory 270 stores various data for an overall operation of the second home appliance 200, such as a program for processing or controlling of the processor 240. Specifically, the memory 270 may store a plurality of application programs driven in the second home appliance 200 and data and instructions for the operation of the second home appliance 200.
  • the memory 270 may store driving frequency information of the motor 230 corresponding to the control command input through the input device 260.
  • the processor 240 may identify a driving frequency corresponding to the input control command and generate reference data using the identified driving frequency.
  • noise source is illustrated and described as being the motor 230 in FIG. 4, but the noise source may be a fan or another component, not the motor 230 when implemented.
  • FIG. 4 a single motor is illustrated and described, but a plurality of motors may be provided at the time of implementation, a plurality of motors may be provided, and noise may be estimated using reference data for each of the plurality of motors.
  • the second home appliance includes both a microphone and an accelerometer, but only the accelerometer may be provided at the time of implementation and reference data may be obtained through the accelerometer. Alternatively, only the microphone may be provided and second voice data may be obtained through the microphone. Alternatively, the second microphone and the accelerometer may not be provided, and reference data may be obtained through an input device.
  • FIG. 4 it is illustrated and described that the operation of generating the reference data using the operating frequency of the motor is performed by the processor, but a sine wave signal generator generating a sine wave signal upon receiving operating frequency information may be provided at the time of implementation. In this case, the signal generated by the sine wave signal generator may be used as reference data.
  • the second home appliance 200 is a home appliance having a different configuration from that of the first home appliance 100, but the second home appliance 200 may be a home appliance having the same configuration as the first home appliance 100 at the time of implementation.
  • the second home appliance generates reference data regarding the noise of the internal noise source and provides the generated reference data to the first home appliance or the server so that the first home appliance or the server may perform preprocessing to remove noise included in voice data including user's speech by using the reference data.
  • FIG. 5 is a block diagram illustrating a simple configuration of a server according to an embodiment of the disclosure.
  • the server 300 may include a communication device 310, a memory 320, and a processor 330.
  • the server 300 refers to a computer or a program that provides information or services to a client through a network.
  • the communication device 310 may be connected to an external device and may receive various data from the external device. Specifically, the communication device 310 may be connected to an external device through a local area network (LAN) and the Internet, as well as through a universal serial bus (USB) port or wireless communication (Wi-Fi 802.11a/b/g/n, NFC, Bluetooth) port.
  • the external device may be a PC, a notebook, a smartphone, a server, or the like.
  • the communication device 310 may be connected to the first home appliance 100 or the second home appliance 200 in the voice recognition system 1000.
  • the communication device 310 may perform communication with the first home appliance 100 or the second home appliance 200 in the voice recognition system 1000 through Wi-Fi communication.
  • the method for the server 300 to communicate with the first home appliance 100 or the second home appliance 200 is not limited to the example described above.
  • the memory 320 stores various data for an overall operation of the server 300 such as a program for processing or controlling of the processor 330. Specifically, the memory 320 may store a plurality of application programs run in the server 300 and data and instructions for the operation of the server 300.
  • the memory 320 may be accessed by the processor 330, and data reading/writing/modifying/deleting/updating may be performed by the processor 330.
  • the memory 320 may be implemented not only as a storage medium in the server 300, but also as an external storage medium, a removable disk including a USB memory, a web server through a network, and the like.
  • the memory 320 may store device information of a plurality of home appliances. Specifically, the memory 320 may store device information of a plurality of home appliances connected to the server 300 or a plurality of home appliances having a history of connection to the server.
  • the device information of the home appliance may include serial number or universally unique identifier (UUID) information, and the information included is not limited to the example described above.
  • the memory 320 may store a noise path estimation filter required to generate noise data.
  • the memory 320 may store an artificial intelligence model that generates a noise path estimation filter. Meanwhile, a specific operation of generating a noise path estimation filter using an artificial intelligence model will be described later with reference to FIG. 11.
  • the processor 330 controls the server 300. Specifically, the processor 330 may control each component of the server 300 according to a user's control command.
  • the processor 330 may perform preprocessing on the voice data obtained by the first home appliance 100 in place of the first home appliance 100. Specifically, the processor 330 may receive first voice data corresponding to sound in the vicinity the first home appliance 100 from the first home appliance 100. In addition, preprocessing may be performed on the received first voice data.
  • the processor 330 may perform preprocessing on the received first voice data using information on the noise produced in the second home appliance 200.
  • the processor 330 may receive information on the second home appliance 200 connected to the first home appliance 100 from the first home appliance 100. Specifically, the processor 330 may receive information on the second home appliance 200 which may be connected to the first home appliance 100 through short-range wireless communication or the second home appliance 200 connected to an access point (AP) to which the first home appliance 100 is connected, from the first home appliance 100.
  • AP access point
  • the processor 330 may be connected to the second home appliance 200. Specifically, the processor 330 may be connected to the second home appliance 200 using the information of the second home appliance 200 received from the first home appliance 100.
  • the processor 330 may control the communication device 310 to request voice data from the first home appliance 100 and the second home appliance 200. Specifically, the processor 330 may control the communication device 310 to transmit a recording request signal for requesting generation and transmission of voice data to the first home appliance 100 and the second home appliance 200.
  • the recording request signal may include information on a predetermined time and information on a predetermined time length.
  • the information included in the recording request signal is not limited to the example described above.
  • the processor 330 may receive voice data generated based on the request. Specifically, the processor 330 may receive first voice data obtained based on the recording request signal from the first home appliance 100. The processor 330 may receive second voice data obtained based on the recording request signal from the second home appliance 200.
  • the processor 330 may generate noise data which is sound data corresponding to pure noise of a noise source in the second home appliance 200 using the received second voice data.
  • the processor 330 may receive other types of reference data regarding the noise in addition to the second voice data generated by detecting the noise produced in the second home appliance 200.
  • the processor 330 may receive reference data obtained through the accelerometer of the second home appliance 200 or reference data obtained through a control command input to the second home appliance 200.
  • the reference data may be generated based on noise source information including a magnitude and a phase of vibration produced from the noise source, a magnitude and a phase of noise produced from the noise source, and main frequency information.
  • the processor 330 may generate noise data which is sound data corresponding to pure noise of the noise source in the second home appliance 200 using the received reference data.
  • the processor 330 may use a noise path estimation filter to generate noise data. Meanwhile, a specific operation of generating noise data using the noise path estimation filter will be described later with reference to FIGS. 6 through 10.
  • the processor 330 may also be implemented to receive the noise data itself generated in the second home appliance 200, without receiving the second voice data or the reference data from the second home appliance 200.
  • the processor 330 may perform preprocessing on the first voice data using the noise data. Specifically, the processor 330 may perform preprocessing on the first voice data by removing a component corresponding to the noise of the noise source from the first voice data using the noise data generated through the noise path estimation filter.
  • the processor 330 may transmit the preprocessed first voice data to the first home appliance 100.
  • the first home appliance 100 may perform voice recognition using the preprocessed first voice data received from the server 300.
  • the server 300 receives the first voice data from the first home appliance which is to perform the voice recognition function, obtains the second voice data or reference data regarding the noise from the second home appliance that produces large magnitude noise, and performs preprocessing on the first voice data using the obtained second voice data or reference data, whereby the noise included in the voice data may be accurately removed even if the noise is large in magnitude.
  • FIGS. 6 and 7 are diagrams illustrating a noise removing method according to a first embodiment.
  • FIG. 6 is a simple block diagram illustrating a noise removing method according to the first embodiment.
  • noise data y is generated using reference data r, voice data d, and a noise estimation algorithm, and noise of voice data d is removed using the generated noise data y.
  • the reference data r may be at least one of the second voice data obtained through the microphone 210 of the second home appliance 200, the acceleration information of the accelerometer 250 of the second home appliance 200, or reference data generated using a driving frequency checked from the control command.
  • the voice data d may correspond to the first voice data obtained by the microphone 110 of the first home appliance 100.
  • the processor 130 of the first home appliance 100 or the processor 330 of the server 300 may generate the noise data y using the noise estimation algorithm and the reference data r. Specifically, the processors 130 and 330 may extract the noise data y, which is sound data corresponding to a noise of the source noise, from the reference data r using the noise path estimation filter information included in the noise estimation algorithm.
  • the noise path estimation filter may be implemented as a finite impulse response (FIR) filter or an infinite impulse response (IIR) filter in a time domain.
  • the noise path estimation filter may be implemented in the form of a transfer function predetermined for each frequency band in a frequency domain.
  • the noise path estimation filter may have a linear structure as in the example described above, but is not limited thereto and may have a non-linear structure.
  • the noise path estimation filter may be fixed and used as one noise path estimation filter, and in case that a plurality of noise path estimation filter information are stored in advance, one of the noise path estimation filters may be selected and used to generate the noise data y according to situations.
  • the processors 130 and 330 may perform preprocessing on the voice data d by removing a component corresponding to the noise of the noise source included in the voice data d using the generated noise data y.
  • the processor 330 of the server 300 may perform preprocessing and transmit noise-removed voice data e to the first home appliance 100.
  • the processor 130 of the first home appliance 100 may perform voice recognition using the voice data e from which noise was removed by performing preprocessing.
  • the processor 130 or 330 may update the method of generating the noise data y using the voice data d so that the accurate noise data y may be generated even when noise of the noise source is changed or a surrounding environment is changed.
  • the processors 130 and 330 may update the noise path estimation filter using the voice data d including the noise. Details thereof will be described below with reference to FIG. 7.
  • FIG. 7 is a block diagram illustrating a method of removing noise in a frequency domain according to the first embodiment.
  • a noise estimation algorithm using a noise path estimation filter H in the frequency domain and performing updating using voice data d including noise may be identified.
  • the processors 130 and 330 may convert the reference data r into a frequency domain using fast Fourier transform (FFT).
  • FFT fast Fourier transform
  • the processors 130 and 330 may convert the noise data Y into the time domain using inverse fast Fourier transform (IFFT).
  • IFFT inverse fast Fourier transform
  • the processors 130 and 330 may use the converted noise data y to remove noise of the voice data d.
  • the processors 130 and 330 may update the noise path estimation filter H using the voice data d with noise mixed therein. Specifically, the processor 130 or 330 may update the noise path estimation filter H using a correlation between the voice data D converted into the frequency domain through the FFT from the voice data d including noise and the converted reference data R.
  • a k-1th noise path estimation filter is H (k-1)
  • voice data including subsequent kth noise is d( k )
  • kth reference data is r( k )
  • voice data converted into kth frequency domain is D( k )
  • reference data converted into the kth frequency domain is R( k ).
  • G RR(k) is (R (k) H is a Hermitian matrix of reference data (R (k) )) and .
  • a constant determined according to systems may be used, or the value may be variably used for stability of the algorithm.
  • the kth value may be .
  • the processors 130 and 330 When the k+1th reference data (r (k+1) and voice data (d (k+1) ) are obtained, the processors 130 and 330 generate noise data y (k+1) from next reference data using an updated new noise path estimation filter H (k) and remove noise from the voice data d (k+1) using the generated noise data y (k+1) .
  • the method of updating the noise path estimation filter H is not limited to the example described above.
  • updating is performed each time the voice data including noise and the reference data are obtained, but it may be implemented such that updating is performed when voice data including a predetermined number of noise and reference data are obtained. In this case, updating may be performed using the voice data including the predetermined number of noise and the reference data together.
  • FIGS. 8 through 10 are diagrams illustrating a noise removing method according to a second embodiment.
  • FIG. 8 is a simple block diagram illustrating a noise removing method according to a second embodiment.
  • noise data y is generated using the reference data r, the noise-removed voice data e, and the noise estimation algorithm, and noise of the voice data d is removed using the generated noise data y.
  • the reference data r may be at least one of the second voice data obtained through the microphone 210 of the second home appliance 200 and the reference data generated using a driving frequency identified from the control command or the acceleration information of the accelerometer 250 of the second home appliance 200.
  • the voice data d may correspond to the first voice data obtained from the microphone 110 of the first home appliance 100.
  • the processors 130 and 330 may extract noise data y, which is sound data corresponding to noise of the noise source, from the reference data r using the noise path estimation filter information included in the noise estimation algorithm.
  • noise data y which is sound data corresponding to noise of the noise source
  • the configuration of the noise estimation filter is the same as that of FIG. 7, and thus, redundant description thereof will be omitted.
  • the processors 130 and 330 may perform preprocessing on the voice data d by removing a component corresponding to the noise of the noise source included in the voice data d using the generated noise data y.
  • the processor 330 of the server 300 may perform preprocessing to transmit the noise-removed voice data e to the first home appliance 100.
  • the processor 130 of the first home appliance 100 may perform voice recognition using the noise-removed voice data e by performing preprocessing.
  • the processors 130 and 330 may update the method of generating noise data using the noise-removed voice data, rather than the voice data including the noise. Details thereof will be described below with reference to FIGS. 9 and 10.
  • FIG. 9 is a block diagram illustrating a method of removing noise in a time domain according to the second embodiment.
  • a noise estimation algorithm using a noise path estimation filter which is an FIR filter, in the time domain and performing updating using noise-removed voice data may be identified.
  • the processors 130 and 330 may update the noise path estimation filter h using the noise-removed voice data e. Specifically, after performing preprocessing, the processors 130 and 330 may update the noise path estimation filter h using a correlation between the noise-removed voice data e and the reference data r.
  • a k-1th noise path estimation filter is h (k-1)
  • kth noise-removed voice data is e (k)
  • kth reference data is r (k)
  • ⁇ a constant determined according to systems may be used or ⁇ may be variably used for stability of the algorithm.
  • ( ⁇ and ⁇ are constants).
  • the processors 130 and 330 When the processors 130 and 330 obtain next reference data r' and voice data d', the processors 130 and 330 generate noise data y' from the next reference data using an updated new noise path estimation filter h' and remove noise of the voice data d' using the noise data y', thus obtaining noise-removed voice data e'.
  • the method of updating the noise path estimation filter h in the time domain is not limited to the example described above.
  • updating is performed each time the noise-removed voice data and the reference data are obtained, but it may also be implemented such that updating is performed when the voice data from which a predetermined number of noise was removed and reference data are obtained. In this case, updating may be performed using the voice data from which the predetermined number of noise was removed and the reference data together.
  • the noise path estimation filter h may be a filter implemented in the form of a predetermined transfer function for each frequency band in the frequency domain, rather than an FIR filter in the time domain, and the noise path estimation filter h may be updated using the noise-removed voice signal e. Details thereof will be described later with reference to FIG. 10.
  • FIG. 10 is a block diagram illustrating a method of removing noise in a frequency domain according to the second embodiment.
  • a noise estimation algorithm using a noise path estimation filter in a frequency domain and performing updating using noise-removed voice data may be identified.
  • the processors 130 and 330 may convert the reference data r into a frequency domain using the FFT.
  • the processors 130 and 330 may convert the noise data Y into a time domain using IFFT.
  • the processor 130 may use the converted noise data y to remove noise of the voice data d.
  • the processors 130 and 330 may update the noise path estimation filter H in the frequency domain using the noise-removed voice data e. Specifically, the processors 130 and 330 may update the noise path estimation filter H using a correlation between the voice data E converted into the frequency domain through the FFT from the voice data e and the converted reference data R.
  • k-1th noise path estimation filter H (k-1)
  • kth converted noise-removed voice data E (k)
  • kth reference data converted into the frequency domain R (k)
  • ⁇ a constant determined according to systems may be used or ⁇ may be variably used for stability of the algorithm.
  • the kth ⁇ may be .
  • the processors 130 and 330 may generate noise data y (k+1) from the next reference data using the updated new noise path estimation filter H (k) and noise may be removed from the voice data d (k+1) using the generated noise data y (k+1) .
  • the method of updating the noise path estimation filter H is not limited to the example described above.
  • updating is performed each time the noise-removed voice data and the reference data are obtained, but updating may also be performed when a predetermined number of noise-removed voice data and reference data are obtained. In this case, updating may be performed using the predetermined number of noise-removed voice data and the reference data together.
  • FIGS. 8 through 10 it is illustrated and described that the processors 130 and 330 update the noise path estimation filter after performing preprocessing on the voice data d including noise, but it may also be implemented such that the noise path estimation filter is first updated and preprocessing is performed on the voice data d including noise.
  • FIGS. 11 through 13 are diagrams illustrating a noise removing method according to a third embodiment.
  • the information on the noise path estimation filter may be previously stored in the first home appliance 100 or the server 300 and used to generate noise data as described above.
  • the noise path estimation filter may not be pre-stored in the manufacturing stage of the first home appliance 100 but may be generated through an artificial intelligence model after the first home appliance 100 is installed in a home.
  • the server 300 may generate a noise path estimation filter suitable for a home appliance connected through an artificial intelligence model.
  • the first home appliance 100 or the server 300 may perform noise removal using the generated noise path estimation filter.
  • the first home appliance 100 obtains a noise path estimation filter from an external device 400 including an artificial intelligence model.
  • FIG. 11 is a sequence diagram illustrating a method of obtaining a noise path estimation filter by the first home appliance 100.
  • the first home appliance 100 may obtain voice data and reference data at a time when there is no user's speech (S1110). Specifically, in order to generate a noise path estimation filter, voice data and reference data in which there is no user's spoken voice and only noise of a noise source are detected are required. Therefore, when voice data determined not to have a user's speech is determined as a result of performing voice recognition, reference data obtained at the same time point as that of corresponding voice data may be checked.
  • the first home appliance 100 may transmit the obtained voice data and reference data to the external device 400. Specifically, the first home appliance 100 may transmit the obtained voice data and reference data as a signal in a time domain or convert it into a frequency domain and transmit the same to the external device 400.
  • the external device 400 or the first home appliance 100 may request reference data at the corresponding time from the second home appliance 200 and obtain the reference data.
  • the noise of the noise source of the second home appliance 200 may be different according to a change in an operation mode of the second home appliance 200 or an environment. Therefore, it is needed to generate a noise path estimation filter to be applied to each case.
  • the first home appliance 100 may transmit information on an operation mode of the second home appliance 200 or information on a surrounding environment together, when transmitting information to the external device 400.
  • a rotation speed of the motor included in the washing machine may vary according to an operation mode. Therefore, a magnitude or characteristics of noise of the noise source (motor) may vary according to the operation mode.
  • the information on each operation mode of the second home appliance 200 and the voice data and the reference data obtained when the second home appliance 200 operates in each operation mode may be transmitted together, so that the external device 400 may generate a noise path estimation filter applicable to each operation mode.
  • the external device 400 may calculate a noise path estimation filter using the received voice data and reference data (S1130). Specifically, the external device 400 may obtain a noise path estimation filter using an artificial intelligence model that receives voice data and reference data and outputs a noise path estimation filter corresponding thereto.
  • the artificial intelligence model may be a linear regression model.
  • the external device 400 may transmit the calculated noise path estimation filter information to the first home appliance 100 (S1140). Meanwhile, when the second home appliance 200 is implemented in such a manner as to obtain second voice data and generate noise data using the obtained second voice data, the noise path estimation filter may be transmitted to the second home appliance 200.
  • the first home appliance 100 may store information of the received noise path estimation filter (S1150).
  • the first home appliance 100 may generate noise data using the obtained reference data and noise path estimation filter information, and remove noise of the voice data including user's spoken voice using the generated noise data.
  • the first home appliance 100 may perform the voice recognition function using the noise-removed voice data.
  • the external device is a device other than the voice recognition system but it may be implemented as a server in the voice recognition system.
  • the voice data and the reference data are transmitted to the external device and the noise path estimation filter generated by an artificial intelligence mode is received from the external device, but the first home appliance may generate a noise path estimation filter using the previously stored artificial intelligence model when implemented.
  • a separate device in the home appliance distinguished from the processor may generate a noise path estimation filter using the pre-stored artificial intelligence model, and the processor may use the generated noise path estimation filter.
  • the method of generating the noise path estimation filter is not limited to the example described above.
  • FIG. 12 is a block diagram illustrating a method of removing noise in a time domain according to the third embodiment.
  • a noise removing method using a noise path estimation filter h in a time domain may be identified.
  • the processor 130 of the first home appliance 100 may remove noise using the noise path estimation filter h received from the external device 400.
  • the processor 330 of the server 300 may perform noise removal using the generated noise path estimation filter h.
  • the processors 130 and 330 may obtain the noise-removed voice data e by removing the noise of the voice data d using the noise data y.
  • FIG. 13 is a block diagram illustrating a method of removing noise in a frequency domain according to the third embodiment.
  • a noise removing method using a noise path estimation filter H in a frequency domain may be identified.
  • the processors 130 and 330 may convert the reference data r into a frequency domain using FFT.
  • the processors 130 and 330 may remove noise of the converted voice data D through the FFT using the noise data Y. In addition, the processors 130 and 330 may obtain the noise-removed voice data e by converting the noise-removed voice data E into a time domain using IFFT.
  • FIGS. 12 and 13 it is illustrated and described that updating is not performed on the noise path estimation filter, but updating may be performed on the noise path estimation filter according to the updating method described above when implemented.
  • FIGS. 14 through 16 are sequence diagrams illustrating a voice recognition method performed without intervention of a server.
  • FIG. 14 is a sequence diagram illustrating a voice recognition method performed without intervention of a server according to the first embodiment of the disclosure.
  • the first home appliance 100 may transmit a recording request signal to the second home appliance 200 (S1410).
  • the recording request signal may include information on a predetermined time and information on a predetermined time length.
  • the first home appliance 100 may obtain first voice data corresponding to the recording request signal (S1420). Specifically, the first home appliance 100 may obtain first voice data based on predetermined time information and predetermined time length information included in the recording request signal.
  • the second home appliance 200 may obtain second voice data corresponding to the recording request signal (S1425). Specifically, the second home appliance 200 may obtain second voice data based on the predetermined time information and the predetermined time length information included in the recording request signal.
  • the second home appliance 200 may transmit the obtained second voice data to the first home appliance 100 (S1430).
  • the first home appliance 100 may generate noise data using the received second voice data (S1440). Specifically, the first home appliance 100 may generate noise data which is sound data corresponding to pure noise of a noise source in the second home appliance 200 by applying the received second voice data to the noise path estimation filter.
  • the first home appliance 100 may perform preprocessing on the first voice data using the noise data (S1450).
  • the first home appliance 100 may update the noise path estimation filter (S1460). Specifically, the first home appliance100 may update the noise path estimation filter using the second voice data and the first voice data or may update the noise path estimation filter using the second voice data and the preprocessed first voice data. Meanwhile, the method of updating the noise path estimation filter has been described above, and thus, a redundant description thereof will be omitted.
  • the first home appliance 100 may perform voice recognition using the preprocessed first voice data (S1470).
  • FIG. 15 is a sequence diagram illustrating a voice recognition method performed without intervention of a server according to the second embodiment of the disclosure.
  • the first home appliance 100 may transmit a recording request signal to the second home appliance 200 (S1510).
  • the first home appliance 100 may obtain first voice data corresponding to the recording request signal (S1520).
  • the second home appliance 200 may obtain second voice data corresponding to the recording request signal (S1525).
  • the second home appliance 200 may transmit the obtained second voice data to the first home appliance 100 (S1530).
  • the first home appliance 100 may update the noise path estimation filter before performing the preprocessing on the first voice data (S1540).
  • the first home appliance 100 may generate noise data using the received second voice data (S1550). Specifically, the first home appliance 100 may generate noise data which is sound data corresponding to pure noise of a noise source in the second home appliance 200 by applying the received second voice data to the updated noise path estimation filter.
  • the first home appliance 100 may perform preprocessing on the first voice data using the noise data (S1560).
  • the first home appliance 100 may perform voice recognition using the preprocessed first voice data (S1570).
  • FIG. 16 is a sequence diagram illustrating a voice recognition method performed without intervention of a server according to a third embodiment of the disclosure.
  • the first home appliance 100 may transmit a recording request signal to the second home appliance 200 (S1610).
  • the first home appliance 100 may obtain first voice data corresponding to the recording request signal. Specifically, the first home appliance 100 may obtain first voice data based on predetermined time information and predetermined time length information included in the recording request signal.
  • the second home appliance 200 may obtain second voice data corresponding to the recording request signal (S1625).
  • the second home appliance 200 may generate noise data using the second voice data (S1630). Specifically, the second home appliance 200 may generate noise data which is sound data corresponding to pure noise of the noise source in the second home appliance 200 by applying the second voice data to the noise path estimation filter.
  • the noise path estimation filter may be a filter previously stored in the second home appliance 200 or generated by an artificial intelligence model of the server 300 or the external device 400 and transmitted to the second home appliance 200.
  • the second home appliance 200 may transmit the generated noise data to the first home appliance 100 (S1640).
  • the first home appliance 100 may perform preprocessing on the first voice data using the received noise data (S1650).
  • the first home appliance 100 may perform voice recognition using the preprocessed first voice data (S1660).
  • the first home appliance 100 may estimate noise using information on the noise produced in the other home appliance without intervention of the server in various manners, and reduce the estimated noise.
  • FIGS. 17 and 18 are sequence diagrams illustrating a voice recognition method performed according to intervention of a server.
  • FIG. 17 is a sequence diagram illustrating a voice recognition method performed according to intervention of a server according to the first embodiment of the disclosure.
  • the server 300 may transmit a recording request signal to the first home appliance 100 (S1710).
  • the server 300 may transmit a recording request signal to the second home appliance 200 (S1715).
  • the recording request signal may include information on a predetermined time and information on a predetermined time length.
  • the first home appliance 100 may obtain first voice data corresponding to the recording request signal (S1720). Specifically, the first home appliance 100 may obtain first voice data based on the predetermined time information and the predetermined time length information included in the recording request signal.
  • the second home appliance 200 may obtain second voice data corresponding to the recording request signal (S1725). Specifically, the second home appliance 200 may obtain second voice data based on the predetermined time information and the predetermined time length information included in the recording request signal.
  • the first home appliance 100 may transmit the obtained first voice data to the server 300 (S1730).
  • the second home appliance 200 may transmit the obtained second voice data to the server 300 (S1735).
  • the server 300 may generate noise data using the received second voice data (S1740). Specifically, the server 300 may generate noise data which is sound data corresponding to pure noise of the noise source in the second home appliance 200 by applying the received second voice data to the noise path estimation filter.
  • the server 300 may perform preprocessing on the first voice data using the noise data (S1750).
  • the server 300 may transmit the preprocessed first voice data to the first home appliance 100 (S1760).
  • the server 300 may update the noise path estimation filter (S1770). Specifically, the server 300 may update the noise path estimation filter using the second voice data and the first voice data or update the noise path estimation filter using the second voice data and the preprocessed first voice data. Meanwhile, the method of updating the noise path estimation filter has been described above, and thus, a redundant description thereof will be omitted.
  • the first home appliance 100 may perform voice recognition using the preprocessed first voice data (S1780).
  • FIG. 18 is a sequence diagram illustrating a voice recognition method performed under intervention of a server according to the second embodiment of the disclosure.
  • the server 300 may transmit a recording request signal to the first home appliance 100 (S1810).
  • the server 300 may transmit a recording request signal to the second home appliance 200 (S1815).
  • the first home appliance 100 may obtain first voice data corresponding to the recording request signal (S1820).
  • the second home appliance 200 may obtain second voice data based on the recording request signal (S1825).
  • the first home appliance 100 may transmit the obtained first voice data to the server 300 (S1830).
  • the second home appliance 200 may transmit the obtained second voice data to the server 300 (S1835).
  • the server 300 may update the noise path estimation filter before performing the preprocessing on the first voice data (S1840).
  • the operation of updating the noise path estimation filter before performing the preprocessing has been described above with reference to FIG. 15, and thus, a redundant description thereof will be omitted.
  • the server 300 may generate noise data using the received second voice data (S1850). Specifically, the server 300 may generate noise data corresponding to pure noise of the noise source in the second home appliance 200 by applying the received second voice data to the updated noise path estimation filter.
  • the server 300 may perform preprocessing on the first voice data using the noise data (S1860).
  • the server 300 may transmit the preprocessed first voice data to the first home appliance 100 (S1870).
  • the first home appliance 100 may perform voice recognition using the preprocessed first voice data (S1880).
  • the server may estimate the noise using the information on the noise produced in the second home appliance in various manners and reduce the estimated noise from the voice data of the first home appliance.
  • the server 300 is connected to two home appliances but the server may be connected to two or more home appliances and estimate noise using information on the noise produced in the two or more home appliances when implemented.
  • FIG. 19 is a flowchart illustrating a voice recognition method of a first home appliance according to an embodiment of the disclosure.
  • first voice data is obtained through a microphone of a first home appliance (S1910).
  • a second home appliance which may be connected to the first home appliance through short-range wireless communication or a second home appliance connected to an access point AP to which the first home appliance is connected may be detected.
  • the first home appliance may request transmission of voice data from the second home appliance.
  • a recording request signal for requesting generation and transmission of second voice data which is voice data regarding noise produced in the second home appliance, may be transmitted to the second home appliance.
  • the recording request signal may include information on a predetermined time or information on a predetermined time length. Meanwhile, the information that may be included in the recording request signal is not limited to the example described above.
  • Second voice data is received from the second home appliance (S1920). Specifically, the second voice data generated based on the predetermined time information and the predetermined time length information included in the recording request signal may be received from the second home appliance.
  • First voice data may be obtained based on the request transmitted to the second home appliance. Specifically, first voice data may be obtained through a microphone of the first home appliance based on the predetermined time information and the predetermined time length information included in the recording request signal transmitted to the second home appliance.
  • Noise data is generated using the received second voice data (S1930).
  • the noise data refers to sound data corresponding to noise of the noise source.
  • the noise data may be obtained by extracting only a component corresponding to the noise of the noise source from the second voice data or filtering remaining components other than the component corresponding to the noise of the noise source.
  • the noise data may be generated by filtering the remaining components other than the component corresponding to the noise source included in the second voice data using at least one of a finite impulse response (FIR) filter or an infinite impulse response (IIR) filter in the time domain.
  • FIR finite impulse response
  • IIR infinite impulse response
  • the noise data may extract a component corresponding to the noise of the noise source from the second voice data using a transfer function previously determined for each frequency band on the frequency domain.
  • the information on the above-described filter or transfer function may be pre-stored in a home appliance at a manufacturing stage and used to generate noise data, but is not limited thereto.
  • information on a filter or a transfer function may be received through communication with the external device, and noise data may be generated using the information on the received filter or transfer function.
  • the information on the filter or the transfer function received from the external device may be information obtained using an artificial intelligence model included in the external device.
  • Signal processing is performed on the first voice data using the generated noise data (S1940). Specifically, signal processing for removing a component corresponding to noise of a noise source included in the first voice data may be performed using the noise data.
  • the noise source of the second home appliance may be a motor disposed in a housing of the second home appliance to perform a predetermined function of the second home appliance.
  • the second voice data may be voice data obtained from a microphone of the second home appliance.
  • the noise data may be motor noise data of the motor of the second home appliance, and preprocessing on the first voice data may be performed by removing the motor noise from the first voice data.
  • a signal processing method for the first voice data may be changed according to whether noise is produced in the second home appliance.
  • the preprocessing method regarding the first voice data may be changed depending on whether the motor of the second home appliance is driven.
  • preprocessing on the first voice data may be performed using the second voice data according to the method described above.
  • voice recognition may be performed directly using the first voice data without using the second voice data or preprocessing may be performed on the first voice data according to the related art method.
  • the method of preprocessing on the first voice data may be changed according to whether the noise source is driven.
  • Voice recognition is performed using the signal-processed first voice data (S1950).
  • the method of generating noise data may be updated using at least one of the first voice data including noise, the signal-processed first voice data, or the second voice data to generate accurate noise data even when noise of the noise source is changed or a surrounding environment is changed.
  • the method of generating noise data may be updated using a correlation between at least one of the first voice data and the signal-processed first voice data and the second voice data.
  • the operation of updating the method of generating noise data may be performed after signal processing is performed on the first voice data as described above, but alternatively, the operation of updating the method of generating noise data may be first performed before signal processing is performed on the first voice data.
  • the voice data of the noise is obtained from another home appliance which produces large magnitude noise and the preprocessing is performed on the obtained noise using the voice data, whereby the noise included in the voice data obtained through the microphone may be accurately removed even though a magnitude of the noise produced from outside is large.
  • the voice recognition method as shown in FIG. 19 may also be executed on a home appliance having the configuration of FIG. 2 or 3 or may be executed on a home appliance having another configuration.
  • the voice recognition method as described above may be implemented by at least one executable program for executing the voice recognition method as described above, and such an executable program may be stored in a non-transitory readable medium.
  • the non-transitory readable medium refers to a medium which stores data semi-permanently and is readable by a device, not a medium storing data for a short time such as a register, a cache, a memory, and the like.
  • various applications or programs may be stored and provided in a non-transitory readable medium such as a CD, a DVD, a hard disk, a Blu-ray disk, a USB, a memory card, a ROM, or the like.
  • FIG. 20 is a flowchart illustrating a voice recognition method of a second home appliance according to an embodiment of the disclosure.
  • a voice data request signal may be received (S2010).
  • a recording request signal requesting generation and transmission of second voice data which is voice data of the noise produced in the second home appliance may be received from a server or a first home appliance connected to the second home appliance.
  • Second voice data may be obtained through a microphone of the second home appliance (S2020). Specifically, noise may be detected using the microphone of the second home appliance and the second voice data may be obtained based on the received recording request signal. More specifically, the second voice data may be obtained based on information on a predetermined time and predetermined time length information included in the received recording request signal.
  • the obtained second voice data may be transmitted (S2030). Specifically, the obtained second voice data may be transmitted to the server or the first home appliance connected to the second home appliance.
  • the second voice data may not be transmitted to the server or the first home appliance connected to the second home appliance, and noise data may be generated using the second voice data and the generated noise data may be transmitted to the first home appliance or the server.
  • the home appliance or the server may accurately remove the noise included in the voice data including user's speech.
  • the voice recognition method as shown in FIG. 20 may be executed on a home appliance having the configuration of FIG. 4 or may be executed on a home appliance having another configuration.
  • the voice recognition method as described above may be implemented by at least one executable program for executing the voice recognition method as described above, and such an executable program may be stored in a non-transitory readable medium.
  • FIG. 21 is a flowchart illustrating a voice recognition method of a server according to an embodiment of the disclosure.
  • voice data may be requested from a first home appliance and a second home appliance (S2110).
  • a recording request signal for requesting generation and transmission of voice data may be transmitted to the first home appliance and the second home appliance.
  • the recording request signal may include information on a predetermined time and information on a predetermined time length.
  • the information included in the recording request signal is not limited to the examples described above.
  • First voice data and second voice data may be received (S2120). Specifically, the first voice data obtained based on the recording request signal may be received from the first home appliance. The second voice data obtained based on the recording request signal may be received from the second home appliance.
  • Noise data may be generated using the second voice data (S2130).
  • the noise data refers to sound data corresponding to pure noise of a noise source in the second home appliance.
  • Signal processing may be performed on the first voice data using the noise data (S2140). Specifically, signal processing for removing a component corresponding to the noise of the noise source included in the first voice data may be performed using the noise data.
  • the signal-processed first voice data may be transmitted to the first home appliance (S2150).
  • the first voice data is received from the first home appliance which is to perform the voice recognition function
  • the second voice data regarding the noise or reference data is obtained from the second home appliance that produces large magnitude noise
  • preprocessing is performed on the first voice data using the obtained second voice data or reference data, so that the noise included in the voice data may be accurately removed even if the noise is large in magnitude.
  • the voice recognition method as shown in FIG. 21 may be executed on a server having the configuration of FIG. 5 or may be executed on a server having another configuration.
  • the voice recognition method as described above may be implemented with at least one executable program for executing the voice recognition method as described above, and such an executable program may be stored in a non-transitory readable medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Selective Calling Equipment (AREA)
  • Telephonic Communication Services (AREA)

Abstract

La présente invention concerne un appareil ménager. L'appareil ménager comprend un dispositif de communication configuré pour communiquer avec un autre appareil ménager, un microphone configuré pour recevoir une voix d'un utilisateur, et un processeur configuré pour effectuer un traitement de signal sur des premières données vocales obtenues à partir du microphone et effectuer une reconnaissance vocale au moyen des premières données vocales soumises au traitement de signal, le processeur générant des données de bruit au moyen de deuxièmes données vocales reçues depuis l'autre appareil ménager et effectuant le traitement de signal sur les premières données vocales au moyen des données de bruit générées.
PCT/KR2019/018113 2018-12-27 2019-12-19 Appareil ménager et procédé de reconnaissance vocale associé WO2020138844A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AU2019416316A AU2019416316A1 (en) 2018-12-27 2019-12-19 Home appliance and method for voice recognition thereof
EP19904108.8A EP3844750A4 (fr) 2018-12-27 2019-12-19 Appareil ménager et procédé de reconnaissance vocale associé
CN201980086395.2A CN113287168A (zh) 2018-12-27 2019-12-19 家用电器及其语音识别方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2018-0170469 2018-12-27
KR1020180170469A KR102570384B1 (ko) 2018-12-27 2018-12-27 가전기기 및 이의 음성 인식 방법

Publications (1)

Publication Number Publication Date
WO2020138844A1 true WO2020138844A1 (fr) 2020-07-02

Family

ID=71121828

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2019/018113 WO2020138844A1 (fr) 2018-12-27 2019-12-19 Appareil ménager et procédé de reconnaissance vocale associé

Country Status (6)

Country Link
US (1) US11404054B2 (fr)
EP (1) EP3844750A4 (fr)
KR (1) KR102570384B1 (fr)
CN (1) CN113287168A (fr)
AU (1) AU2019416316A1 (fr)
WO (1) WO2020138844A1 (fr)

Families Citing this family (59)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10264030B2 (en) 2016-02-22 2019-04-16 Sonos, Inc. Networked microphone device control
US9947316B2 (en) 2016-02-22 2018-04-17 Sonos, Inc. Voice control of a media playback system
US10095470B2 (en) 2016-02-22 2018-10-09 Sonos, Inc. Audio response playback
US9965247B2 (en) 2016-02-22 2018-05-08 Sonos, Inc. Voice controlled media playback system based on user profile
US9820039B2 (en) 2016-02-22 2017-11-14 Sonos, Inc. Default playback devices
US9811314B2 (en) 2016-02-22 2017-11-07 Sonos, Inc. Metadata exchange involving a networked playback system and a networked microphone system
US9978390B2 (en) 2016-06-09 2018-05-22 Sonos, Inc. Dynamic player selection for audio signal processing
US10134399B2 (en) 2016-07-15 2018-11-20 Sonos, Inc. Contextualization of voice inputs
US10115400B2 (en) 2016-08-05 2018-10-30 Sonos, Inc. Multiple voice services
US9942678B1 (en) 2016-09-27 2018-04-10 Sonos, Inc. Audio playback settings for voice interaction
US10181323B2 (en) 2016-10-19 2019-01-15 Sonos, Inc. Arbitration-based voice recognition
US10475449B2 (en) 2017-08-07 2019-11-12 Sonos, Inc. Wake-word detection suppression
US10048930B1 (en) 2017-09-08 2018-08-14 Sonos, Inc. Dynamic computation of system response volume
US10531157B1 (en) * 2017-09-21 2020-01-07 Amazon Technologies, Inc. Presentation and management of audio and visual content across devices
US10446165B2 (en) 2017-09-27 2019-10-15 Sonos, Inc. Robust short-time fourier transform acoustic echo cancellation during audio playback
US10621981B2 (en) 2017-09-28 2020-04-14 Sonos, Inc. Tone interference cancellation
US10482868B2 (en) 2017-09-28 2019-11-19 Sonos, Inc. Multi-channel acoustic echo cancellation
US10466962B2 (en) 2017-09-29 2019-11-05 Sonos, Inc. Media playback system with voice assistance
WO2019152722A1 (fr) 2018-01-31 2019-08-08 Sonos, Inc. Désignation de dispositif de lecture et agencements de dispositif de microphone de réseau
US10600408B1 (en) * 2018-03-23 2020-03-24 Amazon Technologies, Inc. Content output management based on speech quality
US11175880B2 (en) 2018-05-10 2021-11-16 Sonos, Inc. Systems and methods for voice-assisted media content selection
US10959029B2 (en) 2018-05-25 2021-03-23 Sonos, Inc. Determining and adapting to changes in microphone performance of playback devices
US10681460B2 (en) 2018-06-28 2020-06-09 Sonos, Inc. Systems and methods for associating playback devices with voice assistant services
US11076035B2 (en) 2018-08-28 2021-07-27 Sonos, Inc. Do not disturb feature for audio notifications
US10461710B1 (en) 2018-08-28 2019-10-29 Sonos, Inc. Media playback system with maximum volume setting
US10587430B1 (en) 2018-09-14 2020-03-10 Sonos, Inc. Networked devices, systems, and methods for associating playback devices based on sound codes
US11315553B2 (en) * 2018-09-20 2022-04-26 Samsung Electronics Co., Ltd. Electronic device and method for providing or obtaining data for training thereof
US11024331B2 (en) 2018-09-21 2021-06-01 Sonos, Inc. Voice detection optimization using sound metadata
US11100923B2 (en) 2018-09-28 2021-08-24 Sonos, Inc. Systems and methods for selective wake word detection using neural network models
US11899519B2 (en) 2018-10-23 2024-02-13 Sonos, Inc. Multiple stage network microphone device with reduced power consumption and processing load
EP3654249A1 (fr) 2018-11-15 2020-05-20 Snips Convolutions dilatées et déclenchement efficace de mot-clé
US11183183B2 (en) 2018-12-07 2021-11-23 Sonos, Inc. Systems and methods of operating media playback systems having multiple voice assistant services
US11132989B2 (en) 2018-12-13 2021-09-28 Sonos, Inc. Networked microphone devices, systems, and methods of localized arbitration
US10602268B1 (en) 2018-12-20 2020-03-24 Sonos, Inc. Optimization of network microphone devices using noise classification
US10867604B2 (en) 2019-02-08 2020-12-15 Sonos, Inc. Devices, systems, and methods for distributed voice processing
US11315556B2 (en) 2019-02-08 2022-04-26 Sonos, Inc. Devices, systems, and methods for distributed voice processing by transmitting sound data associated with a wake word to an appropriate device for identification
EP3709194A1 (fr) 2019-03-15 2020-09-16 Spotify AB Comparaison de données basée sur un ensemble
US11120794B2 (en) 2019-05-03 2021-09-14 Sonos, Inc. Voice assistant persistence across multiple network microphone devices
US11361756B2 (en) 2019-06-12 2022-06-14 Sonos, Inc. Conditional wake word eventing based on environment
US10586540B1 (en) 2019-06-12 2020-03-10 Sonos, Inc. Network microphone device with command keyword conditioning
US11200894B2 (en) 2019-06-12 2021-12-14 Sonos, Inc. Network microphone device with command keyword eventing
US10871943B1 (en) 2019-07-31 2020-12-22 Sonos, Inc. Noise classification for event detection
US11138969B2 (en) 2019-07-31 2021-10-05 Sonos, Inc. Locally distributed keyword detection
US11138975B2 (en) 2019-07-31 2021-10-05 Sonos, Inc. Locally distributed keyword detection
US11094319B2 (en) 2019-08-30 2021-08-17 Spotify Ab Systems and methods for generating a cleaned version of ambient sound
US11189286B2 (en) 2019-10-22 2021-11-30 Sonos, Inc. VAS toggle based on device orientation
US11200900B2 (en) 2019-12-20 2021-12-14 Sonos, Inc. Offline voice control
US11562740B2 (en) 2020-01-07 2023-01-24 Sonos, Inc. Voice verification for media playback
US11308958B2 (en) 2020-02-07 2022-04-19 Sonos, Inc. Localized wakeword verification
US11308959B2 (en) 2020-02-11 2022-04-19 Spotify Ab Dynamic adjustment of wake word acceptance tolerance thresholds in voice-controlled devices
US11328722B2 (en) * 2020-02-11 2022-05-10 Spotify Ab Systems and methods for generating a singular voice audio stream
US11308962B2 (en) * 2020-05-20 2022-04-19 Sonos, Inc. Input detection windowing
US11482224B2 (en) 2020-05-20 2022-10-25 Sonos, Inc. Command keywords with input detection windowing
US11727919B2 (en) 2020-05-20 2023-08-15 Sonos, Inc. Memory allocation for keyword spotting engines
US11698771B2 (en) 2020-08-25 2023-07-11 Sonos, Inc. Vocal guidance engines for playback devices
CN112164397A (zh) * 2020-09-30 2021-01-01 广东美的厨房电器制造有限公司 噪声降低控制方法、烹饪器具和可读存储介质
US11984123B2 (en) 2020-11-12 2024-05-14 Sonos, Inc. Network device interaction by range
KR20220069611A (ko) * 2020-11-20 2022-05-27 삼성전자주식회사 전자장치 및 그 제어방법
CN112712683B (zh) * 2020-12-14 2022-06-14 珠海格力电器股份有限公司 家电设备的控制方法、系统、遥控器和服务器

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5563954A (en) * 1993-02-26 1996-10-08 Matsushita Electric Industrial Co., Ltd. Microphone apparatus
US20150287414A1 (en) * 2013-08-29 2015-10-08 Panasonic Intellectual Property Corporation Of America Method for controlling device and device control system
US20160316293A1 (en) 2015-04-21 2016-10-27 Google Inc. Sound signature database for initialization of noise reduction in recordings
US20170084286A1 (en) 2015-09-18 2017-03-23 Qualcomm Incorporated Collaborative audio processing
US20180233147A1 (en) * 2017-02-10 2018-08-16 Samsung Electronics Co., Ltd. Method and apparatus for managing voice-based interaction in internet of things network system
KR20180127100A (ko) * 2017-05-19 2018-11-28 엘지전자 주식회사 홈 어플라이언스 및 음성 인식 서버 시스템의 동작 방법
US20180350379A1 (en) * 2017-06-02 2018-12-06 Apple Inc. Multi-Channel Speech Signal Enhancement for Robust Voice Trigger Detection and Automatic Speech Recognition
US20180350357A1 (en) 2017-06-06 2018-12-06 Cypress Semiconductor Corporation System and methods for audio pattern recognition

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7617099B2 (en) * 2001-02-12 2009-11-10 FortMedia Inc. Noise suppression by two-channel tandem spectrum modification for speech signal in an automobile
KR100413797B1 (ko) 2001-08-23 2003-12-31 삼성전자주식회사 음성 신호 보상 방법 및 그 장치
US6742208B2 (en) * 2001-08-24 2004-06-01 Maytag Corporation Clothes washing machine incorporating noise reduction system
KR100446626B1 (ko) 2002-03-28 2004-09-04 삼성전자주식회사 음성신호에서 잡음을 제거하는 방법 및 장치
KR100474340B1 (ko) 2002-08-02 2005-03-10 엘지전자 주식회사 진공 청소기의 음성제어 장치 및 그 방법
US8345890B2 (en) * 2006-01-05 2013-01-01 Audience, Inc. System and method for utilizing inter-microphone level differences for speech enhancement
KR100913197B1 (ko) 2008-08-06 2009-08-24 주식회사 현대오토넷 잡음제거 장치 및 방법
JP2010217453A (ja) 2009-03-16 2010-09-30 Fujitsu Ltd 音声認識用マイクロホンシステム
JP6012621B2 (ja) * 2010-12-15 2016-10-25 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． リモートノイズ検知器を使用したノイズ削減システム
KR101832952B1 (ko) 2011-04-07 2018-02-28 엘지전자 주식회사 로봇 청소기 및 이의 제어 방법
CN102348151B (zh) 2011-09-10 2015-07-29 歌尔声学股份有限公司 噪声消除系统和方法、智能控制方法和装置、通信设备
CN103632666B (zh) * 2013-11-14 2016-09-28 华为技术有限公司 语音识别方法、语音识别设备和电子设备
KR102188090B1 (ko) * 2013-12-11 2020-12-04 엘지전자 주식회사 스마트 가전제품, 그 작동방법 및 스마트 가전제품을 이용한 음성인식 시스템
US10516941B2 (en) * 2014-06-04 2019-12-24 Cirrus Logic, Inc. Reducing instantaneous wind noise
JP2016042132A (ja) 2014-08-18 2016-03-31 ソニー株式会社 音声処理装置、音声処理方法、並びにプログラム
DE112015003945T5 (de) * 2014-08-28 2017-05-11 Knowles Electronics, Llc Mehrquellen-Rauschunterdrückung
US10755726B2 (en) * 2015-01-07 2020-08-25 Google Llc Detection and suppression of keyboard transient noise in audio streams with auxiliary keybed microphone
JP6927661B2 (ja) 2015-09-30 2021-09-01 東芝ライフスタイル株式会社 電気掃除機
US9922637B2 (en) * 2016-07-11 2018-03-20 Microsoft Technology Licensing, Llc Microphone noise suppression for computing device
US10283138B2 (en) * 2016-10-03 2019-05-07 Google Llc Noise mitigation for a voice interface device
KR101898911B1 (ko) 2017-02-13 2018-10-31 주식회사 오르페오사운드웍스 인이어 마이크와 아웃이어 마이크 수음특성을 이용한 소음 제거 이어셋 및 소음 제거 방법
CN207367598U (zh) * 2017-09-04 2018-05-15 湖南海翼电子商务股份有限公司 车载电子设备
US10602268B1 (en) * 2018-12-20 2020-03-24 Sonos, Inc. Optimization of network microphone devices using noise classification

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5563954A (en) * 1993-02-26 1996-10-08 Matsushita Electric Industrial Co., Ltd. Microphone apparatus
US20150287414A1 (en) * 2013-08-29 2015-10-08 Panasonic Intellectual Property Corporation Of America Method for controlling device and device control system
US20160316293A1 (en) 2015-04-21 2016-10-27 Google Inc. Sound signature database for initialization of noise reduction in recordings
US20170084286A1 (en) 2015-09-18 2017-03-23 Qualcomm Incorporated Collaborative audio processing
US20180233147A1 (en) * 2017-02-10 2018-08-16 Samsung Electronics Co., Ltd. Method and apparatus for managing voice-based interaction in internet of things network system
KR20180127100A (ko) * 2017-05-19 2018-11-28 엘지전자 주식회사 홈 어플라이언스 및 음성 인식 서버 시스템의 동작 방법
US20180350379A1 (en) * 2017-06-02 2018-12-06 Apple Inc. Multi-Channel Speech Signal Enhancement for Robust Voice Trigger Detection and Automatic Speech Recognition
US20180350357A1 (en) 2017-06-06 2018-12-06 Cypress Semiconductor Corporation System and methods for audio pattern recognition

Also Published As

Publication number Publication date
KR102570384B1 (ko) 2023-08-25
EP3844750A1 (fr) 2021-07-07
EP3844750A4 (fr) 2021-10-27
CN113287168A (zh) 2021-08-20
AU2019416316A1 (en) 2021-05-06
US11404054B2 (en) 2022-08-02
US20200211539A1 (en) 2020-07-02
KR20200084458A (ko) 2020-07-13

Similar Documents

Publication Publication Date Title
WO2020138844A1 (fr) Appareil ménager et procédé de reconnaissance vocale associé
WO2016089009A1 (fr) Procédé et serveur cloud pour dispositif de gestion
WO2016018004A1 (fr) Procédé, appareil et système de fourniture de contenu traduit
WO2017155341A1 (fr) Appareil et procédé de commande de mise au point automatique d'un module de caméra
WO2020040452A1 (fr) Appareil de soin de vêtements et procédé de commande de celui-ci
WO2020045950A1 (fr) Procédé, dispositif et système d'utilisation sélective de multiples dispositifs de réception de données vocales pour un service intelligent
WO2014035146A2 (fr) Procédé de cryptage homomorphique et procédé de décryptage utilisant un isomorphisme d'anneau, et dispositif les utilisant
WO2019168383A1 (fr) Dispositif électronique
WO2020032604A1 (fr) Système de gestion d'accès et procédé de gestion d'accès l'utilisant
WO2011076035A1 (fr) Procédé et appareil permettant de mettre en oeuvre un accès à des fentes de cartes multiples
WO2020122593A1 (fr) Dispositif électronique servant à atténuer au moins une partie du signal reçu par une antenne et procédé de commande de signal de communication
WO2015139232A1 (fr) Procédé, système et serveur de recommandation d'application
WO2019083209A1 (fr) Sèche-linge et son procédé permettant de réaliser un programme de stérilisation
WO2020060206A1 (fr) Procédés de traitement audio, appareil, dispositif électronique et support de stockage lisible par ordinateur
WO2015093870A1 (fr) Procédé et dispositif de gestion de données
WO2019000466A1 (fr) Procédé et appareil de reconnaissance faciale, support de stockage et dispositif électronique
WO2020180027A1 (fr) Appareil ménager et son procédé de commande
WO2022158686A1 (fr) Dispositif électronique pour effectuer une inférence sur la base d'informations cryptées en utilisant un modèle d'intelligence artificielle et son procédé de fonctionnement
CN107113172A (zh) 无人机认证方法，安全通信方法及对应系统
WO2022181922A1 (fr) Dispositif de traitement de signal et dispositif d'affichage de véhicule le comprenant
WO2016195414A1 (fr) Procédé, serveur et terminal pour émettre et recevoir des données
EP3830821A1 (fr) Procédé, dispositif et système d'utilisation sélective de multiples dispositifs de réception de données vocales pour un service intelligent
EP3669023A1 (fr) Sèche-linge et son procédé permettant de réaliser un programme de stérilisation
WO2020060297A1 (fr) Dispositif électronique et procédé de commande associé
WO2018166108A1 (fr) Procédé, terminal et dispositif de déploiement de logiciel de machine virtuelle, et support d'informations lisible par ordinateur

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19904108

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019904108

Country of ref document: EP

Effective date: 20210330

ENP Entry into the national phase

Ref document number: 2019416316

Country of ref document: AU

Date of ref document: 20191219

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE