KR20240020087A - Electronic device for performing active noise cancellation and method for thereof - Google Patents

Abstract

According to one embodiment, an electronic device that performs active noise cancellation includes: a first microphone for detecting an input signal; and at least one processor coupled to the microphone. The at least one processor may check a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal has been filtered. The at least one processor may determine a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal. The at least one processor may predict a filter weight to be applied to a filter using machine learning based on the plurality of filter weights.

Description

Electronic device performing active noise cancellation and method of operation thereof {ELECTRONIC DEVICE FOR PERFORMING ACTIVE NOISE CANCELLATION AND METHOD FOR THEREOF}

One embodiment of the present disclosure relates to an electronic device and a method of operating the same for performing feed-forward active noise removal.

A wireless audio device can be wirelessly connected to an electronic device such as a mobile phone and output audio data received from the mobile phone. Wireless audio devices can increase user convenience because they are wirelessly connected to electronic devices (e.g., Bluetooth connection).

An active noise cancellation (ANC) method may be performed to remove noise within a wireless audio device. ANC is a way for users to reduce unwanted sounds by adding a second signal designed to eliminate the first signal.

Recently, the use of wireless audio devices has increased significantly, and there is a need for technologies to improve active noise cancellation methods.

According to one embodiment, the means for solving the problem are not limited to the above-mentioned solution means, and solution means not mentioned are clear to those skilled in the art from the present specification and the attached drawings. It will be understandable.

According to one embodiment, an electronic device that performs active noise cancellation includes: a first microphone for detecting an input signal; and at least one processor connected to the first microphone. The at least one processor may check a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal has been filtered. The at least one processor may determine a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal. The at least one processor may predict a filter weight to be applied to a filter using machine learning based on the plurality of filter weights.

According to one embodiment, a method of operating an electronic device that performs active noise removal may include detecting an input signal. The method of operating the electronic device may include checking a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal has been filtered. there is. The method of operating the electronic device may include determining a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal. The method of operating the electronic device may include predicting a filter weight to be applied to a filter using machine learning based on the plurality of filter weights.

According to one embodiment, at least one computer program including instructions for performing an active noise removal method may be recorded in a computer-readable recording medium. The active noise removal method may include detecting an input signal. The active noise removal method may include the step of checking a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal is filtered. . The active noise removal method may include determining a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal. The active noise removal method may include predicting a filter weight to be applied to a filter using machine learning based on the plurality of filter weights.

1 is a block diagram of an electronic device in a network environment according to one embodiment.
Figure 2 is a block diagram of an audio module according to one embodiment.
Figure 3 shows an example of communication between electronic devices according to an embodiment.
FIG. 4 is a diagram illustrating a process for removing noise from an electronic device according to an embodiment.
Figures 5 and 6 are diagrams illustrating a process in which an electronic device removes noise using FxLMS, according to an embodiment.
FIG. 7 is a diagram illustrating a process for predicting filter weights applied to an electronic device according to an embodiment.
FIG. 8 is a diagram illustrating an electronic device including a weight prediction module according to an embodiment.
FIG. 9 is a diagram illustrating an electronic device including a weight change prediction module according to an embodiment.
FIG. 10 is a diagram explaining the operation of an electronic device according to an embodiment.
FIGS. 11A and 11B are diagrams showing effects resulting from the operation of an electronic device according to an embodiment.
FIG. 12 is a flowchart explaining a method of operating an electronic device according to an embodiment.

Hereinafter, embodiments of the present disclosure are described with reference to the attached drawings. However, this is not intended to limit the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of the present disclosure.

1 is a block diagram of an electronic device in a network environment according to one embodiment.

Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to one embodiment, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes: a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

An electronic device according to an embodiment of the present disclosure may be of various types. Electronic devices include, for example, wireless audio devices (e.g., wireless earphones, earbuds, true wireless stereo (TWS), or earsets), portable communication devices (e.g., smartphones), computer devices, and portable multimedia devices. , may include portable medical devices, cameras, wearable devices, or home appliance devices. Electronic devices according to embodiments of this document are not limited to the above-described devices.

An embodiment of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various changes, equivalents, or substitutes for the embodiment. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to that component in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in one embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be interchangeable with terms such as logic, logic block, component, or circuit, for example. can be used A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

One embodiment of the present document is one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term refers to cases where data is semi-permanently stored in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, a method according to an embodiment disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or via an application store (e.g. Play Store ^TM ) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to one embodiment, each component (e.g., module or program) of the above-described components may include a single or multiple entities, and some of the multiple entities may be separately placed in other components. . According to one embodiment, one or more of the above-described corresponding components or operations may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar manner as those performed by the corresponding component of the plurality of components prior to the integration. . According to one embodiment, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or , or one or more other operations may be added.

Figure 2 is a block diagram of an audio module according to one embodiment.

Referring to FIG. 2, the audio module 170 includes, for example, an audio input interface 210, an audio input mixer 220, an analog to digital converter (ADC) 230, an audio signal processor 240, and a DAC. (digital to analog converter) 250, an audio output mixer 260, or an audio output interface 270.

The audio input interface 210 is a part of the input module 150 or is configured separately from the electronic device 101 to obtain audio from the outside of the electronic device 101 through a microphone (e.g., dynamic microphone, condenser microphone, or piezo microphone). An audio signal corresponding to sound can be received. For example, when an audio signal is acquired from an external electronic device 102 (e.g., a headset or microphone), the audio input interface 210 is directly connected to the external electronic device 102 through the connection terminal 178. , or the audio signal can be received by connecting wirelessly (e.g., Bluetooth communication) through the wireless communication module 192. According to one embodiment, the audio input interface 210 may receive a control signal (eg, a volume adjustment signal received through an input button) related to the audio signal obtained from the external electronic device 102. The audio input interface 210 includes a plurality of audio input channels and can receive different audio signals for each corresponding audio input channel among the plurality of audio input channels. According to one embodiment, additionally or alternatively, the audio input interface 210 may receive an audio signal from another component of the electronic device 101 (eg, the processor 120 or the memory 130).

The audio input mixer 220 may synthesize a plurality of input audio signals into at least one audio signal. For example, according to one embodiment, the audio input mixer 220 may synthesize a plurality of analog audio signals input through the audio input interface 210 into at least one analog audio signal.

The ADC 230 can convert analog audio signals into digital audio signals. For example, according to one embodiment, the ADC 230 converts the analog audio signal received through the audio input interface 210, or additionally or alternatively, the analog audio signal synthesized through the audio input mixer 220 into a digital audio signal. It can be converted into a signal.

The audio signal processor 240 may perform various processing on a digital audio signal input through the ADC 230 or a digital audio signal received from another component of the electronic device 101. For example, according to one embodiment, the audio signal processor 240 may change the sampling rate, apply one or more filters, process interpolation, amplify or attenuate all or part of the frequency band, and You can perform noise processing (e.g., noise or echo attenuation), change channels (e.g., switch between mono and stereo), mix, or extract specified signals. According to one embodiment, one or more functions of the audio signal processor 240 may be implemented in the form of an equalizer.

The DAC 250 can convert digital audio signals into analog audio signals. For example, according to one embodiment, DAC 250 may process digital audio signals processed by audio signal processor 240, or other components of electronic device 101 (e.g., processor 120 or memory 130). The digital audio signal obtained from )) can be converted to an analog audio signal.

The audio output mixer 260 may synthesize a plurality of audio signals to be output into at least one audio signal. For example, according to one embodiment, the audio output mixer 260 may output an audio signal converted to analog through the DAC 250 and another analog audio signal (e.g., an analog audio signal received through the audio input interface 210). ) can be synthesized into at least one analog audio signal.

The audio output interface 270 transmits the analog audio signal converted through the DAC 250, or additionally or alternatively, the analog audio signal synthesized by the audio output mixer 260 to the electronic device 101 through the audio output module 155. ) can be output outside of. The sound output module 155 may include, for example, a speaker such as a dynamic driver or balanced armature driver, or a receiver. According to one embodiment, the sound output module 155 may include a plurality of speakers. In this case, the audio output interface 270 may output audio signals having a plurality of different channels (eg, stereo or 5.1 channels) through at least some of the speakers. According to one embodiment, the audio output interface 270 is connected to the external electronic device 102 (e.g., external speaker or headset) directly through the connection terminal 178 or wirelessly through the wireless communication module 192. and can output audio signals.

According to one embodiment, the audio module 170 does not have a separate audio input mixer 220 or an audio output mixer 260, but uses at least one function of the audio signal processor 240 to generate a plurality of digital audio signals. At least one digital audio signal can be generated by synthesizing them.

According to one embodiment, the audio module 170 is an audio amplifier (not shown) capable of amplifying an analog audio signal input through the audio input interface 210 or an audio signal to be output through the audio output interface 270. (e.g., speaker amplification circuit) may be included. According to one embodiment, the audio amplifier may be composed of a module separate from the audio module 170.

Figure 3 shows an example of communication between electronic devices according to an embodiment.

Referring to FIG. 3, the first electronic device 301 and the second electronic device 302 (e.g., the first wireless audio device 302-1 and/or the second wireless audio device 302-2) are shown in FIG. At least some of the electronic devices shown in FIG. 1 or FIG. 2 may include the same or similar components, and at least some of them may perform the same or similar functions.

The first electronic device 301 may include a user terminal, such as a smartphone, tablet, desktop computer, or laptop computer. The second electronic device 302 may include, but is not limited to, wireless earphones, a headset, earbuds, true wireless stereo (TWS), or an earset. The second electronic device 302 may include various types of devices (eg, hearing aids or portable audio devices) that receive audio signals and output the received audio signals.

For example, the first electronic device 301 and the second electronic device 302 may perform wireless communication in a short distance according to a Bluetooth network. The Bluetooth network may include, for example, a Bluetooth legacy network or a BLE network. According to one embodiment, the first electronic device 301 and the second electronic device 302 may perform wireless communication through one of a Bluetooth legacy network or a BLE network, or may perform wireless communication through two networks. You can.

For example, when music is played on the first electronic device 301, the first electronic device 301 connects to the second electronic device 302 and the created link (e.g., the first link 305 and/or the second A data packet including content (e.g., music data) may be transmitted through link 310). For example, at least one of the second electronic devices 302 may transmit a data packet including content (eg, audio data) to the first electronic device 301 through the created link. For example, the first wireless audio device 302-1 and the second wireless audio device 302-2 may transmit and receive data packets or messages through an established link (eg, third link 315).

FIG. 4 is a diagram illustrating a process for removing noise from an electronic device according to an embodiment.

Referring to FIG. 4, the electronic device may be implemented as a wireless earphone, headset, earbud, true wireless stereo (TWS), or earset. According to one embodiment, the electronic device may include components that are at least partially the same as or similar to each of the electronic devices shown in FIGS. 1 to 3 , and at least partially may perform the same or similar functions.

In step 410, the electronic device may detect an incoming ambient sound input through a microphone. In step 420, the detected ambient sound may be transmitted to a noise cancellation circuitry within the electronic device.

In step 430, a noise cancellation circuitry in the electronic device may invert a wave corresponding to the surrounding sound and transmit the inverted wave to a speaker in the electronic device.

In step 440, the electronic device may perform an active noise cancellation (ANC) operation by removing surrounding sounds using inverted waves provided by the noise removal circuit.

According to one embodiment, signal filtering is required for ANC, and a filter that provides complete attenuation may not exist due to physical limitations. Additionally, the SoTA adaptive filter algorithm may not converge quickly enough to achieve optimal fixed weight filter performance.

This disclosure proposes faster and better filter adaptation compared to techniques relying on the gradient descent method, overcomes the weaknesses of the filtered reference least mean squares (FxLMS) algorithm, and can improve the efficiency of ANC. .

Figures 5 and 6 are diagrams illustrating a process in which an electronic device removes noise using FxLMS, according to an embodiment. The electronic device including a feed-forward active noise control system in FIGS. 5 and 6 includes components that are at least partially the same or similar to each of the electronic devices shown in FIGS. 1 to 4. And at least some of them may perform the same or similar functions.

The ANC technique uses the FXLMS algorithm in a DSP (digital signal processor) to detect a noise signal generated from a noise source using an adaptive signal processing technique, and generates an additional signal of an anti-phase to reduce this noise. By doing so, the principle of actively reducing noise can be used.

Figure 5 shows an example of a feed-forward active noise control system using the FxLMS algorithm. According to one embodiment, in a feed-forward active noise control system, the ANC system may be implemented with a processor (eg, digital signal processor (DSP)) included in an electronic device.

Referring to Figure 5, the feed-forward active noise control system may include a primary path (P) 510, an ANC system (or ANC filter) 520, and a secondary path (S) 530. . The ANC system 520 includes a filter (F) 522, an estimated secondary path ( ) 524, and an LMS module 526.

The input signal (x) is a signal (or surrounding signal) generated by a noise source and may also be referred to as a reference signal. The input signal (x) may be measured by at least one sensor in the electronic device.

According to one embodiment, the input signal (x) is divided into a primary path (P) 510, a filter (F) 522, and an estimated secondary path ( )(524) can be provided respectively. The input signal (x) may be converted into a noise signal (d) through the primary path (P) 510. The filter (F) 522 may filter the input signal (x) and output the filtered signal (y). According to one embodiment, filter (F) 522 may impose an additional 180° phase shift on the signal path. The input signal (x) is the estimated secondary path ( ) can be converted to a modified input signal (x') through 524.

According to one embodiment, the signal (y) filtered by the filter (F) 522 at the nth stage (eg, nth time) may be expressed as Equation 1.

[Equation 1]

Here, x(n) may be the input signal (x) in the nth step, and f(n) may be a weight (or weight function) applied to the filter in the nth step. According to one embodiment, LMS module 526 may generate (or calculate) f(n).

The signal (y) filtered by the filter (F) 522 may be converted into a modified filtered signal (y') through a secondary path (S) 530. The modified filtered signal (y') may overlap with the noise signal (d), which is the output of the primary path (P) 510. An error signal (e) may be generated as a result of overlap between the modified filtered signal (y') and the noise signal (d).

The LMS module 526 determines the estimated secondary path ( ) (524) to calculate the optimal weight (or filter coefficient) to be used in the filter (F) (522) based on the corrected input signal (x') and error signal (e) (least mean square (LMS)) Algorithms can be used. According to one embodiment, the LMS module 526 may determine a weight (or filter coefficient) and provide it to the filter (F) 522. According to one embodiment, the LMS module 526 may calculate an optimal set of filter coefficients for the filter (F) 522, and the filter (F) 522 may calculate a filter transfer function W ( z).

According to one embodiment, the LMS module 526 may update the weight (or weight function) for the filter (F) 522 as shown in Equation 2 using the LMS algorithm at each step.

[Equation 2]

here, is a preset coefficient, f(n) is the weight (or weight function) at the nth step, x'(n) is the modified input signal (x') at the nth step, and f(n+1 ) may be the weight (or weight function) at the n+1th step.

According to one embodiment, the ANC system 520 may include at least two speakers (e.g., noise speaker, canceling speaker), at least two microphones (e.g., reference microphone, error microphone), and a controller. . A noise speaker may be a speaker that generates noise, and a canceling speaker may be a speaker that generates sound for noise control. A reference microphone may be a microphone that measures noise to be controlled in advance, and an error microphone may be a microphone that measures noise control results.

According to one embodiment, x(n) may be a signal measured by a reference microphone, and e(n) may be a signal measured by an error microphone. According to one embodiment, d(n) may be noise that must be controlled through ANC.

Referring to Figure 6, the feed-forward active noise control system may include a primary path (P) 610, an ANC system (or ANC filter) 620, and a secondary path (S) 630. . The ANC system 620 includes a filter (F) 622, an estimated secondary path ( ) 624, and an LMS module 626. The ANC system 620 may be implemented with a processor (eg, digital signal processor (DSP)) included in an electronic device.

According to one embodiment, the LMS module 626 may be controlled to minimize the mean squared error for the error signal e in the long term, as shown in Equation 3.

[Equation 3]

According to one embodiment, FxLMS may converge from the average to the optimal solution as shown in Equation 4.

[Equation 4]

where F ₀ (z) is the transfer function in the z domain for the filter (F) 622, P(z) is the transfer function in the z domain for the primary path (P) 610, and S (z) may be a transfer function in the z domain for the secondary path (S) 630.

According to one embodiment, referring to Equation 4 and Equation 5, the filter (F) 622 is expressed as a finite impulse response filter (e.g., finite impulse response filter (zeros only)) and has an infinite response transfer function (infinite It may have a tendency to approximate the response transfer function.

[Equation 5]

According to one embodiment, S(z) may have poles outside the unit circle, a false approximation by a casual FIR filter is performed, and only limited noise can be removed.

FIG. 7 is a diagram illustrating a process for predicting filter weights applied to an electronic device according to an embodiment.

This disclosure proposes a method for predicting future weights (or parameters) for an adaptive filter when applying ANC. According to one embodiment, when a filtered signal (y) is output using a filter weight at a different time point to a filter, ANC performance at the current time point may be improved.

Referring to FIG. 7, the degree of signal attenuation according to the time advancement of filter weights (K) can be displayed in a graph. According to one embodiment, a section in which signal attenuation further increases may occur depending on the time progress value (K) of the filter weight. According to one embodiment, the time progression value (K) of the filter weight and the resulting signal attenuation may be predicted by a machine learning model.

According to one embodiment, the signal y(n) filtered by the filter F at the nth step (eg, nth time) may be expressed as Equation 6.

[Equation 6]

Here, x(n) is the input signal (x) at the nth step (e.g., the nth time), and f(n+K) is the weight (or weight function) applied to the filter at the n+Kth step. ) can be. According to one embodiment, the time progression value (K) of the filter weight may be calculated by a machine learning model. According to one embodiment, a machine learning model may be a technology in which an electronic device analyzes data on its own and predicts future filter weights.

FIGS. 8 and 9 are diagrams illustrating a process in which an electronic device removes noise using FxLMS, according to an embodiment. The electronic device including the feed-forward active noise control system in FIGS. 8 and 9 includes components that are at least partially the same or similar to each of the electronic devices shown in FIGS. 1 to 4. And at least some of them may perform the same or similar functions.

FIG. 8 is a diagram illustrating an electronic device including a weight prediction module according to an embodiment. According to one embodiment, in a feed-forward active noise control system, the ANC system may be implemented with a processor (eg, digital signal processor (DSP)) included in an electronic device.

Referring to Figure 8, the feed-forward active noise control system may include a primary path (P) 810, an ANC system (or ANC filter) 820, and a secondary path (S) 830. . The ANC system 820 includes a first filter (F) 821, a second filter (F _pred ) 822, and an estimated secondary path ( ) (823, 824, 825), an LMS module (826), and a weight prediction module (weight prediction) (827).

The input signal (x) is a signal (or surrounding signal) generated by a noise source and may also be referred to as a reference signal (ref). The input signal (x) may be measured by at least one sensor or microphone within the electronic device.

According to one embodiment, the input signal (x) includes a primary path (P) 810, a first filter (F) 821, a second filter (F _pred ) 822, and an estimated secondary path ( )(824) can be provided respectively. The input signal (x) may be converted into a noise signal (d) through the primary path (P) 810.

According to one embodiment, the first filter (F) 821 may filter the input signal (x) and output a first filtered signal (y). The first filter (F) 821 may provide a first filtering signal (y) repeatedly calculated through a feedback loop to the weight prediction module 827. The weight prediction module 827 may calculate (or generate) a prediction weight to be applied to the second filter (F _pred ) 822 based on the first filtering signal (y). According to one embodiment, the weight prediction module 827 calculates (or generates) a prediction weight to be applied to the second filter (F _pred ) 822 through machine learning based on the plurality of first filtering signals (y). can do.

According to one embodiment, the input signal (x) is an estimated secondary path ( ) can be converted into a modified input signal (x') through 824. The modified input signal (x') is provided to the LMS module 826, which determines a weight (or filter coefficient) and provides it to the first filter (F) 821. According to one embodiment, the LMS module 826 may calculate an optimal filter coefficient set for the first filter (F) 821, and the first filter (F) 821 may be a filter corresponding to the optimal filter coefficient set. We can have a transfer function W(z).

According to one embodiment, the second filtered signal (y) filtered by the second filter (F _pred ) 822 is converted into a modified filtered signal (y') via a secondary path (S) 830. It can be. The modified filtered signal (y') may overlap with the noise signal (d) that is the output of the primary path (P) 810. An error signal (e) may be generated as a result of overlap between the modified filtered signal (y') and the noise signal (d).

According to one embodiment, the second filtering signal (y) filtered by the second filter (F _pred ) 822 at the nth stage (e.g., nth time) may be expressed as Equation 7: .

[Equation 7]

Here, x(n) is the input signal (x) at the nth stage (e.g., nth time), and f(n+K) is the second filter (F _pred ) at the n+Kth stage (822 ) may be a weight (or weight function) applied to According to one embodiment, the time progression value (K) of the filter weight may be calculated by a machine learning model. According to one embodiment, the weight prediction module 827 may predict (or calculate) f(n+K).

According to one embodiment, the weight prediction module 827 may predict the weight (or weight function) for the second filter (F _pred ) 822 as shown in Equation 8 based on a machine learning model.

[Equation 8]

Here, f(n) is the weight (or weight function) at the nth step, f(n-1) is the weight (or weight function) at the n-1th step, and f(n-2) is the nth step. -This is the weight (or weight function) in the second stage, and f(n-p) may be the weight (or weight function) in the n-pth stage. M(f(n),f(n-1),f(n-2),..., f(n-p)) is machine learning considering the weights (or weight function) from the nth step to the n-pth step. It may be a model function.

FIG. 9 is a diagram illustrating an electronic device including a weight change prediction module according to an embodiment.

Figure 9 shows an example of a feed-forward active noise control system using the FxLMS algorithm. According to one embodiment, in a feed-forward active noise control system, the ANC system may be implemented with a processor (eg, digital signal processor (DSP)) included in an electronic device.

Referring to Figure 9, the feed-forward active noise control system may include a primary path (P) 910, an ANC system (or ANC filter) 920, and a secondary path (S) 930. . The ANC system 920 includes a filter (F) 922, an estimated secondary path ( ) 924, an LMS module 926, and a weight change prediction module 928.

The input signal (x) is a signal (or surrounding signal) generated by a noise source and may also be referred to as a reference signal. The input signal (x) may be measured by at least one sensor or microphone within the electronic device.

According to one embodiment, the input signal (x) is divided into a primary path (P) 910, a filter (F) 922, and an estimated secondary path ( )(924) can be provided respectively. The input signal (x) may be converted into a noise signal (d) through the primary path (P) 910. The filter (F) 922 may filter the input signal (x) and output the filtered signal (y). The input signal (x) is the estimated secondary path ( ) can be converted to a modified input signal (x') through 924.

According to one embodiment, the signal (y) filtered by the filter (F) 922 may be converted into a modified filtered signal (y') through a secondary path (S) 930. The modified filtered signal (y') may overlap with the noise signal (d) that is the output of the primary path (P) 910. An error signal (e) may be generated as a result of overlap between the modified filtered signal (y') and the noise signal (d).

According to one embodiment, the LMS module 926 determines the estimated secondary path ( ) (924) to calculate the optimal weight (or filter coefficient) to be used in the filter (F) (922) based on the corrected input signal (x') and error signal (e) (least mean square (LMS)) Algorithms can be used. According to one embodiment, the LMS module 926 may determine a weight (or filter coefficient) and provide it to the weight change prediction module 928.

According to one embodiment, the LMS module 926 may calculate the weight (or filter coefficient) change df(n) at the nth step (eg, nth time) as shown in Equation 9.

[Equation 9]

here, is a preset coefficient, e(n) may be an error signal (e) at the nth stage, and x'(n) may be a modified input signal (x') at the nth stage.

According to one embodiment, the LMS module 926 may provide the weight (or filter coefficient) change df(n) to the weight change prediction module 928.

According to one embodiment, the weight change prediction module 928 may predict the weight (or weight function) for the filter (F) 922 as shown in Equation 10 using a machine learning model.

[Equation 10]

Here, df(n) is the change in weight (or weight function) at the nth step, df(n-1) is the change in weight (or weight function) at the n-1th step, and df(n-2) is the weight (or weight function) change in the n-2th step, and df(n-p) may be the weight (or weight function) change in the n-pth step. M(df(n),df(n-1),df(n-2),..., df(n-p)) is a machine that considers the change in weight (or weight function) from the nth step to the n-pth step. It may be a learning model function. f(n+K-1) may be a weight (or weight function) at step n+K-1, and f(n+K) may be a weight (or weight function) at step n+K.

According to one embodiment, the signal (y) filtered by the filter (F) 922 at the nth stage (eg, nth time) may be expressed as Equation 11.

[Equation 11]

Here, x(n) is the input signal (x) at the nth step (e.g., nth time), and f(n+K) is applied to the filter (F) 922 at the n+Kth step. It may be a weight (or weight function) that is According to one embodiment, the time progression value (K) of the filter weight may be calculated by a machine learning model. According to one embodiment, the weight change prediction module 928 may predict (or calculate) f(n+K).

FIG. 10 is a diagram explaining the operation of an electronic device according to an embodiment.

Referring to FIG. 10, the electronic device may start an ANC system to perform active noise cancellation (ANC) (1000). The electronic device may execute a first mode 1010 and/or a second mode 1020 for ANC. The electronic device in FIG. 10 includes at least some of the same or similar components as each of the electronic devices shown in FIGS. 1 to 9 , and at least some of the electronic devices may perform the same or similar functions.

According to one embodiment, when the electronic device executes the first mode 1010 for ANC, weights for the filter may be collected after initial convergence (1011) (filter weight gathering, 1013). The electronic device can estimate the optimal weight shift to be applied to the filter after collecting the weights for the filter (estimation of optimal weight shift, 1015). The electronic device may perform prediction model training (1017) on the weights after estimating the optimal weight shift to be applied to the filter.

According to one embodiment, the electronic device may apply a predictive model for filter weights when executing the second mode 1020 for ANC (apply predictive model, 1021). The electronic device may monitor ANC performance (monitor ANC performance, 1023). After monitoring ANC performance, the electronic device may collect filter weights (filter weight gathering, 1025) and perform prediction model retraining (prediction model retraining, 1027) for the filter weights. The electronic device may re-apply the prediction model for the filter weight after retraining the prediction model for the filter weight (apply predictive model, 1021).

FIGS. 11A and 11B are diagrams showing effects resulting from the operation of an electronic device according to an embodiment.

Referring to FIG. 11A, attenuation for each frequency according to simulations of the feed-forward FxLMS method, the weight prediction method shown in FIG. 8, the weight update prediction method shown in FIG. 9, and the wiener method. represents.

Referring to FIG. 11B, the weight prediction method shown in FIG. 8 or the weight update prediction method shown in FIG. 9 has about 6 to 10 dB for non-minimum phase zero in the range of 200 to 300 Hz. performance can be improved.

Figure 12 is a flowchart for explaining a method of operating an electronic device according to an embodiment.

Referring to FIG. 12, in step 1210, the electronic device may check a noise signal included in the input signal and an error signal representing the noise signal that has not been removed according to the operation result of the first filtering signal from which the input signal has been filtered. . According to one embodiment, the electronic device may be one of wireless earphones, a headset, earbuds, true wireless stereo (TWS), or an earset. In FIG. 12 , the electronic device includes at least some components that are the same or similar to each of the electronic devices shown in FIGS. 1 to 10 , and at least some of them may perform the same or similar functions.

In step 1220, the electronic device may determine a plurality of filter weights using an LMS algorithm based on the input signal and the error signal. In step 1230, the electronic device may predict a filter weight to be applied to the filter using machine learning based on the plurality of filter weights.

In step 1240, the electronic device may generate a second filtering signal by filtering the input signal based on the filter weight. In step 1250, the electronic device may remove the noise signal so that the error signal falls within a preset range based on the input signal and the second filtering signal.

According to one embodiment, the electronic device may further include a first microphone for detecting the input signal and a second microphone for detecting the error signal. According to one embodiment, the electronic device may further include a weight prediction module that predicts the filter weight to be applied to the filter using machine learning based on the filter and the plurality of filter weights. According to one embodiment, the electronic device may further include a weight change prediction module that predicts a change in the filter weight to be applied to the filter using the machine learning based on the plurality of filter weights. According to one embodiment, an electronic device may perform wireless communication with an external electronic device in a short distance based on a Bluetooth network.

According to one embodiment, at least one computer program including instructions for performing an active noise removal method may be recorded in a computer-readable recording medium. The active noise removal method may include detecting an input signal. The active noise removal method may include the step of checking a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal is filtered. . The active noise removal method may include determining a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal. The active noise removal method may include predicting a filter weight to be applied to a filter using machine learning based on the plurality of filter weights.

Claims (17)

In an electronic device that performs active noise cancellation,
A first microphone (150 in FIG. 1) for detecting an input signal; and
It includes at least one processor (120 in FIG. 1; 820 in FIG. 8; 920 in FIG. 9) connected to the first microphone, and the at least one processor (120 in FIG. 1; 820 in FIG. 8; 920 in FIG. 9). 920;) is,
Checking a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal is filtered,
Determining a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal,
An electronic device configured to predict a filter weight to be applied to a filter using machine learning based on the plurality of filter weights. The method of claim 1, wherein the at least one processor:
An electronic device configured to generate a second filtering signal by filtering the input signal based on the filter weight. The method of claim 1 or 2, wherein the at least one processor:
An electronic device configured to remove the noise signal so that the error signal falls within a preset range based on the input signal and the filter weight. According to either paragraph 1 or 3,
The electronic device is configured to further include a second microphone for detecting the error signal. The method of claim 1 or 4, wherein the at least one processor:
the filter; and
The electronic device is configured to further include a weight prediction module that predicts the filter weight to be applied to the filter using the machine learning based on the plurality of filter weights. The method of claim 1 or 5, wherein the at least one processor:
The electronic device is configured to further include a weight change prediction module that predicts a change in the filter weight to be applied to the filter using the machine learning based on the plurality of filter weights. According to any one of paragraphs 1 and 6,
The electronic device is one of wireless earphones, a headset, earbuds, true wireless stereo (TWS), or an earset. According to either paragraph 1 or 7,
The electronic device performs wireless communication with an external electronic device at a short distance based on a Bluetooth network. In a method of operating an electronic device that performs active noise cancellation,
detecting an input signal;
checking a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal is filtered;
determining a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal; and
A method comprising predicting a filter weight to be applied to a filter using machine learning based on the plurality of filter weights. According to clause 9,
The method further includes generating a second filtered signal by filtering the input signal based on the filter weight. According to either paragraph 9 or 10,
The method further includes removing the noise signal so that the error signal falls within a preset range based on the input signal and the filter weight. According to either paragraph 9 or 11,
The method of claim 1, wherein the electronic device further includes a second microphone for detecting the error signal. The method of claim 9 or 12, wherein the electronic device,
the filter; and
The method is configured to further include a weight prediction module that predicts the filter weight to be applied to the filter using the machine learning based on the plurality of filter weights. The method of claim 9 or 13, wherein the electronic device,
The method is configured to further include a weight change prediction module that predicts a change in the filter weight to be applied to the filter using the machine learning based on the plurality of filter weights. According to either clause 9 or clause 14,
The method of claim 1, wherein the electronic device is one of wireless earphones, a headset, earbuds, true wireless stereo (TWS), or an earset. According to either clause 9 or clause 15,
A method wherein the electronic device performs wireless communication at a short distance with an external electronic device according to a Bluetooth network. In the computer-readable recording medium on which at least one computer program including instructions for performing an active noise cancellation method is recorded,
The method is:
detecting an input signal;
checking a noise signal included in the input signal and an error signal representing a noise signal that has not been removed according to an operation result of a first filtering signal from which the input signal is filtered;
determining a plurality of filter weights using a least mean square (LMS) algorithm based on the input signal and the error signal; and
A computer-readable recording medium comprising predicting filter weights to be applied to a filter using machine learning based on the plurality of filter weights.

Images