WO2024080723A1

WO2024080723A1 - Electronic device and method for processing audio signal

Info

Publication number: WO2024080723A1
Application number: PCT/KR2023/015590
Authority: WO
Inventors: 양현철; 김현욱; 문한길; 박재하; 방경호; 이상훈; 허승
Original assignee: 삼성전자 주식회사
Priority date: 2022-10-14
Filing date: 2023-10-11
Publication date: 2024-04-18

Abstract

An embodiment of the present disclosure relates to a device and a method for minimizing quantization noise by reflecting a user's individual hearing characteristics when quantizing or de-quantizing an audio signal. A control method therefor may comprise the operations of: obtaining the number of quantization bits for each divided audible frequency band obtained by diving an audible frequency band on the basis of the user's preconfigured hearing characteristics; performing quantization by using the number of quantization bits corresponding to a divided audible frequency band-specific audio signal, which is extracted from an audio signal output by reproducing audio content; and generating a bitstream from a quantized audio signal for each divided audible frequency band, and transmitting same to an external electronic device through a wireless channel. Various other embodiments are possible.

Description

Electronic devices and methods for processing audio signals

This disclosure relates to electronic devices and methods for processing audio signals through quantization or inverse quantization.

Electronic devices (e.g., computers, portable terminals, tablets, etc.) may be connected to external electronic devices such as wireless earphones (e.g., true wireless stereo (TWS)) using a short-range wireless communication method such as Bluetooth. Electronic devices can transmit content data, such as audio signals, to a connected external electronic device using a wireless communication method. Electronic devices may experience delays or damage to content data such as audio signals transmitted due to distance from wireless earphones, channel congestion, or unexpected failures. Electronic devices need to compress and transmit data using a predetermined encoding method to prevent transmission data from being damaged in the process of being transmitted to wireless earphones, which are external electronic devices. The wireless earphone, which is an external electronic device, can restore compressed data using a predetermined decoding method corresponding to the encoding method used in the other electronic device.

In short-distance communication methods such as Bluetooth, transmission performance may be proportional to the amount of data encoded by the electronic device. A large amount of data encoded in an electronic device may mean that high-quality service can be provided by transmitting a lot of information to wireless earphones, which are external electronic devices. However, as the amount of data to be transmitted increases, traffic in the wireless channel connecting the electronic device and the wireless earphone, which is an external electronic device, may increase.

One embodiment of the present disclosure may provide an electronic device and method that performs an encoding or decoding operation on an audio signal by reflecting the user's hearing characteristics.

A control method of an electronic device according to an embodiment of the present disclosure includes an operation of dividing the audible frequency band based on preset hearing characteristics of a user, obtaining the number of quantization bits for each divided audible frequency band, and outputting by playing audio content. An operation of performing quantization using the number of quantization bits corresponding to the audio signal for each divided audible frequency band extracted from the audio signal, and generating the quantized audio signal for each divided audible frequency band as a bit stream and transmitting the quantized audio signal to an external electronic signal through a wireless channel. It may include the operation of transmitting to a device.

A control method of an electronic device according to an embodiment of the present disclosure includes an operation of analyzing a bit stream received from an external electronic device, an operation of obtaining the number of inverse quantization bits for each divided audible frequency band included in the bit stream, and the inverse quantization bits. It may include an operation of performing inverse quantization on the bit stream for each audible frequency band using the number of quantization bits, and an operation of outputting an audio signal generated by the inverse quantization.

An electronic device according to an embodiment of the present disclosure may include at least one processor and a communication module. The at least one processor obtains the number of quantization bits for each divided audible frequency band that divides the audible frequency band based on preset hearing characteristics of the user, and divides the audible frequency band extracted from the audio signal output by playing the audio content. Quantization may be performed using the number of quantization bits corresponding to the audio signal for each band, and the quantized audio signal for each divided audible frequency band may be generated as one bit stream and transmitted to an external electronic device through a wireless channel.

An electronic device according to an embodiment of the present disclosure may include at least one processor and a communication module. The at least one processor analyzes a bit stream received from an external electronic device, obtains the number of inverse quantization bits for each divided audible frequency band included in the bit stream, and uses the number of inverse quantization bits for each audible frequency band. Inverse quantization may be performed on the bit stream, and an audio signal generated through the inverse quantization may be output.

In the present disclosure, an electronic device may include a non-transitory computer-readable storage medium that stores one or more programs. One or more programs stored in the computer-readable storage medium obtain the number of quantization bits for each divided audible frequency band that divides the audible frequency band based on preset hearing characteristics of the user, and produce an audio signal output by playback of audio content. Quantization is performed using the number of quantized bits corresponding to the audio signal for each divided audible frequency band extracted from the divided audible frequency band, and the quantized audio signal for each divided audible frequency band is generated as one bit stream and transmitted to an external electronic device through a wireless channel. May include instructions to be transmitted.

In the present disclosure, an electronic device may include a non-transitory computer-readable storage medium that stores one or more programs. One or more programs stored in the computer-readable storage medium analyzes a bit stream received from an external electronic device, obtains the number of inverse quantization bits for each divided audible frequency band included in the bit stream, and calculates the number of inverse quantization bits. It may include instructions for performing inverse quantization on the bit stream for each audible frequency band and outputting an audio signal generated by the inverse quantization.

The technical problems to be achieved by the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not mentioned above may be derived from the exemplary embodiments of the present disclosure by those skilled in the art. there is.

According to an embodiment of the present disclosure, the electronic device divides the audible frequency band, which is an audio signal, into a plurality of audible frequency bands in consideration of the user's hearing characteristics and applies different coding rates to each divided frequency band, so that if the amount of data is not increased, It can also increase the satisfaction of users who listen to audio signals through wireless earphones.

Effects that can be obtained from the exemplary embodiments of the present disclosure can be clearly derived and understood by those skilled in the art from the following description. That is, unintended effects resulting from implementing the exemplary embodiments of the present disclosure may also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

1 is a block diagram of an electronic device in a network environment, according to various embodiments.

2 is a block diagram of an audio module, according to various implementations.

Figure 3 is a block diagram of a first electronic device according to an embodiment of the present disclosure.

FIG. 4 is a flowchart showing an operation in which a first electronic device quantizes an audio signal, according to an embodiment of the present disclosure.

FIG. 5 is a control flowchart for a first electronic device to quantize an audio signal, according to an embodiment of the present disclosure.

Figure 6 is a block diagram of a second electronic device according to an embodiment of the present disclosure.

FIG. 7 is a flowchart illustrating an operation in which a second electronic device inversely quantizes an audio signal, according to an embodiment of the present disclosure.

FIG. 8 is a control flowchart for a second electronic device to inverse quantize an audio signal, according to an embodiment of the present disclosure.

Figure 9 is a block diagram of an encoder of a first electronic device, according to an embodiment of the present disclosure.

FIG. 10 is a block diagram of a portion of an encoder of a first electronic device according to an embodiment of the present disclosure.

Figure 11 is a block diagram of a decoder of a second electronic device, according to an embodiment of the present disclosure.

In relation to the description of the drawings, identical or similar reference numerals may be used for identical or similar components.

Hereinafter, with reference to the drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, identical or similar reference numerals may be used for identical or similar components. Additionally, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

1 is a block diagram of an electronic device 101 in a network environment 100, according to various embodiments. Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or operations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a auxiliary processor 123, the auxiliary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support a high frequency band (eg, mmWave band), for example, to achieve a high data rate. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band), And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

When the electronic device 101 compresses an audio signal, the audio signal to be compressed may be quantized. Quantization is the process of dividing the actual value of an audio signal into regular intervals. In other words, quantization expresses the size of the waveform of an audio signal with several quantization levels at a predetermined quantization interval.

If the quantization interval is too wide, noise due to quantization may occur, which is called quantization noise. If quantization noise increases, the sound quality of the audio signal perceived by the user may deteriorate. Conversely, if the quantization noise is too narrow, the quantization noise may be reduced, but the number of segments of the audio signal to be expressed after quantization processing increases, which increases the bitrate required for encoding per unit time. The amount of data that needs to be transmitted increases.

Therefore, it would be necessary to find the optimal quantization interval to minimize audio signal deterioration due to quantization noise and at the same time prevent the bit rate from increasing.

When quantizing, different quantization intervals can be determined for each frequency. According to the general psychoacoustic model, most users are sensitive to relatively low frequency bands in the audible frequency band (e.g., 250 Hz to 8000 Hz), so beats at low frequencies with narrow quantization intervals are used. can be assigned.

According to one example, a source electronic device (e.g., the electronic device 101 of FIG. 1 (hereinafter referred to as 'source electronic device 101' or 'first electronic device 101')) transmits audio data to an external electronic device ( Example: The second electronic device 102 of FIG. 1 (hereinafter referred to as 'external electronic device 102', 'consumer electronic device 102', or 'second electronic device 102') may be provided. The electronic device 102 can process audio data provided from the first electronic device 101 and output an audio signal, which is an audible signal. The audio output device may be an electronic device such as a wireless earphone or a Bluetooth speaker.

As an example, the electronic device 101 may perform signal processing to provide an audio signal in the audible frequency band (eg, 250 Hz to 8000 Hz) to the second electronic device 102. For example, the electronic device 101 uses a predetermined encoding method to transmit audio data recorded in the internal or external memory 130 in digital form to an external electronic device 102, such as wireless earphones. It can be compressed. The first electronic device 101 may output a bitstream in which an audio signal is compressed using a predetermined encoding method.

More specifically, the first electronic device 101 may use hearing characteristic information to encode an audio signal to be transmitted to the second electronic device 102. The hearing characteristic information may be information collected through a hearing test for the user. As an example, the hearing characteristic information may be information that divides the audible frequency band into a plurality of frequency bands and defines the user's hearing ability for each of the divided frequency bands (hereinafter referred to as 'divided audible frequency band'). In this case, the hearing characteristic information may define a frequency band to which the user responds sensitively and a frequency band to which the user does not respond relatively sensitively. The fact that the user responds sensitively means that there is a high probability of hearing audio signals in a specific frequency band well even at relatively low decibels (e.g. audio signals at low volume) or recognizing even relatively low pollution or deterioration. You can. The fact that the user responds insensitively means that the probability of hearing the audio signal in the corresponding frequency band well even at a relatively high decibel (e.g., an audio signal at a loud volume) or recognizing it even when the audio signal is relatively high in pollution or deterioration is low. can do.

As an example, the first electronic device 101 classifies the audio signal in the audible frequency band into a plurality of divided audible frequency bands, and divides the audio signal into a plurality of divided audio signals (hereinafter referred to as 'segmented audio signals') in consideration of hearing characteristic information. You can determine the number of quantization bits to use to perform quantization. The first electronic device 101 may perform quantization for each divided audio signal using the number of quantization bits determined for each divided audible frequency band. The first electronic device 101 may configure quantized bit strings for each divided audible frequency band into one bit stream and transmit it to the second electronic device 102. For example, the bit stream can be generated by sequentially arranging quantized bit streams (hereinafter referred to as 'segmented quantization bit streams') generated for each divided audible frequency band according to the generation order.

In the bit stream, for example, one or more padding bits may be filled between quantization bit sequences to distinguish two consecutive quantization bit sequences. To this end, the first electronic device 101 and the second electronic device 102 may perform a synchronization procedure in advance to unify whether padding bits are used or the type of padding bits. The padding bits may make it easier for the second electronic device 102 to separate the quantized bit string for each divided audible frequency band from the bit stream.

For example, the bit stream may include information to be referred to (hereinafter referred to as 'meta information') to restore split quantized bit strings included in the bit stream. The meta information may include information that the second electronic device 102 can consider in order to allocate the number of quantization bits to each segmented quantization bit stream included in the bit stream. As an example, the meta information may include a quantization classification identifier that indicates whether quantization has been performed in consideration of the user's hearing characteristics. As an example, meta information may include information about a quantization bit allocation table referenced to allocate the number of quantization bits for quantization. For example, the information about the quantization bit allocation table indicates the quantization allocation table actually used for quantization among the quantization bit allocation tables synchronized between the first electronic device 101 and the second electronic device 102. Can include table identifier. For example, the information regarding the quantization bit allocation table may include information configuring a quantization allocation table actually used by the first electronic device 101 to allocate the number of quantization bits for quantization.

As an example, the second electronic device 102 may receive a bit stream corresponding to audio content from the first electronic device 101. The second electronic device 102 may obtain meta information included in the bit stream. The second electronic device 102 can determine the inverse quantization bit allocation table based on the meta information. As an example, the second electronic device 102 may generate or select an inverse quantization bit allocation table reflecting the user's hearing characteristics using meta information for quantization.

The second electronic device 102 may use the inverse quantization bit allocation table to determine the number of inverse quantization bits for each split quantization bit stream included in the bit stream received from the first electronic device 101. The second electronic device 102 may perform inverse quantization on the split quantization bit string using the determined number of inverse quantization bits. The second electronic device 102 may perform a restoration procedure including inverse quantization for each divided audible frequency band and merge the restored audio signals for each divided audible frequency band to generate and output one audio signal.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external

electronic devices

102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 does not execute the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

Figure 2 is a block diagram 200 of the audio module 170, according to various implementations. Referring to FIG. 2, the audio module 170 includes, for example, an audio input interface 210, an audio input mixer 220, an analog to digital converter (ADC) 230, an audio signal processor 240, and a DAC. (digital to analog converter) 250, an audio output mixer 260, or an audio output interface 270.

The audio input interface 210 is a part of the input module 150 or is configured separately from the electronic device 101 to obtain audio from the outside of the electronic device 101 through a microphone (e.g., a dynamic microphone, a condenser microphone, or a piezo microphone). An audio signal corresponding to sound can be received. For example, when an audio signal is acquired from an external electronic device 102 (e.g., a headset or microphone), the audio input interface 210 is directly connected to the external electronic device 102 through the connection terminal 178. , or the audio signal can be received by connecting wirelessly (e.g., Bluetooth communication) through the wireless communication module 192. According to one embodiment, the audio input interface 210 may receive a control signal (eg, a volume adjustment signal received through an input button) related to the audio signal obtained from the external electronic device 102. The audio input interface 210 includes a plurality of audio input channels and can receive different audio signals for each corresponding audio input channel among the plurality of audio input channels. According to one embodiment, additionally or alternatively, the audio input interface 210 may receive an audio signal from another component of the electronic device 101 (eg, the processor 120 or the memory 130).

The audio input mixer 220 may synthesize a plurality of input audio signals into at least one audio signal. For example, according to one embodiment, the audio input mixer 220 may synthesize a plurality of analog audio signals input through the audio input interface 210 into at least one analog audio signal.

The ADC 230 can convert analog audio signals into digital audio signals. For example, according to one embodiment, the ADC 230 converts the analog audio signal received through the audio input interface 210, or additionally or alternatively, the analog audio signal synthesized through the audio input mixer 220 into a digital audio signal. It can be converted into a signal.

The audio signal processor 240 may perform various processing on a digital audio signal input through the ADC 230 or a digital audio signal received from another component of the electronic device 101. For example, according to one embodiment, the audio signal processor 240 may change the sampling rate, apply one or more filters, process interpolation, amplify or attenuate all or part of the frequency band, and You can perform noise processing (e.g., noise or echo attenuation), change channels (e.g., switch between mono and stereo), mix, or extract specified signals. According to one embodiment, one or more functions of the audio signal processor 240 may be implemented in the form of an equalizer.

The DAC 250 can convert digital audio signals into analog audio signals. For example, according to one embodiment, DAC 250 may process digital audio signals processed by audio signal processor 240, or other components of electronic device 101 (e.g., processor 120 or memory 130). The digital audio signal obtained from )) can be converted to an analog audio signal.

The audio output mixer 260 may synthesize a plurality of audio signals to be output into at least one audio signal. For example, according to one embodiment, the audio output mixer 260 may output an audio signal converted to analog through the DAC 250 and another analog audio signal (e.g., an analog audio signal received through the audio input interface 210). ) can be synthesized into at least one analog audio signal.

The audio output interface 270 transmits the analog audio signal converted through the DAC 250, or additionally or alternatively, the analog audio signal synthesized by the audio output mixer 260 to the electronic device 101 through the audio output module 155. ) can be output outside of. The sound output module 155 may include, for example, a speaker such as a dynamic driver or balanced armature driver, or a receiver. According to one embodiment, the sound output module 155 may include a plurality of speakers. In this case, the audio output interface 270 may output audio signals having a plurality of different channels (eg, stereo or 5.1 channels) through at least some of the speakers. According to one embodiment, the audio output interface 270 is connected to the external electronic device 102 (e.g., external speaker or headset) directly through the connection terminal 178 or wirelessly through the wireless communication module 192. and can output audio signals.

According to one embodiment, the audio module 170 does not have a separate audio input mixer 220 or an audio output mixer 260, but uses at least one function of the audio signal processor 240 to generate a plurality of digital audio signals. At least one digital audio signal can be generated by synthesizing them.

According to one embodiment, the audio module 170 is an audio amplifier (not shown) capable of amplifying an analog audio signal input through the audio input interface 210 or an audio signal to be output through the audio output interface 270. (e.g., speaker amplification circuit) may be included. According to one embodiment, the audio amplifier may be composed of a module separate from the audio module 170.

FIG. 3 is a block diagram of a source electronic device (eg, electronic device 101 of FIG. 1 ) according to one embodiment.

Referring to FIG. 3, at least one processor 310 (e.g., processor 120 of FIG. 1) included in the first electronic device (or source electronic device) 101 encodes (or encodes) the audio signal 320. Compression) can be used to generate or output the bit stream 340. The processor 310 may use the hearing characteristic information 330 when encoding the audio signal 320.

According to one example, the processor 310 transmits the audio signal 320 in the audible frequency band to an internal memory (e.g., memory 130 in FIG. 1) or an external memory connected to an interface (e.g., interface 177 in FIG. 1). Alternatively, the input may be received through a communication network (e.g., the first network 198 or the second network 199 in FIG. 1). For example, the audio signal 320 may be an electrical signal generated by running a music player. The audio signal 320 may be a digital audio signal. For example, the audio signal 320 may be a signal modulated using pulse code modulation (PCM).

The processor 310 may include, for example, a main processor (e.g., main processor 121 in FIG. 1) or an auxiliary processor (e.g., auxiliary processor 123 in FIG. 1). In the processor 310, For example, the compression may be performed by an encoding operation on the input audio signal 320, for example, by software in the processor 310 or by a separate encoder. It can be performed in hardware through a unit.

The audio signal 320 may be encoded in the main processor 121 or the auxiliary processor 123. For example, the audio signal 320 may be encoded in legacy mode in the main processor 121. The audio signal 320 may be encoded, for example, in a separate communication chip provided in the auxiliary processor 123. Encoding of low-power mode audio signals may be performed in a separate communication chip of the auxiliary processor 123. The user may select to perform compression of the audio signal 320 in at least one of the main processor 121 or the auxiliary processor 123.

For example, the hearing characteristic information 330 may be information that reflects the user's hearing characteristics. For example, the hearing characteristic information 330 may be information indicating which frequency band of the audible frequency band the user is sensitive to. For example, the hearing characteristic information 330 may be information indicating whether each user is more or less sensitive to a specific frequency. For example, the hearing characteristic information 330 may be measured differently depending on the left ear and right ear of the same user. For example, the hearing characteristic information 330 may be measured differently depending on the user's age.

For example, the hearing characteristic information 330 may be used to detect hearing information by an application (e.g., the application 146 of FIG. 1) stored in the memory (e.g., the memory 130 of FIG. 1) of the first electronic device 101. Measurements can be performed. An application that measures a user's hearing characteristics can be called a 'hearing test app'.

When a user runs the hearing test app, a voice signal in a specific frequency band may be output to the second electronic device (or external electronic device) 102 connected to the first electronic device 101. The second electronic device 102 may be, for example, a sound output device such as earphones. The first electronic device 101 and the second electronic device 102 may be connected wired or wirelessly. For example, the first electronic device 101 may be wiredly connected to the second electronic device 102 through a connection terminal provided therein (e.g., the connection terminal 178 in FIG. 1). The first electronic device 101 may be wirelessly connected to the second electronic device 102 using, for example, Bluetooth.

When the user runs the hearing test app, the hearing test app outputs a voice signal in a specific frequency band transmitted from the first electronic device 101 to the audio output of the second electronic device 102 connected to the first electronic device 101. It can be sent to the device. An audio signal in a specific frequency band transmitted from the hearing test app may be referred to as a test signal. For example, the test signal may be output continuously or multiple times at predetermined time intervals. For example, the test signal may become increasingly louder over time. For example, the first electronic device 101 may reduce the decibel (dB) of the test signal over time and output it. The user can determine whether the test signal output from the audio output device of the second electronic device 102 can be heard. If the user determines that the test signal is not heard, he or she may touch a button displayed at a predetermined location on the display of the first electronic device 101 (e.g., the display module 160 of FIG. 1). The first electronic device 101 may record the time it takes the user to touch the button.

In the hearing test app, you can repeat the test for each divided audible frequency band by dividing a specific frequency band into predetermined intervals. For example, the predetermined interval may be an interval divided by n audible frequency bands (bands) between 250 [Hz] and 8000 [Hz]. When the audible frequency band is divided into predetermined intervals, each divided audible frequency band may be referred to as a subband (Sb). For convenience of explanation below, the divided audible frequency band is referred to as a subband.

For example, if the audible frequency band is divided into 12, each subband is divided into 1st subband (Sb #1), 2nd subband (Sb #2), in the order of bands from the low frequency band to the high frequency band. … … It may be referred to as the 11th subband (Sb #11) or the 12th subband (Sb #12). The spacing of each subband may be the same or different. Depending on the time it takes to determine that the user cannot hear the voice signal in a specific frequency band, the first electronic device 101 may record the user's hearing measurement results in each subband. For example, the hearing measurement results may be divided into three levels: very good, good, and average.

For example, if the user determines that the test signal cannot be heard after 5 seconds in each subband and clicks the button displayed on the display 160, the first electronic device 101 determines that the user can hear the test signal at a relatively low decibel in the subband. It can be predicted that the sound of can be heard. In this case, the first electronic device 101 may record the hearing measurement result in the subband as 'very good'.

For example, if the user determines that the test signal cannot be heard after 3 seconds in a specific subband and clicks the button displayed on the display 160, the first electronic device 101 allows the user to hear the test signal in the corresponding subband. It can be predicted that sounds of up to an audible decibel level can be heard. In this case, the first electronic device 101 may record the hearing measurement result in the subband as 'good'.

For example, if the user determines that the test signal cannot be heard after 1 second in a specific subband and clicks the button displayed on the display 160, the first electronic device 101 determines that the user can hear the test signal at a relatively high decibel level in the subband. It can be predicted that only sounds can be heard. In this case, the first electronic device 101 may record the hearing measurement result in the subband as 'normal'. However, it is not limited to this, and if necessary, the first electronic device 101 may record the hearing measurement results in more detail or set a different hearing measurement method.

Hearing measurement results measured in the hearing test app may be stored in the memory 130 in the form of a database (DB). The user may additionally perform hearing measurement through the hearing test app. The user's additional hearing measurement results may be stored in the memory 130 in the form of the database. When audiometry results from multiple hearing measurements are stored in a database, the processor 120 may store the average value of the plurality of audiometry results obtained in the database. The average value of the plurality of stored hearing measurement results may be stored as the user's hearing characteristic information 330. However, the present invention is not limited to this, and the hearing measurement results according to a hearing measurement test recently performed by the user may be stored as the user's hearing characteristic information 330.

As an example, the user may include information on values arbitrarily entered for hearing in each frequency band. In addition to the hearing measurement results measured when the user runs the hearing test app, the processor 310 can store information about hearing for each frequency band input by the user as hearing characteristic information 330. .

The stored hearing characteristic information 330 of the user may be referred to as hearing data.

The processor 310 may perform a compression process (or encoding process) on the input audio signal 320. In performing the compression process, the processor 310 may use hearing data stored in the memory 130. The compression process may include performing quantization on the input audio signal 320. The quantization operation may include an operation of allocating the number of quantization bits for performing quantization for each subband. The operation of allocating the number of quantization bits may include determining a bit allocation table. The bit allocation table can define information about the number of quantization bits to be allocated for each subband. The operation of allocating the number of quantization bits may include the processor 310 determining the presence or absence of the hearing data stored in the memory 130 and selecting or generating a bit allocation table accordingly. The processor 310 can determine the number of quantization bits corresponding to the audio signal for each subband using the bit allocation table.

For example, if there is hearing data, the processor 310 may select or create a bit allocation table that reflects the hearing data. The bit allocation table reflecting the hearing data may, for example, allocate a relatively large number of bits to the subband to which the user responded as sensitive, and allocate a relatively small number of bits to the subband to which the user responded as insensitive. The bit allocation table in which the hearing data is reflected can be referred to as an 'attribute bit allocation table'. The processor 310 can perform quantization on an audio signal using the characteristic bit allocation table. Hereinafter, the characteristic bit allocation table is referred to as the 'first bit allocation table'.

For example, if there is no hearing data, the processor 310 may select or generate a normalized bit allocation table that does not consider hearing data. The standard bit allocation table may exist, for example, in the form of a database in the memory 130. The standard bit allocation table may be, for example, a bit allocation table that takes psychoacoustics into account. For example, the standard bit allocation table may allocate a relatively large number of quantization bits to a subband to which a statistically majority response was sensitive, and a relatively small number of quantization bits to a subband to which a majority statistically responded as insensitive. Since statistically, an unspecified number of people often respond as being sensitive to low-band subbands, the standard bit allocation table can be prepared to allocate a relatively large number of quantization bits to low-band subbands. However, it is not limited to this, and the standard bit allocation table may be, for example, a table in which a relatively large number of bits are allocated to the mid-band or high-band subbands rather than the low-band, depending on the characteristics of the samples for generating the standard bit allocation table. . Hereinafter, the standard bit allocation table is referred to as the 'second bit allocation table'.

The processor 310 may generate and output a bit stream 340 as a result of performing a compression process on the input audio signal 320. The processor 310 may transmit the bit stream 340 to the second electronic device 102 through a communication module (eg, the communication module 190 of FIG. 1). The bit stream 340 may be transmitted to the second electronic device 102 using, for example, Bluetooth. The bit stream may be transmitted to the second electronic device 102 through, for example, an advanced audio distribution profile (A2DP).

FIG. 4 is a flowchart showing an operation in which the first electronic device 101 quantizes an audio signal, according to an embodiment.

Referring to FIG. 4, in operation 410, the first electronic device 101 may obtain the number of quantization bits based on hearing characteristics for each divided audible frequency band. The divided audible frequency band may mean, for example, a specific frequency band in which the audible frequency band is divided into a predetermined number. The divided audible frequency band may also be referred to as a subband.

The first electronic device 101 uses a first bit allocation table according to hearing characteristics (e.g., hearing characteristic information 330 of FIG. 3 or hearing data of FIG. 3) to obtain the number of quantization bits based on hearing characteristics. Quantization can be performed using . For example, the first allocation bit table may be a table for the number of quantization bits to perform quantization by reflecting hearing characteristics for each divided audible frequency.

The first electronic device 101 may perform quantization for each segmented audible frequency band using a second bit allocation table, which is a standard bit allocation table unrelated to hearing characteristics.

The first electronic device 101 may obtain the number of quantization bits to perform quantization for each divided audible frequency using either the first bit allocation table or the second bit allocation table.

In operation 420, the first electronic device 101 may perform quantization using the number of quantization bits corresponding to the divided audio signal, which is an audio signal for each divided audible frequency band. The number of quantization bits may mean the number of quantization bits for each divided audible frequency obtained by the first bit allocation table or the second bit allocation table generated in operation 410. For example, the first electronic device 101 may perform quantization on a segmented audio signal according to the number of quantization bits.

In operation 430, the first electronic device 101 may generate a quantized audio signal as a bit stream. The first electronic device 101 may transmit the bit stream to the second electronic device 102. As a result of the first electronic device 101 performing quantization in operation 420, a bit stream may be generated. The first electronic device 101 may set a wireless channel to connect the second electronic device 102 to transmit the generated bit stream. The first electronic device 101 may transmit a bit stream to the second electronic device 102 through the wireless channel.

FIG. 5 is a control flowchart for the first electronic device 101 to quantize an audio signal, according to an embodiment.

Referring to FIG. 5, the first electronic device 101 may receive an audio signal in operation 510. For example, the audio signal may be an audio signal generated by running a music player. The audio signal may include a digital audio signal. The audio signal may be an audio signal modulated using a pulse code method.

In operation 520, the first electronic device 101 may determine whether hearing data exists. For example, the hearing data may be information on the user's hearing characteristics measured by a hearing measurement app of the first electronic device 101. For example, the hearing data may be hearing characteristic information arbitrarily input by the user rather than measured using the hearing measurement app. For example, the hearing data may be information indicating which subband of the audible frequency band the user is sensitive to. The hearing data may be either an average of the results measured using a hearing measurement app or the user's most recent measurement results using a hearing measurement app.

If hearing data exists, the first electronic device 101 may select or generate a first bit allocation table in operation 530. For example, the first bit allocation table may be a bit allocation table that reflects the user's hearing characteristics. The first bit allocation table may be a bit allocation table in which a relatively large number of bits are allocated to subbands to which users are sensitive. For example, the first bit allocation table may be a result of assigning a predetermined weight to each subband in which the user's hearing is measured and calculating the bits to be allocated to each subband through calculation with the weights.

Although not shown, the first electronic device 101 may analyze hearing data. As an example, the first electronic device 101 may analyze which band among the divided audible frequency bands is sensitive in relation to the user's hearing characteristics indicated by the hearing data.

For example, if the first electronic device 101 analyzes the hearing data and obtains a result that the user is sensitive to audio signals in the low frequency range of the divided audible frequency band, the first electronic device 101 stores the memory 130. You can select the second bit allocation table stored in . The second bit allocation table may mean, for example, the standard bit allocation table of FIG. 3. As a result, the first electronic device 101 does not need to select or generate a first bit allocation table reflecting hearing data, thereby increasing computational efficiency or transmission efficiency.

Although not shown, the first electronic device 101 may compare the first bit allocation table and the second bit allocation table and then determine which bit allocation table to use to perform quantization.

For example, a characteristic bit allocation table generated based on hearing data of a user sensitive to low-band subbands may be substantially the same as a standard bit allocation table. For example, the first electronic device 101 may compare the allocation bits for each subband in the characteristic bit allocation table with the allocation bits for each subband in the standard bit allocation table. For example, the first electronic device 101 may compare a predetermined weight for each subband according to the user's hearing data with a predetermined weight used to generate a standard bit allocation table.

The first electronic device 101 may determine whether there is a substantial difference between the first bit allocation table, which is a characteristic bit allocation table, and the second bit allocation table, which is a standard bit allocation table. If there is a substantial difference between the first bit allocation table and the second bit allocation table, the first electronic device 101 may determine a quantization bit using the first bit allocation table in operation 530.

If hearing data does not exist in the first electronic device 101 or if there is substantially no difference between the first bit allocation table and the second bit allocation table, the electronic device 101 selects the second bit in operation 540. Quantization bits can be determined by selecting an allocation table. Even if hearing data exists, if the first electronic device 101 analyzes that the hearing data is sensitive to the divided audible band in the low frequency range, the first electronic device 101 determines the second bit allocation table in consideration of computational efficiency or transmission efficiency. Quantization can be performed by selecting . The second bit allocation table may be a standard bit allocation table that does not reflect the user's individual hearing characteristics.

In operation 550, the first electronic device 101 may perform compression on each segmented audio signal using the number of quantization bits determined for each segmented audible frequency band. As an example, when the first electronic device 101 quantizes each divided audio signal using the first bit allocation table, a relatively large number of bits are recorded in the subband to which the user responded as sensitive (e.g., a subband in the mid-band or high-band). A quantized bit string of can be generated. For example, when the first electronic device 101 quantizes each segmented audio signal using the second bit allocation table, a quantized bit stream with a relatively large number of bits may be generated in the low-band subband.

In operation 560, the first electronic device 101 may configure a bit stream to include quantized bit streams generated by performing quantization for each segmented audio signal. The first electronic device 101 may transmit a bit stream to the second electronic device 102. The first electronic device 101 may transmit a bit stream to the second electronic device 102 through a communication module (eg, the communication module 190 of FIG. 1). For example, the first electronic device 101 may transmit a bit stream using a Bluetooth method through a wireless communication module (e.g., the wireless communication module 192 of FIG. 1).

The first electronic device 101 may establish a wireless channel with the second electronic device 102 through the wireless communication module 192. The first electronic device 101 may transmit a bit stream to the second electronic device 102 through the established wireless channel. The first electronic device 101 may transmit the bit stream to the second electronic device 102 through, for example, an advanced audio distribution profile (A2DP).

Although not shown, when the first electronic device 101 is connected to the second electronic device 102 through a wireless channel, it can share information necessary to quantize or dequantize an audio signal. Each shared information can be synchronized so that quantization or inverse quantization of the audio signal can be easily performed. As an example, the first electronic device 101 may share a second bit allocation table used to perform quantization of an audio signal with the second electronic device 102. The second electronic device 102 may perform inverse quantization for each divided audible frequency band using the shared second bit allocation table.

The order of each operation shown in FIG. 5 is not limited to that shown, and the order may be changed as needed. For example, the operation of the first electronic device 101 setting up a wireless channel with the second electronic device 102 may be performed before operation 560.

FIG. 6 is a block diagram of the second electronic device 102 according to one embodiment.

Referring to FIG. 6, the second electronic device 102 restores the bit stream 620 (e.g., the bit stream 340 of FIG. 3) received from the first electronic device 101 into an audio signal 640. You can.

The second electronic device 102 may include, for example, a sound output device such as a speaker or earphone. The second electronic device 102 may be connected to the first electronic device 101 by wire or wirelessly. For example, the second electronic device 102 may be connected by wire through a connection terminal provided in the first electronic device 101 (e.g., the connection terminal 178 in FIG. 1). The second electronic device 102 may be wirelessly connected to the first electronic device 101 using, for example, Bluetooth. When the second electronic device 102 is connected to the first electronic device 101 through a wireless link, it can share information necessary for quantization or inverse quantization of the audio signal.

The second electronic device 102 may set a wireless channel for connection with the first electronic device 101. The second electronic device 102 may respond to the wireless channel setup request of the first electronic device 101.

A bit stream 620 may be input to the processor 610 included in the second electronic device 102. The bit stream 620 may be a signal obtained by quantizing and compressing an audio signal 320 (e.g., the audio signal 320 of FIG. 3) in the first electronic device 101.

The operations performed by the processor 610 may correspond in whole or in part to the operations performed by the processor 310 of the first electronic device 101 (eg, the processor 310 of FIG. 3). The processor 610 may, for example, perform inverse quantization on the input bit stream 620. Decompression, including the inverse quantization, may be performed, for example, in software by the processor 610 or in hardware through a separate decoder unit. The processor 610 may be implemented as, for example, an audio signal processor (eg, the audio signal processor 240 of FIG. 2).

The processor 610 may analyze information stored in the input bit stream 620. The information stored in the bit stream 620 may include, for example, information about hearing measurement results measured by the first electronic device 101 or bit allocation information used when performing quantization of an audio signal. For example, the processor may analyze the information in the bit stream 620 and then request additional information needed to perform inverse quantization on the input bit stream 620 from the first electronic device 101.

The processor 610 may use the hearing characteristic information 630 when restoring the bit stream 620 into an audio signal. For example, the hearing characteristic information 630 may be information that reflects the user's hearing characteristics. For example, the hearing characteristic information 630 may include information measured through a hearing test app of the first electronic device 101. The hearing characteristic information 630 may include, for example, an average value of a plurality of audiometry results or a hearing measurement result most recently measured by the user. For example, the hearing characteristic information 630 may include information about values arbitrarily entered by the user for hearing in each frequency band.

As an example, the hearing characteristic information 630 may be information stored in the bit stream 620 received from the first electronic device 101. The processor 610 may analyze the received bit stream 620 and obtain hearing characteristic information 630 stored in the bit stream 620. The processor 610 may analyze the received bit stream 620 to obtain bit allocation information used when quantizing the audio signal 320. As an example, the hearing characteristic information 630 may be information separately acquired from the first electronic device 101 through a wireless channel. As an example, the hearing characteristic information 630 may be information stored by the second electronic device 102 in the form of a database.

The processor 610 may perform inverse quantization on the bit stream 620 received from the first electronic device 101 in the process of restoring the bit stream 620 to the audio signal 640. The inverse quantization may be performed, for example, using a bit allocation table used during quantization in the first electronic device 101. The processor 610 may select or generate a bit allocation table to be used for inverse quantization. The bit allocation table may include, for example, a first bit allocation table or a second bit allocation table.

For example, if hearing data exists in the second electronic device 102, the bit stream 620 is inversely quantized using a bit allocation table reflecting the hearing data (e.g., the characteristic bit allocation table in FIG. 3) to produce an audio signal. It can be restored with (640). If hearing data does not exist in the second electronic device 102, the second electronic device 102 may request the hearing data from the first electronic device 101. When the second electronic device 102 receives hearing data from the first electronic device 101, the second electronic device 102 may generate a first bit allocation table. The second electronic device 102 may request information about the bit allocation table used by the first electronic device 101 to quantize the audio signal 320.

For example, if hearing data does not exist in the second electronic device 102 or hearing data is not received even though the hearing data is requested from the first electronic device 101, the second bit allocation table (e.g., in FIG. 3 The bit stream 620 can be inversely quantized using the second bit allocation table and restored into the audio signal 640. The standard bit allocation table may be information stored in the memory of the second electronic device 102 or information received by the first electronic device 101.

The processor 610 may select either the first bit allocation table or the second bit allocation table and perform inverse quantization using the number of inverse quantization bits corresponding to the bit stream 620 for each audible frequency band. An audio signal 640 can be generated by performing inverse quantization for each divided audible frequency band and then combining them into one signal.

The generated audio signal 640 may be, for example, an audio signal modulated using a pulse code modulation method. The audio signal 640 may be converted into an analog signal by an audio module provided in the second electronic device 102 and then output through an audio output device.

FIG. 7 is a control flowchart for the second electronic device 102 to inverse quantize an audio signal, according to an embodiment.

Referring to FIG. 7 , the second electronic device 102 may analyze the bit stream received from the first electronic device 101 in operation 710. The second electronic device 102 may receive a bit stream through a wireless channel established with the first electronic device 101.

Information that the second electronic device 102 can obtain by analyzing the bit stream may be referred to as ‘bit stream analysis information.’ For example, the bit stream analysis information may include information about the number of quantization bits used when the first electronic device 101 performs compression on an audio signal. For example, the bit stream analysis information may include a bit allocation table used when the first electronic device 101 performs compression on an audio signal. For example, the bit stream analysis information may include hearing measurement information for each user's divided audible band required to generate the bit allocation table.

In operation 720, the second electronic device 102 may obtain the number of inverse quantization bits for each divided audible frequency band included in the bit stream. For example, the number of inverse quantization bits for each divided audible frequency band may correspond to the number of quantization bits for each divided audible frequency band used when the first electronic device 101 performs compression for each divided audio signal.

In operation 730, the second electronic device 102 may perform inverse quantization on the bit stream for each audible frequency band using the obtained number of inverse quantization bits. For example, the inverse quantization may be sequentially performed on segmented audio signals for each audible frequency band included in the bit stream. The inverse quantization may be performed, for example, using a first bit allocation table that reflects the user's hearing characteristics or a second bit allocation table that does not reflect the user's hearing characteristics. As a result of performing the inverse quantization, an audio signal can be obtained. For example, the audio signal may be a signal modulated using a pulse code modulation method. For example, the audio signal may be an audio signal generated by the first electronic device 101 using a signal from a music player.

The second electronic device 102 may output the generated audio signal in operation 740. The second electronic device 102 may synthesize segmented audio signals for the inverse quantized bit stream for each segmented frequency band into one audio signal. Since the pulse code modulation audio signal is a digital audio signal, the synthesized audio signal can be converted into an analog signal through a digital signal processor or DAC (digital to analog converter) module provided in the second electronic device 102. there is. The audio signal converted into the analog signal can be output to an audio output device provided in the second electronic device 102.

FIG. 8 is a control flowchart that the second electronic device 102 performs to decompress an audio signal, according to an embodiment.

Referring to FIG. 8, the second electronic device 102 may receive a bit stream from the first electronic device 101 in operation 810. The second electronic device 102 may establish a wireless channel with the first electronic device 101 through a communication module, or may be connected wirelessly by approving a wireless channel setting request from the first electronic device 101. The second electronic device 102 may receive a bit stream from the first electronic device 101 through a wireless channel established with the first electronic device 101.

Although not shown, when the second electronic device 102 is first connected to the first electronic device 101, information necessary for quantizing or inverse quantizing an audio signal may be shared.

For example, the information may include data about a second bit allocation table that does not reflect the user's hearing characteristics. For example, the information may include data indicating weights that can generate a first bit allocation table for each divided audible frequency.

The second electronic device 102 may analyze information about the received bit stream in operation 820. The second electronic device 102 may analyze meta information included in the bit stream. The meta information may include information to be referenced to inverse quantize the bit stream.

As an example, the meta information may include a first identifier indicating whether the bit stream was generated by quantization considering hearing data. When the meta information includes the first identifier, the second electronic device 102 inversely quantizes the quantized data for each divided frequency band included in the bit stream using the first bit allocation table held by the second electronic device 102. You can determine the number of inverse quantization bits to use. For example, the first identifier may consist of one bit value (1 bit). If the bit value corresponding to the first identifier is '1', the second electronic device 102 can recognize that the bit stream was generated by quantization considering hearing data. If the bit value corresponding to the first identifier is '0', the second electronic device 102 can recognize that the bit stream was generated by quantization unrelated to hearing data.

As an example, meta information may include information about a first bit allocation table applied to quantize the bit stream. The information about the first bit allocation table may include, for example, a second identifier indicating the first bit allocation table used for quantization in the encoder. If the information about the first bit allocation table includes the second identifier, the second electronic device 102 performs inverse quantization among one or more first bit allocation tables owned by the second electronic device 102 using the second identifier. The first bit allocation table used for this purpose can be determined. The second electronic device 102 may use the determined first bit allocation table to determine the number of inverse quantization bits to be used to inverse quantize the quantized data for each divided frequency band included in the bit stream.

As an example, meta information may include information about a first bit allocation table applied to quantize the bit stream. The information about the first bit allocation table may be, for example, data that can configure the first bit allocation table. If the information about the first bit allocation table includes data that can configure the first bit allocation table, the second electronic device 102 can generate the first bit allocation table using the data. there is. The second electronic device 102 may use the generated first bit allocation table to determine the number of inverse quantization bits to be used to inverse quantize the quantized data for each divided frequency band included in the bit stream.

As an example, the second electronic device 102 may analyze information about the transmission environment included in the bit stream. The transmission environment may analyze, for example, information on whether the first electronic device 101 and the second electronic device 102 can smoothly receive a bit stream or wireless signal transmitted and received through a set wireless channel. . Depending on the transmission environment analyzed by the first electronic device 101 or the second electronic device 102, the capacity of the bit stream to be transmitted from the first electronic device 101 to the second electronic device 102 may be determined. . For example, if the transmission environment is smooth, the capacity of the bit stream that the first electronic device 101 needs to transmit to the second electronic device 102 may be relatively small. For example, if the transmission environment is not smooth, the capacity of the bit stream that the first electronic device 101 must transmit to the second electronic device 102 may be relatively large.

In operation 830, the second electronic device 102 may analyze meta information of the received bit stream and determine whether the bit stream was generated through quantization considering hearing data.

If the bit stream is generated with the number of quantized bits considering the hearing data, in operation 840, the second electronic device 102 divides the bit stream using the first bit allocation table for each divided audio signal included in each audible frequency band. The number of inverse quantization bits to perform inverse quantization can be determined.

If the bit stream is generated with a quantization bit number unrelated to the hearing data, in operation 850, the second electronic device 102 divides the bit stream using the second bit allocation table and divides the bit stream into each divided audio signal included for each audible frequency band. For this reason, the number of inverse quantization bits to perform inverse quantization can be determined.

In operation 860, the second electronic device 102 performs inverse quantization on each segmented audio signal included in the bit stream for each segmented audible frequency band using the number of bits to perform inverse quantization determined for each segmented audible frequency band. You can.

In operation 870, the second electronic device 102 may generate an audio signal using segmented audio signals obtained for each segmented audible frequency band as a result of performing inverse quantization. The second electronic device 102 may synthesize segmented audio signals obtained by performing inverse quantization for each segmented audible frequency band. The second electronic device 102 may output synthesized audio signals for each divided audible frequency band. The audio signal may be converted into an analog audio signal and output to an audio output device provided in the second electronic device 102.

FIG. 9 is a block diagram of the encoder 700 of the first electronic device 101, according to one embodiment.

Referring to FIG. 9, the encoder 900 includes a transient detection unit 910, a domain transformation unit 920, a signal classification unit 930, and a bit allocation selection. It may include a bit allocation selection unit 940, a quantization unit 950, and a lossless coder unit 960. Each component may be integrated into at least one module and implemented with at least one processor (eg, processor 310 in FIG. 3). Additionally, each of the above components may be added or omitted as needed.

For example, the transient detection unit 910 may analyze the input audio signal 320 to detect a section showing transient characteristics and generate transient signaling information for each frame in response to the detection result. there is. The audio signal may be generated, for example, by executing a music player of the first electronic device 101. The transient detection unit 910 may first determine whether the frame is a transient frame, and then secondarily perform verification on the current frame determined to be a transient frame. The transient signaling information may be included in a bit stream through a multiplexer (not shown) and provided to the domain converter 920.

The domain converter 920 may determine the window size used for conversion according to the detection result of the transient section and perform time-frequency conversion based on the determined window size. The transformation may be performed, for example, using a Fourier transform method. The Fourier transform method may include, for example, a discrete Fourier transform (DFT) or a fast Fourier transform (FFT).

For example, in the case of a subband in which a transient section is detected, a short window can be applied, and in the case of a subband in which a transient section is not detected, a long window can be applied. Alternatively, a short section window may be applied to a frame including a transient section.

The signal classifier 930 may divide the audio signal in the frequency domain into audible frequency bands at predetermined intervals. The audible frequency band divided at predetermined intervals can be referred to as a subband. For example, the predetermined interval may be an interval divided by n audible frequency bands. For example, if the audible frequency band is divided into n, in order from low frequency band to high frequency band, first subband (Sb #1), second subband (Sb #2),... … Alternatively, it may be named nth subband (Sb #n). The spacing of each subband may be the same or different. As an example, n divided subbands may be sequentially arranged in the frequency domain. The n subbands may be arranged in series in the order from low-band subbands to high-band subbands.

The bit allocation selection unit 940 can allocate the number of quantization bits for each subband. The bit allocation selection unit 940 selects the number of quantization bits for each frequency band within the limit that the quantization noise that exists when quantized according to the masking threshold calculated by the processor 120 does not exceed the masking threshold. can be assigned.

For example, the bit allocation selection unit 940 may select a bit allocation table and allocate the number of quantization bits for each divided audible frequency band. The bit allocation table may include, for example, a first bit allocation table or a second bit allocation table.

For example, the first bit allocation table may be a bit allocation table that takes into account the user's hearing characteristics. For example, if the user is sensitive to high frequencies as a result of hearing measurement, the first bit allocation table may be a table in which a relatively large number of quantization bits are allocated to frequencies in the high frequencies. For example, if the user is sensitive to the mid-range as a result of the hearing measurement, the first bit allocation table may be a table in which a relatively large number of quantization bits are allocated to the frequencies of the mid-range. For example, if the user is sensitive to low-band frequencies as a result of hearing measurement, the first bit allocation table may be a table in which a relatively large number of quantization bits are allocated to low-band frequencies.

The second bit allocation table may be, for example, a table in which a relatively large number of bits are allocated to a divided audible frequency band that a statistically significant majority responded as being sensitive. For example, since statistically many people respond as being sensitive to low-band frequencies, the second bit allocation table may be a table in which a relatively large number of quantization bits are generally allocated to low-band frequencies.

When the bit allocation selection unit 940 is allocated the number of bits according to the first bit allocation table, the number of quantization bits can be allocated in the following manner. First, the bit allocation selection unit 940 can allocate the number of quantization bits for each subband using the Norm value. The Norm value may be a value indicating energy for each subband. As an example, if it is most sensitive to the n-th subband (Sb #n) and, in reverse order, most insensitive to the first sub-band (Sb #1), the bit allocation selection unit 940 is the n-th subband (Sb #n). ) may be assigned the most weight, and the least weight may be assigned to the first subband (Sb #1). For example, the bit allocation selection unit 940 may sequentially allocate the number of quantization bits starting from the subband with the largest Norm value. That is, the largest number of quantization bits is allocated to the n-th subband (Sb #n) with the highest priority, and the number of bits allocated to the n-th subband (Sb #n) is reduced from the total number of allocated bits. Next, the number of quantization bits equal to the corresponding weight can be assigned to the high-priority subband. By repeating this process, bits can be repeatedly allocated until the total number of bits is exhausted.

The bit allocation selection unit 940 may determine the number of quantization bits to be finally allocated by limiting the number of bits allocated to each subband so that it does not exceed the allowable number of bits (eg, the total number of bits to be transmitted). For example, the number of quantization bits to be allocated may be affected by the communication environment between the first electronic device 101 and the second electronic device 102.

The quantization unit 950 may quantize the audio signal with the number of quantization bits allocated to each subband according to the bit allocation table selected by the bit allocation selection unit 940. For example, the quantization unit 950 may perform quantization through calculation according to the number of quantization bits allocated to each subband. The quantization unit 950 may perform quantization on the corresponding segmented audio signal according to the number of quantization bits for each subband.

The quantization unit 950 can quantize the Norm value for each subband. At this time, the Norm value can be quantized in various ways, such as vector quantization, scalar quantization, TCQ, and LVQ (Lattice vector quantization). The quantization unit 950 may additionally perform lossless coding to further improve coding efficiency.

The lossless encoding unit 960 can losslessly encode the result quantized by the quantization unit 950. For example, Trellis Coded Quantizer (TCQ), Uniform Scalar Quantizer (USQ), Factorial Pulse Coder (FPC), Analog Vector Quantizer (AVQ), Predictive Vector Quantizer (PVQ), or a combination thereof, and each quantization unit 950. The corresponding lossless encoding unit 960 can be used. Additionally, various encoding techniques can be applied depending on the environment in which the codec is installed or the user's needs. Information about the audio signal encoded in the lossless encoder 960 may be included in the bit stream 340.

The lossless encoding unit 960 can hierarchically perform lossless encoding on the audio signal quantized in the quantization unit 950. In the lossless encoding unit 960, for example, lossless encoding is performed using groups grouped with codes corresponding to the most significant bits as the highest layer, and groups grouped with codes corresponding to the lowest bits are sequentially used as the lower layer. Lossless encoding is possible. For example, the lossless encoder 960 may perform encoding on the audio signal by considering overlapping values and frequencies for each subband.

The bit stream 340 encoded in the lossless encoder 960 may be transmitted to the second electronic device 102.

FIG. 10 is a block diagram of a portion 1000 of an encoder of a first electronic device according to an embodiment.

Referring to FIG. 10, parts of the encoder of FIG. 9 related to the present disclosure are shown in more detail. Accordingly, the components of FIG. 10 may correspond in whole or in part to the components of FIG. 9. Detailed descriptions of overlapping components are omitted.

The audio signal converted to the frequency domain in the domain conversion unit 1020 may be input to the signal classification unit 1030. The signal classification unit 1030 may correspond to the signal classification unit 930 of FIG. 9 (eg, the signal classification unit 930 of FIG. 7). The signal classification unit 1030 may divide the input audio signal into predetermined divided audible frequency bands. The audio signal is, for example, a first sub-band (Sb #1), a second sub-band (Sb #2), a third sub-band (Sb #3), . … It can be divided into n subbands of the nth subband (Sb #n). As an example, n divided subbands may be sequentially arranged in the frequency domain. The n subbands may be arranged in series in the order from low-band subbands to high-band subbands.

Hearing data (eg, hearing data in FIG. 3) may be stored in the memory 1080 in the form of a database 1081. The hearing data can be expressed in <Table 1>.

서브 밴드(Sb)Subband (Sb)	청력 측정 결과(Hearing loss)Hearing loss	가중치 부여(weight)Weighting
제1 서브 밴드(Sb #1)1st subband (Sb #1)	매우 좋음/좋음/보통(HL #1)Very Good/Good/Average (HL #1)	w_a/w_b/w_c(w#1)w _a /w _b /w _c (w#1)
제2 서브 밴드(Sb #2)Second subband (Sb #2)	매우 좋음/좋음/보통(HL #2)Very Good/Good/Average (HL #2)	w_a/w_b/w_c(w#2)w _a /w _b /w _c (w#2)
제3 서브 밴드(Sb #3)Third subband (Sb #3)	매우 좋음/좋음/보통(HL #3)Very Good/Good/Average (HL #3)	w_a/w_b/w_c(w#3)w _a /w _b /w _c (w#3)
............	............	............
제n 서브 밴드(Sb #n)nth subband (Sb #n)	매우 좋음/좋음/보통(HL #n)Very Good/Good/Average (HL #n)	w_a/w_b/w_c(w#n)w _a /w _b /w _c (w#n)

<Table 1> is a table that divides the audible frequency band into n subbands (Sb) and summarizes the hearing measurement results for each subband (Sb). Hearing loss can be recorded for each subband through a hearing measurement app (e.g., the hearing measurement app in FIG. 3). Hearing measurement results may indicate either 'very good', 'good' or 'average'. The audiometry results of the first subband (Sb #1) are HL #1, the audiometry results of the second subband (Sb #2) are HL #2, and the audiometry results of the third subband (Sb #3) are It can be expressed as HL #3, and the audiometry results of the nth subband (Sb #n) can be expressed as HL #n.

Weights may be set differently depending on the hearing measurement results for each subband. The weight may be determined according to the hearing loss. For example, if the hearing measurement result is 'very good', the weight of w _a may be determined. For example, if the hearing measurement result is 'good', the weight of w _b may be determined. For example, if the hearing measurement result is 'normal', the weight of w _c may be determined. The w _a, w _{b, and} w _c may have a relationship of w _a ≥w _b ≥w _c .

The user may directly input the audiometry results for each segmented audible frequency without performing audiometry.

According to the audiometry results, hearing data considering the hearing characteristics of each user may be stored in the memory 1080 as a database 1081.

A processor (eg, processor 310 in FIG. 3) may generate a characteristic bit allocation table by considering hearing data. <Table 2> shows an example in which the processor 310 generates a characteristic bit allocation table by reflecting hearing data.

서브 밴드(Sb)Subband (Sb)	기본 비트 할당 값 (basis bit allocation value)Default bit assignment value (basis bit allocation value)	가중치 부여 (weight)weighting (weight)	특성 비트 할당 값 (characterized bit allocation value)Characteristic Bit Assignment Value (characterized bit allocation value)
제1 서브 밴드(Sb #1)1st subband (Sb #1)	A₁ A ₁	w_a/w_b/w_c(w#1)w _a /w _b /w _c (w#1)	A₁'A ₁ '
제2 서브 밴드(Sb #2)Second subband (Sb #2)	A₂ A ₂	w_a/w_b/w_c(w#2)w _a /w _b /w _c (w#2)	A₂'A ₂ '
제3 서브 밴드(Sb #3)Third subband (Sb #3)	A₃ A ₃	w_a/w_b/w_c(w#3)w _a /w _b /w _c (w#3)	A₃'A ₃ '
............	............	............	............
제n 서브 밴드(Sb #n)nth subband (Sb #n)	A_n A _n	w_a/w_b/w_c(w#n)w _a /w _b /w _c (w#n)	A_n'A _n '

<Table 2> divides the frequency band into m subbands (Sb), and assigns a weight to the basic bit allocation value for each subband according to the audiometry results of each subband (Sb) to determine the characteristic bit value. This is a table deriving the allocation value. Unlike in <Table 1>, where the audible frequency band is divided into n subbands, in <Table 2>, the audible frequency band can be divided into m subbands. m and n may be the same or different. The table composed of the characteristic allocation bit values may be referred to as the first bit allocation table.

The weight for each subband (Sb) is w _a, w _b, orIt can be determined by any one value of w _c .

Characterized bit allocation values can be derived by assigning weights for each subband (Sb) according to the user's hearing measurement results to each basic bit allocation value for each subband (Sb). The characterized bit allocation value may be, for example, a result derived by multiplying the bit allocation value by a weight. However, it is not limited to this, and the characteristic allocation bit value can be derived by calculating the bit allocation value and weight in various defined ways.

For example, even if the basic bit allocation values A ₁ and A ₃ of the first subband (Sb #1) and the second subband (Sb #2) are the same, the hearing loss of the first subband (Sb #1) If the measurement result (HL #1) is recorded as 'very good' and the audiometry result (HL #2) of the second subband (Sb #2) is recorded as 'average', each characteristic bit allocation value A ₁ ' and A ₃ ' can be created to have the relationship A ₁ '≥A ₃ '.

Data that organizes the characteristic bit allocation values for each subband in a table can be referred to as the first bit allocation table. The generated first bit allocation table may be stored in the form of a database 1081 in the memory 1080. Additionally, if necessary, the processor 310 may generate a plurality of first allocation tables through calculation.

서브 밴드(sb)Subband (sb)	기본 비트 할당 값 (basis bit allocation value)Default bit assignment value (basis bit allocation value)	가중치 부여 (weight)weighting (weight)	표준 비트 할당 값 (normalized bit allocation value)Standard bit assignment value (normalized bit allocation value)
제1 서브 밴드(Sb #1)1st subband (Sb #1)	A₁ A ₁	w_a/w_b/w_c(w#1)w _a /w _b /w _c (w#1)	A₁''A ₁ ''
제2 서브 밴드(Sb #2)Second subband (Sb #2)	A₂ A ₂	w_a/w_b/w_c(w#2)w _a /w _b /w _c (w#2)	A₂'' _A2 ''
제3 서브 밴드(Sb #3)Third subband (Sb #3)	A₃ A ₃	w_a/w_b/w_c(w#3)w _a /w _b /w _c (w#3)	A₃'' _A3 ''
............	............	............	............
제n 서브 밴드(Sb #n)nth subband (Sb #n)	A_n A _n	w_a/w_b/w_c(w#n)w _a /w _b /w _c (w#n)	An''An''

<Table 3> divides the frequency band into m subbands (Sb), and weights the basic bit allocation value for each subband according to the audiometry results of each subband (Sb) to obtain the normalized bit value. This is a table deriving the allocation value. The table composed of the standard allocation bit values may be referred to as a second bit allocation table.

The weight for each subband (Sb) is w _a, w _b, orIt can be predetermined as any one value among w _c .

A standardized bit allocation value (characterized bit allocation value) can be derived by assigning a predetermined weight for each subband (Sb) according to the user's hearing measurement results to each basic bit allocation value for each subband (Sb). there is. The characterized bit allocation value may be, for example, a result derived by multiplying the basic bit allocation value by a weight. However, it is not limited to this, and the characteristic allocation bit value can be derived by calculating the bit allocation value and weight in various defined ways.

For example, even if the basic bit allocation values A ₁ and A ₃ of the first subband (Sb #1) and the second subband (Sb #2) are the same, the hearing loss of the first subband (Sb #1) If the measurement result (HL #1) is pre-stored as 'very good' and the hearing measurement result (HL #2) of the second sub-band (Sb #2) is pre-stored as 'average', each characteristic bit allocation value A ₁ '' and A ₃ '' can be created to have the relationship A ₁ ''≥A _3' ''.

The data organizing the standard bit allocation values for each subband in a table can be referred to as the second bit allocation table. The generated second bit allocation table may be stored in the form of a database 1081 in the memory 1080. Additionally, if necessary, the processor 310 may generate a plurality of second allocation tables through calculation.

The bit allocation selection unit 1040 may correspond to the bit allocation selection unit 940 of FIG. 9. The bit allocation selection unit 1040 can select one of the bit allocation tables depending on the presence or absence of hearing data. The bit allocation table may include a first table 1041 or a second table 1043. The first bit allocation table may be referred to as the first table 1041. The second bit allocation table may be referred to as the second table 1043.

The bit allocation selection unit 1040 may select a bit allocation table by comparing the first table 1041 and the second table 1043 as well as the presence or absence of hearing data. As an example, if there is no significant difference by comparing the first table 1041 created by reflecting hearing characteristics with the second table 1043, which is a standard bit allocation table that does not reflect hearing characteristics, the bit allocation selection unit 1040 ) can determine the number of quantization bits using the second table 834. This takes into account data efficiency or computational efficiency required for quantization.

The quantization unit 1050 may correspond to the quantization unit 950 of FIG. 9 . The quantization unit 1050 may perform quantization for each subband (Sb) according to the bit allocation table selected by the bit allocation selection unit 1040. The number of quantization bits can be determined according to the bit allocation value for each subband (Sb) indicated by the bit allocation table selected by the bit allocation selection unit 1040. The quantization unit 1050 may perform quantization on the corresponding segmented audio signal according to the number of quantization bits for each subband.

The lossless encoding unit 1060 may correspond to the lossless encoding unit 960 of FIG. 9 . The lossless encoding unit 1060 can losslessly encode the result quantized by the quantization unit 1050. The lossless encoder 1060 can perform lossless encoding on the corresponding segmented audio signal quantized for each subband.

FIG. 11 is a block diagram of the decoder 1100 of the second electronic device 102, according to one embodiment.

Referring to FIG. 11, the decoder 1100 includes a lossless decoder 1110, a bit stream analysis unit 1120, a bit allocation selection unit 1130, an inverse quantization unit 1140, a signal classification unit 1150, and a domain conversion unit. It may include unit 1160. Each component may be integrated into at least one module and implemented with at least one processor (eg, processor 610 in FIG. 6). Additionally, each of the above components may be added or omitted as needed.

The lossless decoding unit 1110 can hierarchically perform lossless decoding on the bit stream 620 received from the first electronic device 101. In the lossless decoding unit 1110, for example, lossless decoding is performed using a group grouped with codes corresponding to the highest bit as the highest layer, and groups grouped with codes corresponding to the lowest bit are sequentially used as the lower layer. Lossless decoding is possible.

In the lossless decoding unit 1110, lossless decoding may be performed using the method used in the lossless encoding unit (e.g., the lossless encoding unit 1160 of FIG. 7).

The bit stream analysis unit 1120 can analyze the received bit stream 620. The bit stream analysis unit 1120 can analyze meta information included in the bit stream. The meta information may include information to be referred to in order to inverse quantize the bit stream 620.

As an example, the meta information may include a first identifier indicating whether the bit stream 620 is generated by quantization considering hearing data. When the meta information includes the first identifier, the second electronic device 102 uses the first bit allocation table held by the second electronic device 102 to generate quantized data for each divided frequency band included in the bit stream 620. You can determine the number of inverse quantization bits to use to inverse quantize. For example, the first identifier may consist of one bit value (1 bit). If the bit value corresponding to the first identifier is '1', the second electronic device 102 can recognize that the bit stream 620 is generated by quantization considering hearing data. If the bit value corresponding to the first identifier is '0', the second electronic device 102 can recognize that the bit stream 620 is generated by quantization unrelated to hearing data.

As an example, the meta information may include information about the first bit allocation table applied to quantize the bit stream 620. For example, the information about the first bit allocation table may include a second identifier indicating the first bit allocation table used for quantization in an encoder (eg, encoder 900 in FIG. 9). If the information about the first bit allocation table includes the second identifier, the second electronic device 102 performs inverse quantization among one or more first bit allocation tables owned by the second electronic device 102 using the second identifier. The first bit allocation table used for this purpose can be determined. The second electronic device 102 may use the determined first bit allocation table to determine the number of inverse quantization bits to be used to inverse quantize the quantized data for each divided frequency band included in the bit stream 620.

As an example, the meta information may include information about the first bit allocation table applied to quantize the bit stream 620. The information about the first bit allocation table may be, for example, data that can configure the first bit allocation table. If the information about the first bit allocation table includes data that can configure the first bit allocation table, the second electronic device 102 can generate the first bit allocation table using the data. there is. The second electronic device 102 may use the generated first bit allocation table to determine the number of inverse quantization bits to be used to inverse quantize the quantized data for each divided frequency band included in the bit stream 620.

As an example, the bit stream analysis unit 1120 may analyze whether there is an error in the bit stream received from the first electronic device 101. For example, if there is an error in the bit stream received from the first electronic device 101 or the transmission environment is not smooth, the second electronic device 102 retransmits the bit stream to the first electronic device 101 (re -transmission) can be requested. The retransmission may be, for example, based on a TCP (transmission control protocol) retransmission method. The retransmission may be requested by adopting at least one of time-based retransmission, explicit retransmission feedback, or fast retransmission.

The bit allocation selection unit 1130 may determine which bit allocation table to use to perform inverse quantization on the bit stream for each frequency band. As an example, the bit allocation selection unit 1130 may select one of the first bit allocation table and the second bit allocation table.

The bit allocation selection unit 1130 may compare the first bit allocation table or the second bit allocation table and then determine which bit allocation table to use to perform inverse quantization. For example, when the second electronic device 102 has both a first bit allocation table and a second bit allocation table in the memory, if the two tables do not differ by a threshold level, the bit allocation selection unit 1130 Inverse quantization can be performed by selecting a standard bit allocation table. The threshold level may mean, for example, a case where the difference in the number of bit allocations for each subband is 1% or more of the total number of allocated bits. However, this is only an exemplary value, and the threshold level may be set differently depending on the situation.

The inverse quantization unit 1140 may inverse quantize the bit stream with the number of bits allocated to each subband using the bit allocation table selected by the bit allocation selection unit 1130. For example, the inverse quantization unit 1140 may perform inverse quantization in the same manner as the quantization unit (e.g., the quantization unit 1050 of FIG. 10) performs quantization. As an example, the inverse quantization unit 1140 may perform inverse quantization on each audio signal included in each divided audible frequency band using either the first bit allocation table or the second bit allocation table.

The signal classification unit 1150 may subdivide the divided audio signals obtained for each divided frequency band into subbands as a result of performing inverse quantization. For example, the signal classification unit 1150 may partition the bit stream by subband. When dividing partitions into subbands, the signal classification unit 1150 may reflect the energy value for each partition. For example, the signal classification unit 1150 may apply different gain values to each partition.

The domain converter 1160 can convert the region of the bit stream that has undergone inverse quantization. As an example, the domain converter 1160 may convert a bit stream in the frequency domain to the time domain. The transformation may be performed, for example, using an inverse Fourier transform method. The inverse Fourier transform method may include, for example, an inverse fast Fourier transform (IFFT) method. The domain converter 1160 may convert the decoded bit stream into the time domain and generate a restored audio signal.

The domain converter 1160 can synthesize a bit stream that has been inversely quantized for each divided audible frequency.

The audio signal 640 generated through the domain converter 1160 may be converted into an analog audio signal by a digital signal processor or ADC provided in the second electronic device 102 and then output to an audio output device.

An embodiment of the present disclosure may provide an apparatus and method for minimizing quantization noise by reflecting the user's individual hearing characteristics when quantizing or inverse quantizing an audio signal.

The control method of the first electronic device 101 according to an embodiment of the present disclosure includes an operation 410 of obtaining the number of quantization bits for each divided audible frequency band in which the audible frequency band is divided based on preset hearing characteristics of the user. , an operation 420 of performing quantization using the number of quantization bits corresponding to the audio signal for each divided audible frequency band extracted from the audio signal 320 output by reproduction of audio content, and quantization for each divided audible frequency band. An operation 430 of generating an audio signal as a bit stream 340 and transmitting it to the second electronic device 102 through a wireless channel may be included.

The control method of the first electronic device 101 according to an embodiment of the present disclosure may include setting the hearing characteristics 330 of the user by performing hearing measurement for each divided audible frequency band on the user. You can.

The control method of the first electronic device 101 according to an embodiment of the present disclosure generates a quantization bit allocation table in which the number of quantization bits for each divided audible frequency band is updated by reflecting the preset hearing characteristics 330 of the user. Operation 530 may be included.

A control method of the first electronic device 101 according to an embodiment of the present disclosure may include

operations

530 and 540 of selecting one quantization bit allocation table from among a plurality of generated quantization bit allocation tables.

The control method of the first electronic device 101 according to an embodiment of the present disclosure may operate to perform lossless encoding on the quantized audio signal for each divided audible frequency band.

The control method of the second electronic device 102 according to an embodiment of the present disclosure includes

operations

710 and 820 of analyzing

bit streams

340 and 620 received from the first electronic device 101, and the bit stream ( Obtaining the number of inverse quantization bits for each segmented audible frequency band included in 340, 620 (720), performing inverse quantization on the

bit stream

340, 620 for each audible frequency band using the inverse quantization bit number. It may include an operation 730 of performing an operation 730 and an operation 740 of outputting the audio signal 640 generated by the inverse quantization.

In the control method of the second electronic device 102 according to an embodiment of the present disclosure, the number of inverse quantization bits is determined by each audible frequency band used by the first electronic device 101 to quantize the audio signal 320. It may correspond to the number of quantization bits.

The control method of the second electronic device 102 according to an embodiment of the present disclosure may include requesting information 630 about the user's hearing characteristics necessary to perform inverse quantization from the first device 101. there is.

A control method of the second electronic device 102 according to an embodiment of the present disclosure includes the operation of acquiring information 630 about the hearing characteristics of a preset user from the first electronic device 101 and the obtained information 630. ) may include an operation of generating an inverse quantization bit allocation table for each divided audible frequency band.

The first electronic device 101 according to an embodiment of the present disclosure may include at least one processor 120 or 310 and a communication module 190. The at least one processor 120, 310 obtains the number of quantization bits for each divided audible frequency band that divides the audible frequency band based on the preset hearing characteristics of the user 330, and outputs by playing the audio content. Quantization is performed using the number of quantization bits corresponding to the audio signal for each divided audible frequency band extracted from the audio signal 320, and the quantized audio signal for each divided audible frequency band is generated as one bit stream 340. It can be transmitted to the second electronic device 102 through a wireless channel.

In the first electronic device 101 according to an embodiment of the present disclosure, the at least one processor 120, 310 performs hearing measurement for each divided audible frequency band on the user to determine the user's hearing characteristics ( 330) can be set.

In the first electronic device 101 according to an embodiment of the present disclosure, the at least one processor 120 or 310 determines the number of quantization bits for each divided audible frequency band by reflecting the preset hearing characteristics 330 of the user. An updated quantization bit allocation table can be created.

In the first electronic device 101 according to an embodiment of the present disclosure, the at least one processor 120 or 310 may select one quantization bit allocation table from a plurality of generated quantization bit allocation tables.

In the first electronic device 101 according to an embodiment of the present disclosure, the at least one processor 120 or 310 may perform lossless encoding on the quantized audio signal for each divided audible frequency band.

The second electronic device 102 according to an embodiment of the present disclosure may include at least one processor 610 and a communication module. The at least one processor 610 analyzes the bit streams 340 and 620 received from the first electronic device 101 and inverse quantization bits for each divided audible frequency band included in the bit streams 340 and 620. A number may be obtained, inverse quantization may be performed on the bit streams 340 and 620 for each audible frequency band using the inverse quantization bit number, and an audio signal 640 generated by the inverse quantization may be output.

In the second electronic device 102 according to an embodiment of the present disclosure, the number of inverse quantization bits is the number of quantization bits for each audible frequency band used by the first electronic device 101 to quantize the audio signal 320. can correspond to .

At least one processor 610 of the second electronic device 102 according to an embodiment of the present disclosure may provide

information

330, 630 about the user's hearing characteristics necessary to perform inverse quantization with the first electronic device 101. ) can be requested.

At least one processor 610 of the second electronic device 102 according to an embodiment of the present disclosure obtains

information

330 and 630 about the user's preset hearing characteristics from the first electronic device 101, , an inverse quantization bit allocation table for each segmented audible frequency band can be generated from the obtained information.

The electronic devices 101 and 102 according to an embodiment of the present disclosure may quantize or inverse quantize the

audio signals

320 and 630 by selecting a quantization model in consideration of the communication environment.

The electronic devices 101 and 102 according to an embodiment of the present disclosure minimize the quantization noise generated when quantizing or inverse quantizing the

audio signals

320 and 630 based on the user's

hearing data

330 and 630, thereby providing optimal It can provide audio signals with sound quality.

Electronic devices according to various embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to those components in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” Where mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. It can be used as A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is stored semi-permanently in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (e.g., module or program) of the above-described components may include a single or plural entity, and some of the plurality of entities may be separately placed in other components. there is. According to various embodiments, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, or omitted. Alternatively, one or more other operations may be added.

Claims

An operation 410 to obtain the number of quantization bits for each divided audible frequency band by dividing the audible frequency band based on the preset hearing characteristics of the user (410);

An operation 420 of performing quantization using the number of quantization bits corresponding to the audio signal for each divided audible frequency band extracted from the audio signal 320 output by playing audio content; and

A method comprising an operation (430) of generating a quantized audio signal for each divided audible frequency band as a bit stream (340) and transmitting it to an external electronic device (102) through a wireless channel.
According to paragraph 1,

A method comprising setting hearing characteristics (330) of the user by performing hearing measurement for each divided audible frequency band on the user.
According to paragraph 1,

An operation 530 of generating a quantization bit allocation table in which the number of quantization bits for each divided audible frequency band is updated by reflecting the preset hearing characteristics of the user 330; and

A method comprising the operations (530, 540) of selecting one quantization bit allocation table from among the plurality of generated quantization bit allocation tables.
According to paragraph 1,

A method comprising losslessly encoding the quantized audio signal for each divided audible frequency band.
Operations 710 and 820 of analyzing bit streams 340 and 620 received from the external electronic device 101;

An operation 720 of obtaining the number of inverse quantization bits for each divided audible frequency band included in the bit stream;

An operation 730 of performing inverse quantization on the bit streams 340 and 620 for each audible frequency band using the number of inverse quantization bits;

A method comprising an operation (740) of outputting an audio signal (640) generated by the inverse quantization.
According to clause 5,

The number of inverse quantization bits is,

A method corresponding to the number of quantization bits for each audible frequency band used by the external electronic device 101 to quantize the audio signal 320.
According to clause 5,

A method comprising requesting information about the user's hearing characteristics (630) necessary to perform inverse quantization from the external device (101).
In clause 7,

An operation of acquiring information 630 about the hearing characteristics of a preset user from the external electronic device 101 and

A method comprising generating an inverse quantization bit allocation table for each divided audible frequency band from the obtained information 630.
The electronic device 101 for transmitting an audio signal is:

Includes at least one processor (120, 310) and a communication module (190),

The at least one processor 120, 310,

Obtaining the number of quantization bits for each divided audible frequency band that divides the audible frequency band based on the user's preset hearing characteristics (330),

Quantization is performed using the number of quantization bits corresponding to the audio signal for each divided audible frequency band extracted from the audio signal output by playing audio content,

The electronic device 101 generates a quantized audio signal for each divided audible frequency band as one bit stream 340 and transmits it to the external electronic device 102 through a wireless channel.
The method of claim 9, wherein the at least one processor (120, 310):

An electronic device (101) that sets hearing characteristics (330) of the user by performing hearing measurement for each divided audible frequency band on the user.
The method of claim 9, wherein the at least one processor (120, 310):

Generating a quantization bit allocation table in which the number of quantization bits for each divided audible frequency band is updated by reflecting the preset hearing characteristics 330 of the user,

The electronic device 101 selects one quantization bit allocation table from among the plurality of generated quantization bit allocation tables.
The method of claim 9, wherein the at least one processor (120, 310):

An electronic device (101) that performs lossless encoding on the quantized audio signal for each divided audible frequency band.
The electronic device 102 for outputting an audio signal,

At least one processor 610; and

Includes a communication module,

The at least one processor 610,

Analyzing the bit stream received from the external electronic device 101,

Obtaining the number of inverse quantization bits for each divided audible frequency band included in the bit streams 340 and 620,

Perform inverse quantization on the bit streams 340 and 620 for each audible frequency band using the number of inverse quantization bits,

An electronic device 102 that outputs an audio signal 640 generated by the inverse quantization.
According to clause 13,

The number of inverse quantization bits is,

An electronic device 102 corresponding to the number of quantization bits for each audible frequency band used by the external electronic device 101 to quantize an audio signal.
The method of claim 13, wherein the at least one processor 610:

Request information (330, 630) about the user's hearing characteristics necessary to perform inverse quantization from the external electronic device (101),

Obtain information (330, 630) about the user's hearing characteristics preset from the external electronic device (101),

An electronic device (102) that generates an inverse quantization bit allocation table for each divided audible frequency band from the obtained information (330, 630).