WO2022186470A1 - Audio processing method and electronic device including same - Google Patents

Abstract

In various embodiments of the present invention, an electronic device may comprise: a speaker; an audio connector; a wireless communication circuit; and a processor operatively connected to the speaker, the audio connector, and the wireless communication circuit. The processor may be configured to: identify, in an audio file, format information related to a format configuring an audio signal; determine the type of the audio file on the basis of the format information; convert the audio file into the audio signal; process the audio signal by using a processing scheme optimized for the determined type among multiple designated processing schemes; and output the processed audio signal to at least one of the speaker, the audio connector, and the wireless communication circuit.

Description

Audio processing method and electronic device including same

Various embodiments relate to an audio processing method, and more particularly, to a method and apparatus for improving the sound quality of an audio signal.

The electronic device may apply a processing method for improving the sound quality of the audio to the audio signal. Accordingly, when an audio signal is reproduced, a user's desired sound field effect (or sound effect) can be expected. For example, an audio component of a low frequency band in an audio signal may be amplified through an equalizer of an electronic device, and thus a relatively richer sound may be reproduced than before amplification.

Fidelity indicating a degree to which an audio signal is reproduced close to the original sound may be different depending on a method of encoding the original sound.

Audio files of varying fidelity may exist. However, when an audio signal processed in a specified manner is reproduced without considering such a situation, there is no expected sound field effect and only disadvantages may be highlighted. For example, when a processing method optimized for a high-fidelity audio signal is applied to a relatively low-fidelity audio signal, only the low-quality sound quality can be further emphasized when the low-fidelity audio signal is reproduced (Garbage-In). Garbage-Out).

In various embodiments, the electronic device may provide the user with a sound field effect optimized for an audio signal when audio is reproduced.

The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. There will be.

In various embodiments of the present disclosure, an electronic device may include a speaker; audio connector; radio communication circuit; and a processor operatively coupled to the speaker, the audio connector, and the wireless communication circuitry. The processor is configured to check format information related to a format constituting an audio signal from an audio file, determine a type of the audio file based on the format information, convert the audio file into the audio signal, and It may be configured to process the audio signal using a processing method optimized for the determined type among processing methods, and output the processed audio signal to at least one of the speaker, the audio connector, and the wireless communication circuit.

According to various embodiments of the present disclosure, a method of operating an electronic device may include: checking format information related to a format constituting an audio signal from an audio file; determining a type of the audio file based on the format information; converting the audio file into the audio signal; processing the audio signal by using a processing method optimized for the determined type from among a plurality of specified processing methods; and an audio connector for connecting the processed audio signal to a speaker provided in the electronic device, an external audio output device and a wired connection between the electronic device, or the wireless communication for wirelessly connecting an external audio output device and the electronic device It may include an operation of outputting to at least one of the circuits. The format information may include at least one of a bit depth, a number of channels, a sampling rate, and a bit rate.

Various embodiments may provide an electronic device capable of providing a user with a sound field effect optimized for an audio signal during audio reproduction. In addition, various effects directly or indirectly identified through this document may be provided.

1 is a block diagram of an electronic device in a network environment, according to various embodiments of the present disclosure;

2 is a block diagram of an audio module, according to various embodiments.

3 is a block diagram of an electronic device configured to process and reproduce an audio signal in a manner optimized for a format of an audio file, according to various embodiments of the present disclosure;

4 illustrates operations of a processor for determining a processing scheme for an audio signal, according to an embodiment.

5 illustrates operations of a processor for processing an audio signal based on format information, according to an embodiment.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with at least one of the electronic device 104 and the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 . In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in , process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is a main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 uses less power than the main processor 121 or is set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The secondary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or when the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the coprocessor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190). have. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176 ) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used by a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 or an external electronic device (eg, a sound output module 155 ) directly or wirelessly connected to the electronic device 101 . The electronic device 102) (eg, a speaker or headphones) may output a sound.

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more specified protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a local area network (LAN) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 uses various techniques for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 may support various requirements defined in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: Downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, underside) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or part of the operations performed by the electronic device 101 may be executed by one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 needs to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. The server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

2 is a block diagram 200 of an audio module 170, in accordance with various implementations. Referring to FIG. 2 , the audio module 170 includes, for example, an audio input interface 210 , an audio input mixer 220 , an analog to digital converter (ADC) 230 , an audio signal processor 240 , and a DAC. It may include a digital to analog converter 250 , an audio output mixer 260 , or an audio output interface 270 .

The audio input interface 210 is acquired from the outside of the electronic device 101 as part of the input module 150 or through a microphone (eg, a dynamic microphone, a condenser microphone, or a piezo microphone) configured separately from the electronic device 101 . An audio signal corresponding to the sound may be received. For example, when the audio signal is obtained from the external electronic device 102 (eg, a headset or a microphone), the audio input interface 210 is directly connected to the external electronic device 102 through the connection terminal 178 . , or wirelessly (eg, via Bluetooth communication) through the wireless communication module 192 to receive an audio signal. According to an embodiment, the audio input interface 210 may receive a control signal (eg, a volume adjustment signal received through an input button) related to an audio signal obtained from the external electronic device 102 . The audio input interface 210 may include a plurality of audio input channels, and may receive a different audio signal for each corresponding audio input channel among the plurality of audio input channels. According to an embodiment, additionally or alternatively, the audio input interface 210 may receive an audio signal from another component of the electronic device 101 (eg, the processor 120 or the memory 130 ).

The audio input mixer 220 may synthesize a plurality of input audio signals into at least one audio signal. For example, according to an embodiment, the audio input mixer 220 may synthesize a plurality of analog audio signals input through the audio input interface 210 into at least one analog audio signal.

The ADC 230 may convert an analog audio signal into a digital audio signal. For example, according to one embodiment, the ADC 230 converts an analog audio signal received via the audio input interface 210, or additionally or alternatively, an analog audio signal synthesized via the audio input mixer 220 to digital audio. can be converted into a signal.

The audio signal processor 240 may perform various processing on the digital audio signal input through the ADC 230 or the digital audio signal received from other components of the electronic device 101 . For example, according to one embodiment, the audio signal processor 240 may change a sampling rate for one or more digital audio signals, apply one or more filters, perform interpolation processing, amplify or attenuate all or part of a frequency band, You can perform noise processing (such as noise or echo reduction), changing channels (such as switching between mono and stereo), mixing, or specified signal extraction. According to an embodiment, one or more functions of the audio signal processor 240 may be implemented in the form of an equalizer.

The DAC 250 may convert a digital audio signal into an analog audio signal. For example, according to an embodiment, the DAC 250 is a digital audio signal processed by the audio signal processor 240 , or another component of the electronic device 101 (eg, the processor 120 or the memory 130 ). ))) can be converted into an analog audio signal.

The audio output mixer 260 may synthesize a plurality of audio signals to be output into at least one audio signal. For example, according to one embodiment, the audio output mixer 260 may include an audio signal converted to analog through the DAC 250 and another analog audio signal (eg, an analog audio signal received through the audio input interface 210 ). ) can be synthesized into at least one analog audio signal.

The audio output interface 270 transmits an analog audio signal converted through the DAC 250 or an analog audio signal synthesized by the audio output mixer 260 additionally or alternatively through the audio output module 155 to the electronic device 101 . ) can be printed out. The sound output module 155 may include, for example, a speaker such as a dynamic driver or a balanced armature driver, or a receiver. According to an embodiment, the sound output module 155 may include a plurality of speakers. In this case, the audio output interface 270 may output an audio signal having a plurality of different channels (eg, stereo or 5.1 channel) through at least some of the plurality of speakers. According to an embodiment, the audio output interface 270 is directly connected to the external electronic device 102 (eg, an external speaker or headset) through the connection terminal 178 or wirelessly through the wireless communication module 192 . to output an audio signal.

According to an embodiment, the audio module 170 does not separately include the audio input mixer 220 or the audio output mixer 260 , and uses at least one function of the audio signal processor 240 to provide a plurality of digital audio signals. At least one digital audio signal may be generated by synthesizing them.

According to an embodiment, the audio module 170 is an audio amplifier (not shown) capable of amplifying an analog audio signal input through the audio input interface 210 or an audio signal to be output through the audio output interface 270 . (eg speaker amplification circuit). According to an embodiment, the audio amplifier may be configured as a module separate from the audio module 170 .

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C," each of which may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may simply be used to distinguish an element from other elements in question, and may refer elements to other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is “coupled” or “connected” to another (eg, second) component, with or without the terms “functionally” or “communicatively”. When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. According to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document include one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101). may be implemented as software (eg, the program 140) including For example, the processor (eg, the processor 120 ) of the device (eg, the electronic device 101 ) may call at least one of the one or more instructions stored from the storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided in a computer program product (computer program product). Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a memory of a relay server.

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. have. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. , or one or more other operations may be added.

3 is a block diagram of an electronic device 300 configured to process and reproduce an audio signal in a manner optimized for a format of an audio file, according to various embodiments of the present disclosure. The electronic device 300 (eg, the electronic device 101 of FIG. 1 ) includes an audio receiving module 301 , a decoder module 302 , a format checking module 303 , an audio processing module 305 , and a playback device selection module 306 , a speaker 320 , an audio connector 330 , a wireless communication circuit 340 , a DAC 350 , a memory 388 , and a processor 399 . The components of the electronic device 300 may be operatively or electrically connected to each other.

In an embodiment, the audio receiving module 301 may receive an audio file from an audio source. For example, the audio receiving module 301 is an audio to be played (eg, streaming playback) from an external sound source (eg, streaming media server) through the wireless communication circuit 340 (eg, the wireless communication module 192 of FIG. 1 ). may receive an audio file automatically selected by the user or by the server and store it in a buffer (eg, memory 388). The audio receiving module 301 may call an audio file selected by a user as audio to be reproduced among audio files stored in an internal sound source (eg, the memory 388).

In an embodiment, the decoder module 302 may decode the audio file received by the audio reception module 301 into a digital audio signal. For example, the decoder module 302 may decompress an audio file compressed in a lossy compression format such as MP3 or advanced audio coding (AAC) into a playable digital audio signal. The decoder module 302 may decompress an audio file compressed in a lossless compression format such as the true audio (TTA) or free lossless audio codec (FLAC) into a digital audio signal. In addition to the format illustrated above, audio files of various formats may be decompressed by the decoder module 302 .

In an embodiment, the format checking module 303 may check the format of the audio file. For example, the format check module 303 may check format information related to a format constituting a digital audio signal in an audio file (eg, a header). For example, the format check module 303 may check a bit rate, a sampling rate, a bit depth, or the number of channels. The bit rate is defined as the number of bits processed per unit time (eg, 1 second), and the unit may be bps (bit per second). It may be understood that the higher the bit rate, the higher the quality (eg, fidelity) of the corresponding audio file. The sampling rate may be defined as the number of samples extracted per unit time (eg, 1 second) from the original sound, and the unit may be hertz (Hz). For example, an audio file having a sampling rate of 48 kHz may be understood as an audio file included in multimedia, not music. The bit depth corresponds to the resolution of the image, and may be an index indicating how detailed the amplitude of the audio signal can be expressed, and the unit may be a bit. As the bit depth increases, the corresponding audio file may be understood as high resolution. The number of channels may be, for example, 1 (mono), 2 (stereo), 5.1, or 7.1.

According to various embodiments, the audio processing module 305 may process a digital audio signal based on format information. For example, the audio processing module 305 estimates the type (or use) (eg, music, multimedia, voice, high resolution, surround sound) of the corresponding audio file using the format information, and uses a processing method optimized for the type. Thus, by processing the digital audio signal, it is possible to provide a sound field effect expected by the user during audio reproduction. The processing method includes the operation of adjusting the pitch (frequency), volume, or tone of the audio signal, the operation of amplifying (or attenuating) the audio component of a specific frequency band in the audio signal, and the operation of adding sound effects ( Examples include the operation of synthesizing an echo), removing noise from the audio signal, increasing the volume (or decreasing the volume), and increasing the volume of the loud sound to become smaller in the audio signal. can do. In addition to the above operations, there may be various processing methods. The audio processing module 305 may configure a processing method optimized for a type of an audio file by using at least one of the above operations, and apply the optimized processing method to processing of a digital audio signal.

In an embodiment, the audio processing module 305 may process a digital audio signal based on the bit depth. For example, if the bit depth is greater than or equal to a specified value (eg, 24bit), the audio processing module 305 regards the audio file as an audio file having a high resolution, and emphasizes and distorts the characteristic of high resolution so that the audio is reproduced close to the original sound. A processing method for minimizing ? may be applied to the audio signal received from the decoder module 302 . For example, the audio processing module 305, when the playback device has a performance lower than the specified standard (eg, when the playback device is a micro speaker (eg, speaker 320 ) with a built-in mobile phone), the audio signal specified in the An audio component belonging to a frequency band higher than the first frequency (eg, about 16 kHz) and an audio component belonging to a frequency band lower than a specified second frequency (eg, about 40 Hz) may be emphasized (eg, amplified). The playback device may have performance above specified criteria. For example, the playback device may be an external speaker connected to the electronic device 300 through the audio connector 330 or the wireless communication circuit 340 . The electronic device 300 transmits specifications (eg, reproducible frequency band, sound pressure level, recommended amplifier output, and impedance) regarding the performance of the external speaker to the audio connector 330 or the wireless communication circuit 340 . can be received from an external speaker. The electronic device 300 may determine that the playback device (external speaker) is a device having a performance greater than or equal to a specified standard based on the received performance information. According to this determination, the audio processing module 305 may output the audio signal to the reproduction device without the above-described emphasis processing.

In an embodiment, the audio processing module 305 may process a digital audio signal based on the number of channels. For example, when the number of channels is one, the audio processing module 305 regards the audio file as a voice file (eg, an online lecture) for information transfer, and a specific frequency (eg, in the audio signal) for clear information transfer. : You can emphasize (eg, amplify) audio components belonging to about 1 kHz. When the number of channels exceeds 2, the audio processing module 305 regards the corresponding audio file as an audio file having surround sound, and applies a processing method for emphasizing three-dimensionality and presence received from the decoder module 302 . It can be applied to audio signals. When the number of channels is two, the audio processing module 305 may output the audio signal to the reproduction device without applying the above processing method to the audio signal.

In one embodiment, the audio processing module 305 may process the digital audio signal based on the sampling rate. For example, when the sampling rate is a specified first value (eg, about 44.1 kHz), the audio processing module 305 may regard the corresponding audio file as a music file. When the audio file is regarded as a music file, the audio processing module 305 may output the audio signal received from the decoder module 302 to the playback device. When the sampling rate is a specified second value (eg, about 48 kHz) greater than the first value, the audio processing module 305 may regard the corresponding audio file as an audio file recorded in multimedia (eg, a movie or a game). If the sampling rate is the second value, the audio processing module 305 may process the audio signal received from the decoder module 302 so that a dynamic effect appears on the multimedia. When the sampling rate is greater than or equal to a specified third value (eg, 96 kHz) greater than the second value, the audio processing module 305 regards the audio file as an audio file having a high resolution, and based on the performance of the playback device, the decoder module The audio signal received from 302 may be processed. For example, when the playback device has a performance lower than the specified standard (eg, when the playback device is a micro speaker (eg, speaker 320) with a built-in mobile phone), the audio processing module 305 may include a decoder module ( 302), it is possible to emphasize (eg, amplify) audio components of a specified low frequency band (eg, less than 40 Hz) and high frequency band (eg, 16 kHz or more). When the playback device has performance above the specified standard, the audio processing module 305 may output the audio signal received from the decoder module 302 to the playback device without the above-mentioned emphasis processing.

In one embodiment, the audio processing module 305 may process the digital audio signal based on the bit rate. For example, when the bit rate exceeds 100 kbps, the audio processing module 305 may regard the audio file as a low-loss compressed audio file. Accordingly, the audio processing module 305 may output the audio signal received from the decoder module 302 to the reproduction device. When the bit rate is 100 kbps or less, the audio processing module 305 may regard the corresponding audio file as a relatively lossy compressed audio file. If the bit rate is 100 kbps or less, the audio processing module 305 restores the audio component of the high frequency band (eg, 11 kHz or higher) lost in the process of compressing the audio signal, or the operation of attenuating the audio component of the high frequency band and the low frequency band can be performed.

The reproduction device selection module 306 may select a device to reproduce the digital audio signal. For example, when an external audio output device (eg, earphone, headphone) is connected to the audio connector 330 (eg, the connection terminal 178 of FIG. 1 ), the playback device selection module 306 may include the audio connector 330 ) connected to an external audio output device can be selected as the playback device. When an external audio output device (eg, a wireless earphone) is connected to the electronic device 300 through the wireless communication circuit 340 (eg, a Bluetooth module), the playback device selection module 306 configures the wireless communication circuit 340 An external audio output device connected wirelessly through the device can be selected as the playback device. When there is no connected external audio output device, the playback device selection module 306 may select the speaker 320 as the playback device.

In one embodiment, the DAC 350 (eg, the DAC 250 of FIG. 2 ) converts a digital audio signal into an analog audio signal and outputs it to the speaker 320 or to an external audio output device through the audio connector 330 . can be printed out. The speaker 320 and the external audio output device may convert the received analog audio signal into a sound wave and output it.

At least one of modules 301 , 302 , 303 , 305 , 306 is stored as instructions in memory 388 (eg, memory 130 of FIG. 1 ), and processor 399 (eg, FIG. 1 ) 1 processor 120). At least one of the modules 301 , 302 , 303 , 305 , and 306 may be executed by a processor specialized in audio signal processing (eg, the coprocessor 123 of FIG. 1 , or the audio signal processor 240 of FIG. 2 ). have.

4 illustrates operations of the processor 399 for determining a processing scheme for an audio signal, according to one embodiment.

In operation 410, the processor 399 may receive a digital audio signal to be reproduced from a sound source through a decoder (eg, the decoder module 302).

In operation 420, the processor 399 may determine whether the bit depth of the received audio signal exceeds a specified first value (eg, 24 bits). When the bit depth exceeds the first value (operation 420, Yes), in operation 421, the processor 399 may apply a first processing method designated to be used when processing a high-resolution audio signal to processing the audio signal. . As an example of the first processing method, the processor 399, when the playback device has a performance lower than the specified standard (eg, when the playback device is a micro-speaker (eg, speaker 320 ) with a built-in mobile phone), the audio In the signal, an audio component belonging to a frequency band higher than a specified first frequency (eg, about 16 kHz) and an audio component belonging to a frequency band lower than a specified second frequency (eg, about 40 Hz) may be emphasized (eg, amplified). In addition, various processing methods optimized for high-resolution audio may be applied as the first processing method. When the bit depth is equal to or less than the first value, the corresponding audio signal may be regarded as a relatively low-resolution audio signal. A processing method other than the first processing method may be used when it is considered as a low resolution.

When the bit depth is equal to or less than the first value (No in operation 420), in operation 430, the processor 399 may determine whether the number of channels of the received audio signal exceeds two. When the number of channels exceeds 2 (operation 430, yes), in operation 431, the processor 399 selects a second processing method (eg, a processing method for emphasizing three-dimensionality and presence) designated to be used when processing a surround sound audio signal. can be applied to the treatment of

If the number of channels is 2 or less (No in operation 430), the processor 399 may determine whether the number of channels of the received audio signal is 1 in operation 440. When the number of channels is 1 (operation 440, yes), in operation 441 , the processor 399 may apply a third processing method designated to be used when processing an audio signal to processing the audio signal. As an example of the third processing method, when the number of channels is one, the processor 399 regards the audio file as a voice file (eg, online lecture) for information transfer, and a specific frequency in the audio signal for clear information transfer Audio components belonging to (eg about 1 kHz) can be emphasized (eg amplified). In addition, various processing methods optimized for voice may be applied as the third processing method.

When the number of channels is 2 (No in operation 440), in operation 450, the processor 399 may determine whether the sampling rate of the received audio signal is a specified second value (eg, 48 kHz). When the sampling rate is the second value (operation 450, yes), in operation 451, the processor 399 applies a fourth processing method designated to be used when processing audio recorded in multimedia (eg, game, movie) to the processing of the audio signal. can be applied. As an example of the fourth processing method, the processor 399 may consider that the audio signal to be processed is recorded in multimedia (eg, a movie, a game), and may apply a processing method for emphasizing a dynamic effect to the audio signal. In addition, various processing methods optimized for audio in multimedia such as movies or games may be applied as the fourth processing method.

When the sampling rate is a value other than the second value (eg, 44.1 kHz) (operation 450, No), in operation 460, the processor 399 sets the bit rate of the received audio signal to a third value (eg, 100 kbps). It can be determined whether or not When the bit rate exceeds the third value (operation 460, yes), in operation 461, the processor 399 may apply a fifth processing method designated to be used when processing low-loss compressed music to the processing of the audio signal. . When the bit rate is equal to or less than the third value (No in operation 460), in operation 461, the processor 399 may apply a sixth processing method designated to be used when processing high-loss-compressed music to processing the audio signal. As an example of the sixth processing method, the processor 399 performs an operation of restoring an audio component of a high frequency band (eg, 11 kHz or higher) lost in the process of compressing the audio signal or an operation of attenuating the audio component of the high frequency band and the low frequency band can be performed. In addition, various processing methods optimized for music files compressed with high loss may be applied as the sixth processing method. Meanwhile, the determination operations 420 , 430 , 440 , 450 , and 460 are not limited to the order illustrated in FIG. 4 and may be changed.

5 illustrates operations of the processor 399 for processing an audio signal based on format information, according to an embodiment.

In operation 510, the processor 399 may receive an audio file to be reproduced from a sound source.

In operation 520, the processor 399 may estimate the type of the audio file based on the received format information of the audio file. For example, the format information may be included in a header of an audio file and may include a bit depth, the number of channels, a sampling rate, or a bit rate. In an embodiment, the processor 399 may determine the audio file as a high-resolution audio file or a low-resolution audio file based on format information (eg, bit depth). When the audio file is determined to be a low-resolution audio file, the processor 399 may estimate the type of the audio file in more detail. For example, the processor 399, based on the format information (eg, the number of channels, the sampling rate, the bit rate), converts the audio file to a low-loss-compressed music file, a high-loss-compressed music file, an audio file in multimedia, or It can be decided with an audio file.

In operation 530, the processor 399 may convert the audio file received from the sound source into a digital audio signal.

In operation 540, the processor 399 may process the digital audio signal using a processing method optimized for the estimated type. In an embodiment, when the audio file is determined to be a high-resolution audio file, the processor 399 may process the digital audio signal using a processing method optimized for high resolution (eg, the first processing method). When the audio file is determined to be a low resolution audio file, the processor 399 may process the digital audio signal using a processing method optimized for the low resolution. For example, the processor 399 determines, for example, one of the second to sixth processing methods among the second to sixth processing methods, and uses the determined processing method based on the information about the type obtained by estimating in more detail. to process digital audio signals.

In various embodiments of the present disclosure, an electronic device may include a speaker; audio connector; radio communication circuit; and a processor (eg, processor 399 of FIG. 3 ) operatively coupled to the speaker, the audio connector, and the wireless communication circuitry. The processor is configured to check format information related to a format constituting an audio signal from an audio file, determine a type of the audio file based on the format information, convert the audio file into the audio signal, and It may be configured to process the audio signal using a processing method optimized for the determined type among processing methods, and output the processed audio signal to at least one of the speaker, the audio connector, and the wireless communication circuit.

The format information may include at least one of a bit depth, a number of channels, a sampling rate, and a bit rate.

The processor determines the audio file as a high-resolution audio file or a low-resolution audio file based on the bit depth, and when the audio file is determined as the high-resolution audio file, a processing method designated to be used when processing a high-resolution audio signal is applied to the processing of the audio signal, and when the audio file is determined to be the low-resolution audio file, a processing method designated to be used when processing the low-resolution audio signal is applied to the processing of the audio signal.

When the audio file is determined to be the low-resolution audio file, the processor selects the type of the audio file based on the number of channels, the sampling rate, and the bit rate, low-loss-compressed music, high-loss-compressed music, and multimedia. and determine one of my audio and voice, and apply a processing scheme designated to be used when processing the determined type of audio signal to the processing of the audio signal.

The processor checks the bit depth in the format information, and when the bit depth exceeds a specified first value, applies a first processing method designated to be used when processing a high-resolution audio signal to the processing of the audio signal can be configured to

The processor checks the number of channels in the format information when the bit depth is equal to or less than the first value, and when the number of channels exceeds 2, applies a second processing method designated to be used when processing a surround sound to the audio signal It can be configured to apply to the treatment of

The processor may be configured to, when the number of channels is 1, apply a third processing method designated to be used when processing an audio signal to the processing of the audio signal.

When the number of channels is 2, the processor determines a sampling rate from the format information, and when the sampling rate is a specified second value, applies a fourth processing method designated to be used when processing audio recorded in multimedia of the audio signal. may be configured to apply to processing.

The processor is configured to check a bit rate in the format information when the sampling rate is not the second value, and, when the bit rate exceeds a third value, a second value designated to be used when processing low lossly compressed music. 5 may be configured to apply the processing scheme to the processing of the audio signal.

The processor may be configured to, when the bit rate is equal to or less than the third value, apply to the processing of the audio signal a sixth processing method designated to be used when processing high lossy compressed music.

According to various embodiments of the present disclosure, a method of operating an electronic device may include: checking format information related to a format constituting an audio signal from an audio file; determining the type of the audio file based on the format information (eg, operation 520 of FIG. 5 ); converting the audio file into the audio signal; processing the audio signal by using a processing method optimized for the determined type from among a plurality of specified processing methods (eg, operation 540 of FIG. 5 ); and an audio connector for connecting the processed audio signal to a speaker provided in the electronic device, an external audio output device and a wired connection between the electronic device, or the wireless communication for wirelessly connecting an external audio output device and the electronic device It may include an operation of outputting to at least one of the circuits. The format information may include at least one of a bit depth, a number of channels, a sampling rate, and a bit rate.

The determining may include determining the audio file as a high-resolution audio file or a low-resolution audio file based on the bit depth. In the processing, when the audio file is determined as the high-resolution audio file, a processing method designated to be used when processing the high-resolution audio signal is applied to the processing of the audio signal, and the audio file is determined as the low-resolution audio file. case, applying a processing method designated to be used when processing a low-resolution audio signal to the processing of the audio signal.

The determining may include: when the audio file is determined as the low-resolution audio file, the type of the audio file is low-loss-compressed music or high-loss-compressed music based on the number of channels, the sampling rate, and the bit rate. , multimedia in audio, and voice may include an operation of determining one of the following.

The processing includes checking a bit depth in the format information, and when the bit depth exceeds a specified first value, a first processing method designated to be used when processing a high-resolution audio signal is processed for the audio signal It may include an action applied to .

The processing includes checking the number of channels in the format information when the bit depth is equal to or less than the first value, and using a second processing method designated to be used when processing a surround sound when the number of channels exceeds two. It may include an operation applied to processing of an audio signal.

The processing may include, when the number of channels is 1, applying a third processing method designated to be used when processing an audio signal to processing the audio signal.

The processing includes checking a sampling rate in the format information when the number of channels is 2, and applying a fourth processing method designated to be used when processing audio recorded in multimedia when the sampling rate is a specified second value. It may include an operation applied to the processing of the signal.

The processing includes checking a bit rate in the format information when the sampling rate is not the second value, and when the bit rate exceeds a specified third value, to be used when processing low-loss compressed music. and applying a specified fifth processing method to the processing of the audio signal.

The processing may include applying a sixth processing method designated to be used when processing high lossy compressed music to processing of the audio signal when the bit rate is equal to or less than the third value.

The embodiments of the present invention disclosed in the present specification and drawings are merely provided for specific examples in order to easily explain the technical contents according to the embodiments of the present invention and help the understanding of the embodiments of the present invention, and limit the scope of the embodiments of the present invention It's not what you want to do. Therefore, in the scope of various embodiments of the present invention, in addition to the embodiments disclosed herein, all changes or modifications derived from the technical ideas of various embodiments of the present invention should be interpreted as being included in the scope of various embodiments of the present invention. .

Claims (15)

1. In an electronic device,

   speaker;

   audio connector;

   radio communication circuit; and

   a processor operatively coupled with the speaker, the audio connector, and the wireless communication circuitry;

   The processor is

   Check the format information related to the format constituting the audio signal in the audio file,

   determining the type of the audio file based on the format information;

   converting the audio file into the audio signal;

   processing the audio signal using a processing method optimized for the determined type among a plurality of specified processing methods;

   an electronic device configured to output the processed audio signal to at least one of the speaker, the audio connector, and the wireless communication circuit.
2. According to claim 1,

   The format information includes at least one of a bit depth, a number of channels, a sampling rate, and a bit rate.
3. The method of claim 2, wherein the processor comprises:

   determining the audio file as a high-resolution audio file or a low-resolution audio file based on the bit depth;

   When the audio file is determined to be the high-resolution audio file, a processing method designated to be used when processing the high-resolution audio signal is applied to the processing of the audio signal;

   and apply a processing method designated to be used when processing the low-resolution audio signal to the processing of the audio signal when the audio file is determined to be the low-resolution audio file.
4. The method of claim 3, wherein the processor, when the audio file is determined to be the low-resolution audio file,

   Based on the number of channels, the sampling rate, and the bit rate, determining the type of the audio file as one of low loss compressed music, high loss compressed music, audio in multimedia, and voice;

   and apply a processing scheme designated to be used when processing the determined type of audio signal to processing of the audio signal.
5. The method of claim 1, wherein the processor comprises:

   Check the bit depth in the format information,

   an electronic device configured to apply a first processing method designated to be used when processing a high-resolution audio signal to processing of the audio signal when the bit depth exceeds a specified first value.
6. The method of claim 5, wherein the processor comprises:

   If the bit depth is less than or equal to the first value, check the number of channels in the format information,

   and apply a second processing method designated to be used when processing a surround sound to processing of the audio signal when the number of channels exceeds two.
7. The method of claim 6, wherein the processor comprises:

   and when the number of channels is 1, apply a third processing method designated to be used when processing an audio signal to processing the audio signal.
8. The method of claim 7, wherein the processor,

   If the number of channels is 2, check the sampling rate in the format information,

   and apply a fourth processing method designated to be used when processing audio recorded in multimedia to processing of the audio signal when the sampling rate is a specified second value.
9. The method of claim 8, wherein the processor comprises:

   If the sampling rate is not the second value, check the bit rate in the format information,

   and apply a fifth processing scheme designated to be used when processing low lossy compressed music to processing of the audio signal when the bit rate exceeds a third specified value.
10. 10. The method of claim 9, wherein the processor,

    and apply a sixth processing scheme designated to be used when processing high lossy compressed music to processing of the audio signal when the bit rate is equal to or less than the third value.
11. A method of operating an electronic device, comprising:

    checking format information related to a format constituting an audio signal from an audio file;

    determining a type of the audio file based on the format information;

    converting the audio file into the audio signal;

    processing the audio signal by using a processing method optimized for the determined type from among a plurality of specified processing methods; and

    The audio connector for connecting the processed audio signal to a speaker provided in the electronic device, an external audio output device and a wired connection between the electronic device, or the wireless communication circuit for wirelessly connecting the external audio output device and the electronic device A method comprising outputting at least one of
12. 12. The method of claim 11,

    The format information includes at least one of a bit depth, a number of channels, a sampling rate, and a bit rate.
13. 13. The method of claim 12,

    The determining includes determining the audio file as a high-resolution audio file or a low-resolution audio file based on the bit depth,

    In the processing, when the audio file is determined as the high-resolution audio file, a processing method designated to be used when processing the high-resolution audio signal is applied to the processing of the audio signal, and the audio file is determined as the low-resolution audio file. case, applying a processing method designated to be used when processing a low-resolution audio signal to processing of the audio signal.
14. 14. The method of claim 13,

    The determining may include: when the audio file is determined as the low-resolution audio file, the type of the audio file is low-loss-compressed music or high-loss-compressed music based on the number of channels, the sampling rate, and the bit rate. , audio in multimedia, and voice.
15. The method of claim 11 , wherein the processing comprises:

    Checking the bit depth in the format information, and when the bit depth exceeds a specified first value, applying a first processing method designated to be used when processing a high-resolution audio signal to processing the audio signal How to.

Images