WO2022114516A1

WO2022114516A1 - Electronic device and control method therefor

Info

Publication number: WO2022114516A1
Application number: PCT/KR2021/014128
Authority: WO
Inventors: 황인우; 김선민
Original assignee: 삼성전자주식회사
Priority date: 2020-11-25
Filing date: 2021-10-13
Publication date: 2022-06-02
Also published as: KR20220072493A

Abstract

An electronic device and a control method therefor are disclosed. The electronic device of the present disclosure comprises a memory and a processor, wherein, if an audio signal is input, the processor can obtain information about an audio source included in the input audio signal, can generate at least one audio source from the input audio signal on the basis of the obtained information about the audio source, can identify a sound image location matching the at least one audio source, can process the at least one audio source so that the sound image location corresponding to the at least one audio source becomes an identified sound image location, and can output the processed audio source.

Description

Electronic device and control method thereof

The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device capable of identifying a virtual sound image location corresponding to an audio source included in an audio signal and a control method thereof.

As production and demand for large display devices and high-resolution content increase, users' expectations and demands for immersive sound effects are increasing. In addition, as the bezel of the display device is minimized, the speaker tends to be mounted at the bottom of the display device.

On the other hand, when the display screen is small, there are not many cases where the sound image of the object included on the screen and the audio image to be output are greatly different. However, as the display screen becomes larger, the possibility that the object and the sound image included on the screen are mismatched increases, and there is also a possibility that the sound stage is felt insufficient.

Accordingly, various techniques for positioning a sound image corresponding to a sound source output from a display device have been developed. However, when a plurality of different virtual sound images are generated for one sound source output from the display device, there is a possibility that the clarity of the sound source output from the display device may decrease.

The present disclosure has been devised to solve the above problems, and an object of the present disclosure is to identify a sound image location matching an audio source based on information about an audio source included in an audio signal input in real time, and to An object of the present invention is to provide an electronic device that processes an audio source so that a corresponding sound image position becomes an identified sound image position, and a control method thereof.

According to an embodiment of the present disclosure, an electronic device includes a processor, and when an audio signal is input, the processor acquires information about an audio source included in the input audio signal, and the acquired audio source generating at least one audio source from the input audio signal based on information on It is possible to process the at least one audio source so as to become a sound image location, and output the processed audio source.

In addition, the processor analyzes the input audio signal, and the information on the audio source includes information on a type of an audio source included in the audio signal or a ratio of at least one audio source included in the audio signal. can be obtained.

Also, the processor may identify a type of an audio source included in the audio source based on information about the audio source, and generate the identified type of audio source from the audio signal.

The electronic device further includes a speaker, and the sound image location matching the audio source includes a first location corresponding to an area in which the speaker is located, a second location corresponding to an area extending in the first direction from the speaker, and The speaker may be at least one of a third location corresponding to an area extending in the second direction.

Also, among the types of the audio sources, a position of a sound image corresponding to a first audio source including a voice is the first position, and a position of a sound image corresponding to a second audio source including music among types of the audio source is the first position. 2, and the sound image positions corresponding to the third audio source including the background sound among the types of the audio sources may be the first position, the second position, and the third position.

And, when the third audio source is generated from the audio signal, the processor is configured to set the third position so that sound image positions corresponding to the third audio source become the first position, the second position, and the third position. The audio source may be processed, and the processed third audio source may be stereo-mixed to obtain a final audio source, and the final audio source may be output.

Also, the processor may be configured to identify that the audio signal is configured of the first type of audio source when a ratio of the first type among the types of the plurality of audio sources included in the audio signal exceeds a threshold value. .

And, the electronic device further includes a display, and the processor controls the display to display a UI capable of controlling the second position and the third position, and the second position or the second position through the UI. When a user command for controlling at least one of the three locations is input, at least one of the second location and the third location may be controlled based on the inputted user command.

In a method of controlling an electronic device according to an embodiment of the present disclosure, when an audio signal is input, acquiring information on an audio source included in the input audio signal; identifying a type of an audio source included in the audio source based on, generating an audio source of the identified type from the audio signal, identifying a sound image location matching the at least one audio source, and the at least one processing the at least one audio source so that a sound image position corresponding to the audio source of ' becomes the identified sound image position, and outputting the processed audio source.

In addition, the acquiring may include analyzing the input audio signal and adding information about the type of the audio source included in the audio signal or the ratio of at least one audio source included in the audio signal to the audio source. It may include the step of obtaining information about.

The processing may include, when the third audio source is generated from the audio signal, the sound image positions corresponding to the third audio source become the first position, the second position, and the third position. processing a third audio source, wherein the outputting includes performing stereo mixing on the processed third audio source to obtain a final audio source, and outputting the final audio source can

And, the control method further includes the step of identifying that the audio signal is composed of the audio source of the first type when a ratio of the first type among the types of the plurality of audio sources included in the audio signal exceeds a threshold value may include

In addition, the control method includes displaying a UI for controlling the second position and the third position, and when a user command for controlling at least one of the second position and the third position is input through the UI, The method may further include controlling at least one of the second location and the third location based on the input user command.

According to various embodiments as described above, the electronic device may efficiently provide a high-realistic sound to the user by applying a virtual sound image application technology to an audio signal input in real time.

1 is a block diagram schematically illustrating a configuration of an electronic device according to an embodiment of the present disclosure;

FIG. 2 is a diagram for explaining a process in which an electronic device identifies a sound image location matching an audio source included in an audio signal, according to an embodiment of the present disclosure;

3 is a view for explaining a process in which an electronic device processes and outputs an input audio signal according to an embodiment of the present disclosure;

4 is a flowchart for explaining a method of controlling an electronic device according to an embodiment of the present disclosure;

5 is a detailed block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

-

Since the present embodiments can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, like reference numerals may be used for like components.

In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present disclosure, a detailed description thereof will be omitted.

In addition, the following examples may be modified in various other forms, and the scope of the technical spirit of the present disclosure is not limited to the following examples. Rather, these embodiments are provided to more fully and complete the present disclosure, and to fully convey the technical spirit of the present disclosure to those skilled in the art.

The terms used in the present disclosure are used only to describe specific embodiments, and are not intended to limit the scope of rights. The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this disclosure, expressions such as "A or B," "at least one of A and/and B," or "one or more of A or/and B" may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used in the present disclosure, expressions such as “first,” “second,” “first,” or “second,” may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component); When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element).

On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression “configured to (or configured to)” as used in this disclosure, depending on the context, for example, “suitable for,” “having the capacity to” ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware.

Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase "a processor configured (or configured to perform) A, B, and C" refers to a dedicated processor (eg, an embedded processor) for performing the corresponding operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented with at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

Meanwhile, various elements and regions in the drawings are schematically drawn. Accordingly, the technical spirit of the present invention is not limited by the relative size or spacing drawn in the accompanying drawings.

Meanwhile, the electronic device according to various embodiments of the present disclosure may be implemented as a display device, but this is only an example. Electronic devices include smart phones, tablet PCs, desktop PCs, laptop PCs or wearable devices, various medical devices (eg, various portable medical measuring devices (blood glucose monitors, heart rate monitors, blood pressure monitors, or body temperature monitors, etc.), magnetic resonance angiography (MRA) ), magnetic resonance imaging (MRI), computed tomography (CT), an imager, or an ultrasound machine), a navigation device, an automobile infotainment device, or an Internet of Things device.

Meanwhile, in describing the present disclosure, a sound image means a position where a listener feels that an audio source is output. The actual sound is output from the speaker, but the point where the sound source of the audio source is virtually focused is called the sound image.

Hereinafter, embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them.

1 is a block diagram schematically illustrating a configuration of an electronic device 100 according to an embodiment of the present disclosure. 1 , the electronic device 100 may include a memory 110 , a speaker 120 , and a processor 130 . However, the configuration shown in FIG. 1 is an exemplary diagram for implementing embodiments of the present disclosure, and appropriate hardware and software configurations at a level obvious to those skilled in the art may be additionally included in the electronic device 100 .

The memory 110 may store commands or data related to at least one other component of the electronic device 100 . In addition, the memory 110 is accessed by the processor 130 , and reading/writing/modification/deletion/update of data by the processor 130 may be performed.

In the present disclosure, the term "memory" refers to a memory 110, a ROM (not shown) in the processor 130, a RAM (not shown), or a memory card (not shown) mounted in the electronic device 100 (eg, micro SD). card, memory stick). In addition, programs and data for configuring various screens to be displayed in the display area of the display may be stored in the memory 110 . In addition, the memory 110 may be implemented as a volatile memory such as a buffer.

Information about the audio source obtained through the processor 130 may be stored in the memory 110 . The information on the audio source may include information on the type of the audio source included in the audio signal or the ratio of at least one audio source included in the audio signal.

Meanwhile, the type of the audio source may be expressed as an audio scene, and information about the audio source may be expressed as audio scene information. An audio scene refers to a unit classified based on a plurality of sound sources constituting an audio signal.

The memory 110 may include information on a location of a sound image matched with at least one audio source. The information on the location of the sound image matched with the audio source may include information on the location of the sound image that can be matched for each type of audio source. For example, the information on the sound image location may include information that, when the type of the first audio source is voice, the sound image location corresponding to the first audio source is the first location corresponding to the area where the speaker is located. A description related to this will be described in detail in a section to be described later.

The speaker 120 is configured to output various audio data on which various processing tasks such as decoding, amplification, and noise filtering have been performed by the processor 130 . For example, the speaker 120 may output an audio source processed so that a sound image position corresponding to the audio source becomes a sound image position matching the type of the audio source. In addition, the speaker 120 may output various types of audio data processed in the input/output interface, as well as various notification sounds or voice messages.

As shown in FIG. 3 , the speaker 120 may be implemented as a stereo speaker mounted at the bottom of the electronic device 100 . However, this is only an embodiment, and the number of speakers 120 may be one or a plurality. In addition, the speaker 120 may be mounted on various parts of the electronic device 100 .

The processor 130 may be electrically connected to the memory 110 to control overall functions and operations of the electronic device 100 . An operation performed by the processor 130 will be described in detail with reference to FIGS. 2 and 3 .

As shown in FIG. 2 , the processor 130 may receive an audio signal. For example, the processor 130 may receive an audio signal from an external device (eg, a USB flash drive, a set-top box, etc.) through an input/output interface. In another embodiment, the processor 130 may receive an audio signal from an external device (eg, a smart phone, etc.) through a communication interface. In this case, the input audio signal may include an audio streaming signal or the like.

The processor 130 may obtain information about an audio source included in the audio signal by analyzing 210 the input audio signal. The processor 130 may acquire information on the audio source included in the audio source by inputting the audio source to a neural network model trained to output information on the audio source included in the audio signal.

At this time, the trained neural network model includes the first neural network model trained to recognize the type of the audio source based on the time frequency shape of the audio signal, and the first neural network model trained to recognize the type of the audio source based on the spectral envelope shape of the audio signal. A second neural network model and a third neural network model trained to recognize a type of an audio source based on a feature vector extracted from the audio signal may be included.

In this case, the neural network model may be composed of a plurality of neural network layers. Each layer has a plurality of weight values, and the layer operation is performed through the operation of the previous layer and the operation of the plurality of weights. Examples of neural networks include Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), GAN. There are Generative Adversarial Networks and Deep Q-Networks, and the neural network model in the present disclosure is not limited to the above-described examples, except as otherwise specified.

Meanwhile, the information on the audio source may include information on the type of the audio source included in the audio signal or the ratio of at least one audio source included in the audio signal. The type of the audio source may be classified into a first audio signal including voice, a second audio signal including music, and a third audio signal including background sound. However, this is only an embodiment, and the type of the audio source may be divided into various types such as sound effect.

In addition, when various types of audio sources such as voices, music, background sounds, and sounds generated in the air such as helicopter sounds are included in the audio signal, information on the audio source is provided for each of the various types of audio sources constituting the audio signal. Information about the ratio may be included. For example, the information about the audio signal consists of 78% of the first audio source including voice, 20% of the second audio source including music, and 2% of the third audio source including background sound. It may include information that

In an embodiment of the present disclosure, when the ratio of the first type among the types of the plurality of audio sources included in the audio signal exceeds the threshold value, the processor 130 determines that the audio signal is configured as the audio source of the first type. can be identified. The threshold value may be a preset value determined according to an experiment or research, but is not limited thereto and may be changed by a user.

For example, it is assumed that the first audio source including voice in the audio signal is 95% and the threshold is 90%. The processor 130 may identify that the ratio of the first audio source to the audio signal exceeds a threshold, and the audio signal may be identified as being composed of the first audio source.

The processor 130 may generate 220 at least one audio source from the input audio signal based on the information on the audio source. Specifically, the processor 130 may identify the type of the audio source included in the audio source based on information about the audio source. Then, the processor 130 may generate an audio source of the identified type from the audio signal.

For example, it is assumed that the audio signal includes a first audio source in voice. The processor 130 may identify that the first audio source including voice is included in the audio signal by using the information on the audio signal. In addition, the processor 130 may generate a first audio source that is an audio source of a type identified in the audio signal.

As another embodiment, it is assumed that the audio signal includes a second audio source including music and a third audio source including a background sound. The processor 130 may identify that the second and third audio sources are included in the audio signal by using the information on the audio source. In addition, the processor 130 may generate second and third audio sources that are audio sources of a type identified in the audio signal.

The processor 130 may identify a sound image location matching the at least one audio source, and process the at least one audio source so that the sound image location corresponding to the at least one audio source becomes the identified sound image location. As an embodiment, the processor 130 may identify a sound image location corresponding to the audio source by using information about the sound image location matching the audio source stored in the memory 110 .

For example, the sound image location matching the audio source is a first location corresponding to an area in which the speaker 120 is located, a second location corresponding to an area extending in the first direction from the speaker 120 , and a location in the speaker 120 . It may be at least one of the third locations corresponding to the area extended in the second direction. In this case, the first direction may be a horizontal direction and the second direction may be a vertical direction, but is not limited thereto and may be set in various directions.

The location of the sound image matching the audio source will be described in detail with reference to FIG. 3 . FIG. 3 illustrates a case in which the electronic device 100 is implemented as a TV as a display device, and a speaker is mounted on the lower part of the TV and is implemented as stereo speakers 120-1 and 120-2. However, the present invention is not limited thereto, and the type of the electronic device 100 and the number and location of speakers may be variously implemented.

The first position means a position corresponding to an area in which the speakers 120 - 1 and 120 - 2 are located. When the sound image location matching the specific audio source is the first location, it means that the specific audio source is set to be output through the speakers 120 - 1 and 120 - 2 as it is.

The second position refers to

positions

300 and 310 corresponding to regions extended in the first direction (eg, a horizontal direction) from the speakers 120 - 1 and 120 - 2 . The fact that the sound image position matching the specific audio source is the second position creates the sound image position so that it is felt as if the specific audio source is output from the position corresponding to the area extended in the first direction from the speakers 120-1 and 120-2 Or it means you have to control it.

The third position refers to

positions

320 and 330 corresponding to regions extended in the second direction (eg, a vertical direction) from the speakers 120 - 1 and 120 - 2 . The fact that the sound image position matching the specific audio source is the third position creates the sound image position so that it is felt as if the specific audio source is output from a position corresponding to the area extended in the second direction from the speakers 120-1 and 120-2 Or it means you have to control it.

In an embodiment, a sound image position corresponding to a first audio source including voice among types of audio sources is a first sound image position, and a sound image position corresponding to a second audio source including music among types of audio sources is the second 2 sound image positions, and sound image positions corresponding to a third audio source including a background sound among types of audio scenes may be a first position, a second position, and a third position.

For example, outputting the first audio source including voice through the speaker 120 as it is may be most suitable from the viewpoint of voice intelligibility. Accordingly, the sound image position corresponding to the first audio source including the voice may be set as the first sound image position.

Accordingly, when the first audio source is generated from the audio signal, the processor 130 outputs the first audio source through the speaker 120 without performing an operation for generating the virtual sound image position (230-1). can In this case, the processor 130 does not perform an operation for generating the virtual sound image position, but may perform various processing operations such as decoding, amplification, and noise filtering on the first audio source and output the same.

As another example, a mixing engineer often mixes an audio source that includes music composed of instrumental sounds and the like by placing speakers around 30 degrees from the center of the horizontal plane. Accordingly, the sound image position corresponding to the second audio source including music may be set as the second sound image position.

Accordingly, when the second audio source is generated from the audio signal, the processor 130 corresponds to a region in which the position of the sound image corresponding to the second audio source is extended 230 - 2 in the horizontal plane direction with respect to the speaker 120 . The second audio source can be processed to be a second location where the

As another example, in a sports game, background sounds such as shouts of spectators, rain sounds, wind sounds, etc. do not significantly deteriorate even if the sound becomes somewhat unclear, and a sense of presence can be improved when played simultaneously on a plurality of virtual sounds. Accordingly, the position of the sound image corresponding to the third audio source including the background sound may be set to a plurality of positions (the first position, the second position, and the third position).

Accordingly, when the third audio source is generated from the audio signal, the processor 130 determines that the location of the sound image corresponding to the third audio source is a plurality of locations (eg, a first location corresponding to a region in which the speaker 120 is located; A second position corresponding to an area extended (230-2) in the horizontal direction with respect to the speaker 120 and a third position corresponding to an area extended (230-3) in a vertical direction with respect to the speaker 120) The third audio source may be processed to become

In addition, in the case of an audio source including sounds derived from the air, such as a helicopter sound or an airplane sound, a sense of presence may be improved when the sound is output from the upper direction with respect to the speaker. Accordingly, a sound image position corresponding to the fourth audio source including a helicopter sound, an airplane sound, and the like may be set as the third position. Accordingly, when the fourth audio source is generated from the audio signal, the processor 130 corresponds to the area of the sound image corresponding to the fourth audio source extended 230 - 3 in the vertical plane direction with respect to the speaker 120 . The fourth audio source can be processed to be a third location where the

Meanwhile, according to an embodiment of the present disclosure, when the third audio source is generated from the audio signal, the processor 130 determines that the sound image location corresponding to the third audio source is a plurality of locations (a first location, a second location, and a second location). 3 position), the third audio source may be processed, and a final audio source may be obtained by performing stereo mixing 240 on the processed third audio source. Then, the processor 130 may output the final audio source.

The processor 130 uses at least one of a cross-talk cancellation (XTC) function or a head related transfer function (HRTF) so that the position of the sound image corresponding to the audio source becomes a position corresponding to the area extended in the horizontal or vertical direction. It can handle audio sources. In addition, the processor 130 may process the audio source so that the number of sound images corresponding to the audio source is plural by using at least one of the XTC function and the HRTF.

The processor 130 may control the display 180 to display a UI capable of controlling the second position and the third position. A UI element capable of changing the second position and the third position according to a change of a parameter related to an XTC function or HRTF may be included on the UI. For example, changing the second position means changing the extent to which the speaker 120 is expanded in the horizontal direction with respect to the speaker 120 . Changing the third position means changing the extent to which the speaker 120 is extended in the vertical direction with respect to the speaker 120 .

When a user command for controlling at least one of the second location or the third location is input through the UI, the processor 130 may control at least one of the second location or the third location based on the input user command. . For example, when a user command to change the second location is input through the UI, the processor 130 may change the parameter related to the XTC function or HRTF for adjusting the second location to correspond to the user command.

Meanwhile, the processor 130 includes a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a digital signal processor (DSP), and an application processor for processing a digital signal. (AP)), or a communication processor (CP). In addition, the processor 130 may be implemented as a system on chip (SoC), large scale integration (LSI), or a field programmable gate array (FPGA) having a built-in processing algorithm. In addition, the processor 130 may include at least one of a graphics-processing unit (GPU), a neural processing unit (NPU), and a visual processing unit (VPU), which are separate AI-only processors, in order to perform an artificial intelligence function. .

4 is a flowchart illustrating a method of controlling the electronic device 100 according to an embodiment of the present disclosure.

First, the electronic device 100 may receive an audio signal (S410). The electronic device 100 may receive an audio signal from an external device (eg, a set-top box, a USB flash drive, etc.) connected through an input/output interface. As another example, the electronic device 100 may receive an audio signal from an external device (eg, a smart phone, an AI speaker, a microphone, etc.) connected through a communication interface.

The electronic device 100 may obtain information on an audio source included in the input audio signal (S420). The electronic device 100 may analyze the input audio signal to obtain information on the audio source including information on the type of the audio source included in the audio signal or the ratio of at least one audio source included in the audio signal. have.

The electronic device 100 may generate at least one audio source from the input audio signal based on the obtained information on the audio source (S430). Specifically, the electronic device 100 may identify the type of the audio source included in the audio source based on the information on the audio source. In addition, the electronic device 100 may generate an audio source of a type identified on the audio signal.

The electronic device 100 may identify a sound image location matching at least one audio source (S440). The sound image location matching the audio source is at least one of a first location corresponding to an area in which the speaker is located, a second location corresponding to an area extending in a horizontal direction from the speaker, and a third location corresponding to an area extending in a vertical direction from the speaker. can be one

In an embodiment, a sound image position corresponding to a first audio source including voice among types of audio sources is a first position, and a sound image position corresponding to a second audio source including music among types of audio sources is a second position. position, and the sound image position corresponding to the third audio source including the background sound among the types of the audio scene may be a plurality of virtual sound image positions (a first position, a second position, and a third position). As another example, a sound image location corresponding to a fourth audio source including a sound output from an upward direction, such as an airplane sound or a helicopter sound, among types of audio sources may be a second location.

The electronic device 100 may process the at least one audio source so that the sound image position corresponding to the at least one audio source becomes the identified sound image position (S450). For example, when the first audio source is generated from the audio signal, the electronic device 100 may process the first audio source so that the position of the sound image of the first audio source becomes the first position.

As another example, when the third audio source is generated from the audio signal, the electronic device 100 determines that the sound image position corresponding to the third audio source has a plurality of sound image positions (the first position, the second position, and the third position). As much as possible, a third audio source may be processed.

The electronic device 100 may output the processed audio source (S460). As an embodiment, the electronic device 100 may output the processed audio source after performing various processing operations such as decoding, amplification, and noise filtering. As another example, the electronic device 100 processes the third audio source so that the sound image position corresponding to the third audio source becomes a plurality of sound image positions (the first position, the second position, and the third position), and 3 The final audio source may be obtained by performing stereo mixing on the audio source, and the obtained final audio source may be output.

5 is a block diagram illustrating the configuration of the electronic device 100 in detail according to an embodiment of the present disclosure. 5 , the electronic device 100 includes a memory 110 , a speaker 120 , a processor 130 , a communication interface 140 , a microphone 150 , a user interface 160 , and an input/output interface 170 . ), a display 180 and a camera 190 may be included. Since the memory 110 , the speaker 120 , and the processor 130 have been described in detail with reference to FIGS. 1 and 2 , redundant descriptions will be omitted.

The communication interface 140 is configured to communicate with various types of external devices according to various types of communication methods. The communication interface 140 may include various wireless communication modules such as a Wi-Fi module, a Bluetooth module, and an infrared communication module. Here, each communication module may be implemented in the form of at least one hardware chip.

The Wi-Fi module and the Bluetooth module perform communication using a WiFi method and a Bluetooth method, respectively. In the case of using a Wi-Fi module or a Bluetooth module, various types of connection information such as an SSID and a session key are first transmitted and received, and then various types of information can be transmitted/received after communication connection using this.

The infrared communication module communicates according to the infrared data association (IrDA) technology, which wirelessly transmits data in a short distance using infrared that is between visible light and millimeter wave.

In addition to the above-described communication methods, the wireless communication module includes Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), LTE Advanced (LTE-A), 4th Generation (4G), 5G It may include at least one communication chip that performs communication according to various wireless communication standards such as (5th Generation).

In addition, the communication interface 140 is at least one of a wired communication module for performing communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, an optical fiber cable, or a UWB (Ultra Wide-Band) module, etc. may include

According to an example, the communication interface 140 may use the same communication module (eg, Wi-Fi module) to communicate with an external device such as a remote control and an external server.

According to another example, the communication interface 140 may use a different communication module (eg, a Wi-Fi module) to communicate with an external device such as a remote control and an external server. For example, the communication interface 140 may use at least one of an Ethernet module or a WiFi module to communicate with an external server, and may use a BT module to communicate with an external device such as a remote control. However, this is only an embodiment, and when communicating with a plurality of external devices or external servers, the communication interface 140 may use at least one communication module among various communication modules.

Meanwhile, the electronic device 100 may additionally include a tuner and a demodulator according to an embodiment. A tuner (not shown) may receive an RF broadcast signal by tuning a channel selected by a user or all channels previously stored among radio frequency (RF) broadcast signals received through an antenna. A demodulator (not shown) may receive and demodulate the digital IF signal (DIF) converted by the tuner, and may perform channel decoding and the like.

That is, the electronic device 100 may receive the content including the audio signal from the outside in real time through the communication interface 140 or the tuner.

The microphone 150 is configured to receive a user's voice or other sound and convert it into an audio signal. The microphone 150 may receive a user's voice in an activated state. The microphone 150 may be mounted inside the electronic device 100 , but is not limited thereto. The microphone 150 may be provided outside the electronic device 100 and may be electrically connected to the electronic device 100 or may be connected through wireless communication.

For example, the microphone 150 may be integrally formed on the top, front, or side of the electronic device 100 . The microphone 150 includes a microphone for collecting analog user voice, an amplifier circuit for amplifying the collected user voice, an A/D conversion circuit for sampling the amplified user voice and converting it into a digital signal, and a noise component from the converted digital signal. It may include various configurations such as a filter circuit that removes the

The user interface 160 may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or may be implemented as a touch screen capable of performing the above-described display function and manipulation input function together. Here, the button may be various types of buttons such as a mechanical button, a touch pad, a wheel, etc. formed in an arbitrary area such as the front, side, or rear of the exterior of the main body of the electronic device 100 .

Input/output interface 170 is HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt (Thunderbolt), VGA (Video Graphics Array) port, The interface may be any one of an RGB port, a D-subminiature (D-SUB), and a digital visual interface (DVI).

The input/output interface 170 may input/output content including at least one of audio and video signals. Depending on the implementation, the input/output interface 170 may include a port for inputting and outputting only an audio signal and a port for inputting and outputting only a video signal as separate ports, or may be implemented as a single port for inputting and outputting both an audio signal and a video signal.

The display 180 may be implemented as various types of displays, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), and the like.

The display 180 may also include a driving circuit, a backlight unit, and the like, which may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, or an organic TFT (OTFT). Meanwhile, the display 180 may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display, or the like.

The display 180 may display various image contents (eg, broadcast contents, etc.) received from the outside. As another example, the display 180 may display a UI for controlling the second position and the third position. The UI may include a UI element capable of changing the second position and the third position according to changing a parameter related to an XTC function or HRTF function capable of generating a virtual sound image.

The camera 190 is configured to generate a captured image by capturing a subject, wherein the captured image is a concept including both a moving image and a still image. The camera 190 may acquire an image of at least one external device, and may be implemented as a camera, a lens, an infrared sensor, or the like.

Meanwhile, the above-described methods according to various embodiments of the present disclosure may be implemented in the form of an application that can be installed in an existing electronic device.

In addition, the above-described methods according to various embodiments of the present disclosure may be implemented only by software upgrade or hardware upgrade of an existing electronic device.

In addition, various embodiments of the present disclosure described above may be performed through an embedded server provided in an electronic device or an external server of at least one of an electronic device and a display device.

Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage media readable by a machine (eg, a computer). can The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include the electronic device according to the disclosed embodiments. When the instruction is executed by the processor, the processor may perform a function corresponding to the instruction by using other components directly or under the control of the processor. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

Also, according to an embodiment of the present disclosure, the method according to the various embodiments described above may be included in a computer program product and provided. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

In addition, each of the components (eg, a module or a program) according to the above-described various embodiments may be composed of a single or a plurality of entities, and some sub-components of the aforementioned sub-components may be omitted, or other sub-components may be omitted. Components may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallelly, repetitively or heuristically executed, or at least some operations may be executed in a different order, omitted, or other operations may be added. can

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and it is common in the technical field pertaining to the present disclosure without departing from the gist of the present disclosure as claimed in the claims. Various modifications may be made by those having the knowledge of

Claims

In an electronic device,

Memory;

processor; including;

The processor is

When an audio signal is input, information about an audio source included in the input audio signal is obtained,

generating at least one audio source from the input audio signal based on the obtained information on the audio source;

identify a sound image location matching the at least one audio source, and process the at least one audio source so that a sound image location corresponding to the at least one audio source becomes the identified sound image location;

An electronic device for outputting the processed audio source.
According to claim 1,

The processor is

An electronic device that analyzes the input audio signal to obtain information on the audio source including information on a type of an audio source included in the audio signal or a ratio of at least one audio source included in the audio signal.
3. The method of claim 2,

The processor is

identify a type of audio source included in the audio source based on information about the audio source;

An electronic device for generating an audio source of the identified type from the audio signal.
According to claim 1,

speaker; further comprising,

The sound image location matching the audio source is,

at least one of a first position corresponding to an area in which the speaker is located, a second position corresponding to an area extending in the first direction from the speaker, and a third position corresponding to an area extending in the second direction from the speaker Characterized by an electronic device.
5. The method of claim 4,

A sound image position corresponding to a first audio source including a voice among the types of the audio source is the first position,

A sound image location corresponding to a second audio source including music among the types of the audio source is the second location,

Among the types of the audio source, sound image positions corresponding to a third audio source including a background sound are the first position, the second position, and the third position.
6. The method of claim 5,

The processor is

When the third audio source is generated from the audio signal, processing the third audio source so that sound image positions corresponding to the third audio source become the first position, the second position, and the third position;

An electronic device for obtaining a final audio source by performing stereo mixing on the processed third audio source, and outputting the final audio source.
4. The method of claim 3,

The processor is

When a ratio of a first type among types of a plurality of audio sources included in the audio signal exceeds a threshold value, the electronic device identifies that the audio signal is composed of the audio source of the first type.
5. The method of claim 4,

Display; further comprising,

The processor is

Control the display to display a UI that can control the second position and the third position,

When a user command for controlling at least one of the second location and the third location is input through the UI, the electronic device controls at least one of the second location and the third location based on the input user command .
A method for controlling an electronic device, comprising:

when an audio signal is input, obtaining information about an audio source included in the input audio signal;

identifying a type of an audio source included in the audio source based on the obtained information about the audio source, and generating an audio source of the identified type from the audio signal;

identifying a sound image position matching the at least one audio source, and processing the at least one audio source so that the sound image position corresponding to the at least one audio source becomes the identified sound image position; and

and outputting the processed audio source.
10. The method of claim 9,

The obtaining step is

analyzing the input audio signal to obtain information on the audio source including information on the type of the audio source included in the audio signal or the ratio of at least one audio source included in the audio signal; control method including.
10. The method of claim 9,

The sound image location matching the audio source is,

At least one of a first location corresponding to an area in which the speaker of the electronic device is located, a second location corresponding to an area extending in the first direction from the speaker, and a third location corresponding to an area extending in the second direction from the speaker Control method, characterized in that one.
12. The method of claim 11,

A sound image position corresponding to a first audio source including a voice among the types of the audio source is the first position,

A sound image location corresponding to a second audio source including music among the types of the audio source is the second location,

The sound image position corresponding to the third audio source including the background sound among the types of the audio source is the first position, the second position, and the third position.
13. The method of claim 12,

The processing step is

When the third audio source is generated from the audio signal, processing the third audio source so that sound image positions corresponding to the third audio source are the first position, the second position, and the third position; including,

The output step is

and performing stereo mixing on the processed third audio source to obtain a final audio source, and outputting the final audio source.
10. The method of claim 9,

When the ratio of the first type among the types of the plurality of audio sources included in the audio signal exceeds a threshold value, identifying that the audio signal is composed of the audio source of the first type; .
13. The method of claim 12,

displaying a UI capable of controlling the second location and the third location; and

when a user command for controlling at least one of the second location and the third location is input through the UI, controlling at least one of the second location and the third location based on the input user command; A control method further comprising a.