KR102446694B1 - Electronic device and method for controlling the same - Google Patents

Abstract

Disclosed are an electronic device and a method for controlling the electronic device. The present invention provides a plurality of audio objects and the object-based audio content from a communication unit for receiving a broadcast signal including object-based audio content and video content, a touch screen for displaying the video content, a sound output unit, and the object-based audio content a controller for restoring metadata related to , generating an audio output signal including the plurality of audio objects based on the metadata, and outputting the audio output signal through the sound output unit, wherein the control unit comprises: A control screen capable of adjusting the properties of each of the plurality of audio objects based on metadata is displayed on an area of the touch screen, and an audio output signal reflecting the properties adjusted through the control screen is outputted through the sound output unit includes doing According to the present invention, properties of each audio object can be conveniently adjusted when object-based audio content is reproduced.

Description

Electronic device and its control method

The present invention relates to an electronic device and a method for controlling the same, which allow convenient control when playing audio content in consideration of user's convenience.

With the development of technology, more and more various electronic devices are being developed, and various kinds of electronic devices are being used more and more diversely. For example, there are functions for data communication, taking pictures and videos through a camera, recording voice, playing music files through a speaker system, and outputting images or videos to the display unit. Some electronic devices add an electronic game play function or perform a multimedia player function.

In addition, the electronic device may receive a broadcast signal and provide a broadcast service through a display unit, an audio output unit, or the like. Conventional audio signals provided through broadcasting services such as TV broadcasting, radio broadcasting, and Digital Multimedia Broadcasting (DMB) are mixed with audio signals obtained from various sound sources and stored/transmitted as one audio signal. In such an environment, it is possible for the viewer to adjust the intensity of the entire audio signal, but it is impossible to control the characteristics of the audio signal for each sound source, such as adjusting the intensity of the audio signal for each sound source included in the audio signal.

However, according to the object-based audio method, when audio content is written without synthesizing the audio signal for each sound source, the content playback terminal can view the content while controlling the intensity of the audio signal for each sound source. do. That is, the storage/transmission terminal independently stores/transmits several audio signals, and the user can listen while appropriately controlling each audio signal in the receiver (content reproducing apparatus).

With the introduction of such an object-based audio method, a user interface through which a user can more conveniently control properties of each audio object in an electronic device is required.

SUMMARY OF THE INVENTION The present invention aims to solve the above and other problems. Another object of the present invention is to provide an electronic device that provides a user interface for conveniently adjusting properties of each audio object when object-based audio content is reproduced, and a method for controlling the same.

According to an aspect of the present invention to achieve the above or other objects, a plurality of audio objects and the object-based audio from a touch screen, a sound output unit, and the object-based audio content for displaying video content corresponding to the object-based audio content a controller for restoring content-related metadata, generating an audio output signal including the plurality of audio objects based on the metadata, and outputting the audio output signal through the sound output unit, The control unit displays a control screen capable of adjusting the properties of each of the plurality of audio objects based on the metadata on one area of the touch screen, and generates an audio output signal reflecting the properties adjusted through the control screen. An electronic device comprising outputting through an output unit is provided.

The metadata may include information related to the generation of the audio output signal including the type, location, sound intensity, and acoustic characteristics according to the location of each of the plurality of audio objects.

The property of each of the plurality of audio objects may include a volume or a position of each audio object.

The control screen may include a volume control item corresponding to each of the plurality of audio objects, and the controller may adjust a volume of the corresponding audio object according to an input applied to the volume control item.

The control screen may include a position adjustment item corresponding to each of the plurality of audio objects, and the controller may adjust the position of the corresponding audio object according to an input applied to the position adjustment item.

The control screen may include a sound image of the audio output signal based on the metadata, and the controller displays the position adjustment item on the sound image, and an audio object corresponding to the input. position can be adjusted.

The control screen may be displayed in an On Screen Display (OSD) method.

The controller may extract an image object respectively corresponding to the plurality of audio objects from the video content based on the metadata.

The controller may display the extracted image object from the video content displayed on the touch screen to be distinguished from other image objects.

When receiving a touch input for selecting the extracted image object, the controller may display a control screen for adjusting a property of an audio object corresponding to the selected image object on one area of the touch screen.

The electronic device may further include a communication unit for communication connection with the control device, wherein the control unit shares the screen displayed on the touch screen with the control device through the communication unit, and the extraction according to the control signal received from the control device The selected image object may be selected, and a control screen capable of adjusting properties of an audio object corresponding to the selected image object may be displayed on one area of the touch screen.

The controller may receive a control signal including coordinates of a touch input applied to the display of the control device, and select an image object located at coordinates on the touch screen corresponding to the coordinates of the touch input.

The control unit receives the control signal including information on a touch input applied to a control screen displayed on a display of the control device, and outputs an audio output signal reflecting the properties adjusted according to the control signal through the sound output unit can do.

In addition, according to another aspect of the present invention, displaying video content corresponding to object-based audio content on a touch screen, a plurality of audio objects from the object-based audio content and metadata related to the object-based audio content recovering, generating an audio output signal including the plurality of audio objects based on the metadata, outputting the audio output signal through a sound output unit, and the plurality of audio based on the metadata Displaying a control screen capable of adjusting the properties of each object on an area of the touch screen and outputting an audio output signal reflecting the properties adjusted through the control screen through the sound output unit control method is provided.

The effects of the electronic device and the control method thereof according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, by displaying a control screen capable of adjusting the properties of each of a plurality of audio objects in one area of the touch screen, the properties of each audio object can be conveniently displayed when object-based audio content is reproduced. It has the advantage of being adjustable.

In addition, according to at least one of the embodiments of the present invention, by providing a control screen including a volume control item capable of adjusting the volume of each audio object during playback of audio content, the volume of each audio object can be easily adjusted. There is this.

In addition, according to at least one of the embodiments of the present invention, by providing a control screen including a position adjustment item capable of adjusting the position of each audio object when the audio content is reproduced, the position of each audio object can be easily adjusted. There is this.

In addition, according to at least one of the embodiments of the present invention, by displaying the sound image of the audio output signal on the position adjustment item, there is an advantage that the position of the audio object on the sound image can be intuitively adjusted.

In addition, according to at least one of the embodiments of the present invention, there is an advantage in that an image object corresponding to the audio object can be intuitively recognized by displaying the image object corresponding to the audio object to be distinguished.

In addition, according to at least one of the embodiments of the present invention, there is an advantage in that the properties of the audio object can be conveniently adjusted by adjusting the properties of the audio object corresponding to the selected image object when the video content is reproduced.

In addition, according to at least one of the embodiments of the present invention, by allowing the control device sharing the screen to adjust the properties of the audio object for the image object selected, it is possible to conveniently adjust the properties of the audio object through a separate device. There is an advantage that

Further scope of applicability of the present invention will become apparent from the following detailed description. However, it should be understood that the detailed description and specific embodiments such as preferred embodiments of the present invention are given by way of illustration only, since various changes and modifications within the spirit and scope of the present invention may be clearly understood by those skilled in the art.

1 is a block diagram illustrating an electronic device related to the present invention.
2 is a flowchart of a method for controlling an electronic device according to an embodiment of the present invention.
3 to 5 are diagrams for explaining adjusting a volume for each audio object according to an embodiment of the present invention.
6 and 7 are diagrams for explaining adjusting the position of each audio object according to an embodiment of the present invention.
8 is a flowchart of a method for controlling an electronic device according to another embodiment of the present invention.
FIG. 9 is a diagram for explaining displaying an image object corresponding to an audio object to be distinguished according to an embodiment of the present invention.
10 to 13 are diagrams for explaining adjustment of properties for each corresponding audio object using an image object according to an embodiment of the present invention.
14 is a flowchart of a method for controlling an electronic device according to another embodiment of the present invention.
15 to 18 are diagrams for explaining how to remotely adjust an attribute of an audio object through a control device according to an embodiment of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and overlapping descriptions will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Electronic devices described herein include fixed electronic devices such as digital TVs, desktop computers, digital signage, etc. or mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDA), PMP (portable multimedia player), navigation, slate PC (slate PC), tablet PC (tablet PC), ultrabook (ultrabook), wearable device (wearable device, for example, watch-type terminal (smartwatch), glass-type terminal ( mobile electronic devices such as smart glass), head mounted displays (HMDs), and the like.

However, the present invention is not limited thereto, and the configuration according to the embodiment described herein is related to the name or type of an electronic device capable of receiving a broadcast signal including object-based audio content and adjusting an audio output signal for each object. It can be applied to any electronic device without

Referring to FIG. 1 , FIG. 1 is a block diagram illustrating an electronic device related to the present invention.

The electronic device 100 includes a communication unit 110 , an input unit 120 , a sensing unit 140 , an output unit 150 , an interface unit 160 , a memory 170 , a control unit 180 , and a power supply unit 190 . and the like. The components shown in FIG. 1 are not essential for implementing the electronic device, and thus the electronic device described herein may have more or fewer components than those listed above.

More specifically, the communication unit 110 among the components enables communication between the electronic device 100 and the communication system, between the electronic device 100 and another electronic device, or between the electronic device 100 and an external server. It may contain one or more modules that allow Also, the communication unit 110 may include one or more modules for connecting the electronic device 100 to one or more networks.

The communication unit 110 may include at least one of a broadcast reception module 111 , a mobile communication module 112 , an Internet module 113 , a short-range communication module 114 , and a location information module 115 .

The input unit 120 includes a camera 121 or an image input unit for inputting an image signal, a microphone 122 or an audio input unit for inputting an audio signal, and a user input unit 123 for receiving information from a user, for example, , a touch key, a push key, etc.). The voice data or image data collected by the input unit 120 may be analyzed and processed as a user's control command.

The sensing unit 140 may include one or more sensors for sensing at least one of information within the electronic device, surrounding environment information surrounding the electronic device, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. Sensor (G-sensor), gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor: infrared sensor), fingerprint sensor (finger scan sensor), ultrasonic sensor , optical sensors (eg, cameras (see 121)), microphones (see 122), battery gauges, environmental sensors (eg, barometers, hygrometers, thermometers, radiation sensors, It may include at least one of a thermal sensor, a gas sensor, etc.) and a chemical sensor (eg, an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the electronic device disclosed in the present specification may combine and utilize information sensed by at least two or more of these sensors.

The output unit 150 is for generating an output related to visual, auditory or tactile sense, and includes at least one of a display unit 151 , a sound output unit 152 , a haptip module 153 , and an optical output unit 154 . can do. The display unit 151 may implement a touch screen by forming a layer structure with the touch sensor or being formed integrally with the touch sensor. Such a touch screen may function as the user input unit 123 providing an input interface between the electronic device 100 and the user, and may provide an output interface between the electronic device 100 and the user.

The interface unit 160 serves as a passage with various types of external devices connected to the electronic device 100 . Such an interface unit 160, a wired / wireless headset port (port), an external charger port (port), a wired / wireless data port (port), a memory card (memory card) port, for connecting a device equipped with an identification module It may include at least one of a port, an audio I/O (Input/Output) port, a video I/O (Input/Output) port, and an earphone port. In response to the connection of the external device to the interface unit 160 , the electronic device 100 may perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the electronic device 100 . The memory 170 may store a plurality of application programs (or applications) driven in the electronic device 100 , data for operation of the electronic device 100 , and commands. At least some of these application programs may be downloaded from an external server through the communication unit. In addition, at least some of these application programs may exist on the electronic device 100 from the time of shipment for a basic function of the electronic device 100 . Meanwhile, the application program may be stored in the memory 170 , installed on the electronic device 100 , and driven to perform an operation (or function) of the electronic device by the controller 180 .

In addition to the operation related to the application program, the controller 180 generally controls the overall operation of the electronic device 100 . The controller 180 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 170 .

Also, the controller 180 may control at least some of the components discussed with reference to FIG. 1 in order to drive an application program stored in the memory 170 . Furthermore, in order to drive the application program, the controller 180 may operate at least two or more of the components included in the electronic device 100 in combination with each other.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 to supply power to each component included in the electronic device 100 . The power supply 190 includes a battery, and the battery may be a built-in battery or a replaceable battery.

At least some of the respective components may operate in cooperation with each other in order to implement an operation, control, or control method of an electronic device according to various embodiments described below. In addition, the operation, control, or control method of the electronic device may be implemented on the electronic device by driving at least one application program stored in the memory 170 .

Hereinafter, before looking at various embodiments implemented through the electronic device 100 as described above, the above-listed components will be described in more detail with reference to FIG. 1 .

First, referring to the communication unit 110 , the broadcast reception module 111 of the communication unit 110 receives a broadcast signal and/or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more of the broadcast reception modules may be provided to the electronic device 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server may mean a server that generates and transmits a broadcast signal and/or broadcast-related information or a server that receives a previously generated broadcast signal and/or broadcast-related information and transmits the received broadcast signal and/or broadcast-related information to an electronic device. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a broadcast signal in which a data broadcast signal is combined with a radio broadcast signal.

The broadcast signal may be encoded according to at least one of technical standards (or a broadcast method, for example, ISO, IEC, DVB, ATSC, etc.) for transmitting and receiving a digital broadcast signal, and the broadcast reception module 111 is configured to The digital broadcast signal may be received using a method suitable for technical standards set by technical standards.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112 .

The broadcast-related information may exist in various forms, for example, an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H). A broadcast signal and/or broadcast related information received through the broadcast reception module 111 may be stored in the memory 170 .

Mobile communication module 112, the technical standards or communication methods for mobile communication (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV -DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced, etc.) transmits and receives radio signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to (Long Term Evolution-Advanced).

The wireless signal may include various types of data according to transmission/reception of a voice call signal, a video call signal, or a text/multimedia message.

The Internet module 113 refers to a module for Internet access, and may be built-in or external to the electronic device 100 . The Internet module 113 is configured to transmit and receive data signals in a communication network according to Internet technologies.

The short-range communication module 114 is for short-range communication, and includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NFC. At least one of (Near Field Communication), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Wireless Universal Serial Bus (USB), and Magnetic Secure Transmission (MST) may be used to support short-range communication. The short-distance communication module 114, between the electronic device 100 and the wireless communication system, between the electronic device 100 and another electronic device 100, or the electronic device 100 through wireless area networks (Wireless Area Networks) ) and a network in which another electronic device 100 or an external server is located may support wireless communication. The local area network may be local area networks (Wireless Personal Area Networks).

Here, the other electronic device 100 is a wearable device capable of exchanging (or interworking) data with the electronic device 100 according to the present invention, for example, a smart watch, smart glasses (smart glass), HMD (head mounted display)). The short-range communication module 114 may detect (or recognize) a wearable device capable of communicating with the electronic device 100 in the vicinity of the electronic device 100 . Furthermore, when the sensed wearable device is a device authenticated to communicate with the electronic device 100 according to the present invention, the controller 180 transmits at least a portion of data processed by the electronic device 100 to the short-range communication module ( 114) to transmit to the wearable device. Accordingly, the user of the wearable device may use data processed by the electronic device 100 through the wearable device.

The location information module 115 is a module for acquiring a location (or current location) of an electronic device, and a representative example thereof includes a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the electronic device utilizes a GPS module, the electronic device may acquire the location of the electronic device by using a signal transmitted from a GPS satellite. As another example, if the electronic device utilizes the Wi-Fi module, it may acquire the location of the electronic device based on information of the Wi-Fi module and a wireless access point (AP) that transmits or receives a wireless signal. If necessary, the location information module 115 may perform any function of the other modules of the communication unit 110 to obtain data on the location of the electronic device as a substitute or additionally. The location information module 115 is a module used to obtain the location (or current location) of the electronic device, and is not limited to a module that directly calculates or obtains the location of the electronic device.

Next, the input unit 120 is for input of image information (or signal), audio information (or signal), data, or information input from a user. For input of image information, the electronic device 100 has one Alternatively, a plurality of cameras 121 may be provided. The camera 121 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170 . On the other hand, the plurality of cameras 121 provided in the electronic device 100 may be arranged to form a matrix structure, and through the cameras 121 forming the matrix structure in this way, various angles or focal points are applied to the electronic device 100 . A plurality of image information having a plurality of images may be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 122 processes an external sound signal as electrical voice data. The processed voice data may be variously utilized according to a function (or a running application program) being performed by the electronic device 100 . Meanwhile, various noise removal algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122 .

The user input unit 123 is for receiving information from a user, and when information is input through the user input unit 123 , the controller 180 may control the operation of the electronic device 100 to correspond to the input information. . Such, the user input unit 123 is a mechanical input means (or a mechanical key, for example, a button located on the front, rear or side of the electronic device 100, a dome switch (dome switch), a jog wheel, jog switch, etc.) and a touch input means. As an example, the touch input means consists of a virtual key, a soft key, or a visual key displayed on the touch screen through software processing, or a part other than the touch screen. It may be made of a touch key (touch key) disposed on the. On the other hand, the virtual key or the visual key, it is possible to be displayed on the touch screen while having various forms, for example, graphic (graphic), text (text), icon (icon), video (video) or these can be made by a combination of

Meanwhile, the sensing unit 140 senses at least one of information in the electronic device, surrounding environment information surrounding the electronic device, and user information, and generates a sensing signal corresponding thereto. The controller 180 may control the driving or operation of the electronic device 100 or perform data processing, functions, or operations related to an application program installed in the electronic device 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor 141 may be disposed in an inner region of the electronic device covered by the touch screen as described above or near the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, an infrared proximity sensor, and the like. When the touch screen is capacitive, the proximity sensor 141 may be configured to detect the proximity of an object having conductivity by a change in an electric field according to the proximity of the object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of description, the act of approaching an object on the touch screen without being in contact and allowing the object to be recognized that it is located on the touch screen is called “proximity touch”, and the touch The act of actually touching an object on the screen is called "contact touch". The position where the object is touched in proximity on the touch screen means a position where the object is perpendicular to the touch screen when the object is touched in proximity. The proximity sensor 141 may detect a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.) have. Meanwhile, the controller 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 141 as described above, and furthermore, provides visual information corresponding to the processed data. It can be printed on the touch screen. Furthermore, the controller 180 may control the electronic device 100 to process different operations or data (or information) according to whether a touch to the same point on the touch screen is a proximity touch or a contact touch. .

The touch sensor receives a touch (or touch input) applied to the touch screen (or the display unit 151) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. detect

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific part of the touch screen or a change in capacitance occurring in a specific part of the touch screen into an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, capacitance at the time of touch, and the like where a touch object applying a touch on the touch screen is touched on the touch sensor. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen, a stylus pen, or a pointer.

As such, when there is a touch input to the touch sensor, a signal(s) corresponding thereto is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the controller 180 . Accordingly, the controller 180 can know which area of the display unit 151 has been touched, and the like. Here, the touch controller may be a component separate from the controller 180 , or may be the controller 180 itself.

Meanwhile, the controller 180 may perform different controls or may perform the same control according to the type of the touch object that touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different control or the same control according to the type of the touch object may be determined according to the current operating state of the electronic device 100 or a running application program.

On the other hand, the above-described touch sensor and proximity sensor independently or in combination, a short (or tap) touch on the touch screen (short touch), long touch (long touch), multi touch (multi touch), drag touch (drag touch) ), flick touch, pinch-in touch, pinch-out touch, swype touch, hovering touch, etc. It can sense touch.

On the other hand, the camera 121 as seen in the configuration of the input unit 120 includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to detect a touch of a sensing target for a 3D stereoscopic image. The photo sensor may be stacked on the display device, and the photo sensor is configured to scan the motion of the sensing target close to the touch screen. More specifically, the photo sensor mounts photo diodes and transistors (TRs) in rows/columns and scans the contents placed on the photo sensor using electrical signals that change according to the amount of light applied to the photo diodes. That is, the photo sensor calculates the coordinates of the sensing target according to the amount of change in light, and through this, location information of the sensing target can be obtained.

The display unit 151 displays (outputs) information processed by the electronic device 100 . For example, the display unit 151 may display execution screen information of an application program driven in the electronic device 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information. .

Also, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

A three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glassesless method), or a projection method (holographic method) may be applied to the stereoscopic display unit.

The sound output unit 152 may output audio data received from the communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit 152 also outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the electronic device 100 . The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 153 may be vibration. The intensity and pattern of vibration generated by the haptic module 153 may be controlled by a user's selection or setting of the controller. For example, the haptic module 153 may synthesize and output different vibrations or output them sequentially.

In addition to vibration, the haptic module 153 is configured to respond to stimuli such as a pin arrangement that moves vertically with respect to the contact skin surface, a jet or suction force of air through a nozzle or an inlet, a brush against the skin surface, contact of an electrode, an electrostatic force, etc. Various tactile effects can be generated, such as an effect caused by heat absorption and an effect by reproducing a feeling of cold and heat using an element capable of absorbing heat or generating heat.

The haptic module 153 may not only deliver a tactile effect through direct contact, but may also be implemented so that the user can feel the tactile effect through a muscle sense such as a finger or arm. Two or more haptic modules 153 may be provided according to the configuration of the electronic device 100 .

The light output unit 154 outputs a signal for notifying the occurrence of an event by using the light of the light source of the electronic device 100 . Examples of the event generated in the electronic device 100 may be message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is realized when the electronic device emits light of a single color or a plurality of colors toward the front or rear. The signal output may be terminated when the electronic device detects the user's event confirmation.

The interface unit 160 serves as a passage with all external devices connected to the electronic device 100 . The interface unit 160 receives data from an external device, receives power and transmits it to each component inside the electronic device 100 , or transmits data inside the electronic device 100 to an external device. For example, a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port for connecting a device equipped with an identification module (port), audio I/O (Input/Output) port (port), video I/O (Input/Output) port (port), earphone port (port), etc. may be included in the interface unit 160 .

On the other hand, the identification module is a chip storing various information for authenticating the use authority of the electronic device 100, a user identification module (UIM), a subscriber identity module (subscriber identity module; SIM), universal user authentication It may include a universal subscriber identity module (USIM) and the like. A device equipped with an identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device may be connected to the terminal 100 through the interface unit 160 .

In addition, when the electronic device 100 is connected to an external cradle, the interface unit 160 serves as a passage through which power from the cradle is supplied to the electronic device 100 or is input from the cradle by a user. It may be a path through which various command signals are transmitted to the electronic device 100 . The various command signals or the power input from the cradle may be operated as signals for recognizing that the electronic device 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and may temporarily store input/output data (eg, a phone book, a message, a still image, a moving picture, etc.). The memory 170 may store data related to vibrations and sounds of various patterns output when a touch input is performed on the touch screen.

Memory 170 is a flash memory type (flash memory type), a hard disk type (hard disk type), an SSD type (Solid State Disk type), an SDD type (Silicon Disk Drive type), a multimedia card micro type ), card-type memory (such as SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read (EEPROM) -only memory), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The electronic device 100 may be operated in relation to a web storage that performs a storage function of the memory 170 on the Internet.

Meanwhile, as described above, the controller 180 controls an operation related to an application program and generally an overall operation of the electronic device 100 . In order to implement various embodiments described below on the electronic device 100 according to the present invention, the controller 180 may control any one or a combination of the above-described components.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 to supply power required for operation of each component. The power supply unit 190 includes a battery, and the battery may be a built-in battery configured to be rechargeable, and may be detachably coupled to the terminal body for charging or the like.

Also, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of the interface 160 to which an external charger that supplies power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply 190 uses one or more of an inductive coupling method based on a magnetic induction phenomenon or a resonance coupling method based on an electromagnetic resonance phenomenon from an external wireless power transmitter. power can be transmitted.

Meanwhile, various embodiments below may be implemented in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

Hereinafter, embodiments related to a control method that can be implemented in the electronic device configured as described above will be described with reference to the accompanying drawings. It is apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention.

2 is a flowchart of a method for controlling an electronic device according to an embodiment of the present invention. 3 to 5 are diagrams for explaining adjusting a volume for each audio object according to an embodiment of the present invention. 6 and 7 are diagrams for explaining adjusting the position of each audio object according to an embodiment of the present invention.

The method for controlling an electronic device according to an embodiment of the present invention may be implemented in the electronic device 100 described with reference to FIG. 1 . Hereinafter, a method for controlling an electronic device according to an embodiment of the present invention and an operation of the electronic device 100 for implementing the method will be described in detail with reference to the necessary drawings.

Referring to FIG. 2 , the controller 180 may display video content corresponding to object-based audio content on the touch screen [S100].

According to an example, the object-based audio content and video content may be received in real time as a broadcast stream through the communication unit 110 ( FIG. 1 ). In this case, the object-based audio content and the video content may constitute one broadcast content together. However, the present invention is not limited thereto. For example, in the case of content including object-based audio content and video content stored in the memory 170 or the like, the following description may be applied substantially the same.

Referring to FIG. 3 , the controller 180 may display the video content on the touch screen 151 . The controller 180 may display an image on the touch screen 151 by restoring video content from the broadcast stream according to a user input or the like.

3 illustrates that an image including a plurality of objects is displayed on the touch screen 151 . The displayed image may be an image corresponding to one of the frames constituting the video content. The video includes a person (A) speaking as an object and a person (B) playing the guitar (C). In addition, the speech bubble D displayed next to the person A is shown for convenience in explaining that the person A is speaking, and is not an image that is actually displayed.

Referring back to FIG. 2 , the controller 180 restores a plurality of audio objects and metadata related to the object-based audio content from the object-based audio content [S110], and based on the metadata, the plurality of audio objects An audio output signal including the audio object of [S120] may be generated, and the audio output signal may be output through a sound output unit [S130].

According to an example, when generating object-based audio content, each audio object constituting the audio content may be independently stored. Attribute information defining the location of each audio object, sound intensity, and acoustic characteristics according to the location, which can be used when mixing each audio object to generate audio content, may be generated.

The attribute information may be transmitted as metadata in the broadcast stream together with the generated audio content. The metadata may include property information including the type, location, sound intensity, and acoustic characteristics according to the location of each of the plurality of audio objects, that is, information related to the generation of the audio output signal. In this case, the method included in the broadcast signal is not limited to a specific method.

The controller 180 may restore each audio object independently included in the received object-based audio content and restore the metadata. The controller 180 may generate an audio output signal representing the audio content according to attribute information, such as the type, location, sound intensity, and acoustic characteristics according to the location, of each audio object included in the restored metadata. .

The controller 180 may output the generated audio output signal through the sound output unit 152 . According to an example, the controller 180 may output the audio output signal for each independent object using the sound output unit 152 provided in the electronic device 100 . However, when a separate external device capable of outputting audio is connected to the electronic device 100 , the audio output signal may be output by using the external device together.

Referring back to FIG. 2 , the controller 180 may display a control screen for adjusting the properties of each of the plurality of audio objects on one area of the touch screen based on the metadata [S140].

According to an example, the property of each of the plurality of audio objects may include a volume or a location of each audio object. The volume of the audio object may mean the volume of each independent audio object, not the volume of the entire output audio content. The location of the audio object may mean a virtual location where a sound source corresponding to the audio object exists. However, the present invention is not limited thereto, and the properties of the audio object to be adjusted may include other properties as necessary.

Referring to FIG. 4 , a control screen 10 capable of adjusting the volume of each of the plurality of audio objects is displayed on the touch screen 151 . According to an example, the control screen 10 may be displayed according to a user's input. For example, as shown in FIG. 3 , when a user applies a predetermined touch input such as a long touch on an area of the touch screen 151 while video content is being displayed, the controller 180 controls the control screen ( 10) can be displayed.

However, the present invention is not limited thereto, and the predetermined touch input may include a short touch, a long touch, a multi-touch, a drag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipe touch, a hovering touch, etc. It can be set by a touch method or a combination thereof. In addition, the control screen 10 may be set to be displayed when there is input of a specific key or a specific input through a device such as a mouse or a remote control in addition to a touch input.

As shown in FIG. 4 , the control screen 10 may include information 11 and 12 of each of the plurality of audio objects. The controller 180 may acquire information on an audio object currently output through the sound output unit 152 based on the metadata. Referring to the image shown in FIG. 4 , it is assumed that an audio object 11 that is a conversation spoken by a person A and an audio object 12 that is a sound of a guitar C played by the person B are currently being output.

The controller 180 may display volume control items 21 and 22 respectively corresponding to the audio objects 11 and 12 currently being output on the control screen. According to an example, the volume control items 21 and 22 may be displayed in the form of a progress bar. The volume control items 21 and 22 may include handles 23 and 24 for adjusting the volume. The positions of the handles 23 and 24 may indicate the volume at which the corresponding audio object is currently output.

Referring to FIG. 5 , the controller 180 may adjust the volume of a corresponding audio object according to an input applied to the volume control items 21 and 22 . In order to adjust the volume of the desired audio object, the user may control the handle of the volume control item corresponding to the desired audio object.

For example, in order to adjust the volume of the currently output guitar sound 12 , the user may touch the handle 24 of the volume control item 22 and move it left and right. As the handle 24 is moved to the left, the controller 180 may decrease the volume of the guitar sound 12 . When the handle 24 is moved to the leftmost position, the controller 180 may mute the guitar sound 12 . According to an example, the controller 180 may display a mute button (not shown) as a volume control item.

Similarly, the controller 180 may increase the volume of the guitar sound 12 as the handle 24 is moved to the right. In this case, the adjusted volume is the volume of the guitar sound 12 , and the volume of the entire audio content currently being output is not controlled.

In the above description, it is assumed that the volume control item is displayed in the form of a progress bar, but this is an example and is not limited thereto. According to another example, the volume control items 21 and 22 may be displayed in various ways such as a pie graph or a number.

Referring to FIG. 6 , it is shown that a control screen 30 capable of adjusting the position of each of the plurality of audio objects is displayed on the touch screen 151 . According to an example, the control screen 30 may be displayed according to a user's input. For example, as shown in FIG. 3 , when a user applies a predetermined touch input such as multi-touching one area of the touch screen 151 while video content is being displayed, the controller 180 controls the control screen ( 30) can be displayed.

However, the present invention is not limited thereto, and the predetermined touch input may include a short touch, a long touch, a multi-touch, a drag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipe touch, a hovering touch, etc. It can be set by a touch method or a combination thereof. In addition, the control screen 30 may be set to be displayed when there is input of a specific key or a specific input through a device such as a mouse or a remote control in addition to the touch input.

As shown in FIG. 6 , the control screen 30 may include information on each of the plurality of audio objects. The controller 180 may acquire information on an audio object currently output through the sound output unit 152 based on the metadata.

The controller 180 may display the position adjustment items 31 and 32 respectively corresponding to the audio objects 11 and 12 currently being output on the control screen. According to an example, as shown in FIG. 6 , the control screen 30 may include a sound image of the audio output signal based on the metadata.

The sound image may be a virtual image indicating a position sensed according to a sound of a sound source corresponding to each audio object. As indicated by the control screen 30 in FIG. 6 , the sound image may mean the location of the sound source of each audio object that is distinguished from the image displayed on the touch screen 151 . The controller 180 may generate the sound image based on the location of the audio object included in the metadata and acoustic characteristics according to the location.

According to an example, the controller 180 may display the position adjustment items 31 and 32 corresponding to the respective positions of the plurality of audio objects 11 and 12 on the sound image. For example, referring to the control screen 30 shown in FIG. 6 , the conversation sound 31 may be detected as being generated from the left and farther than the guitar sound 32 . The position of the position adjustment items 31 and 32 on the control screen 30 is according to the position of the audio object included in the metadata when the object-based audio content is generated, and the position of the image object displayed on the touch screen 151 is It may be displayed differently from the location.

Referring to FIG. 7 , the controller 180 may adjust the position of the corresponding audio object according to an input applied to the position adjustment items 31 and 32 . The user may change the position of the position adjustment item corresponding to the desired audio object in order to adjust the position of the desired audio object.

For example, in order to adjust the position of the currently output dialogue sound 11 , the user may touch and drag the position adjustment item 31 . As the position adjustment item 31 is moved to the upper end of the control screen 30 , the position of the sound source of the dialogue sound 11 becomes farther away. Accordingly, the controller 180 may decrease the volume of the conversation sound 11 .

Similarly, as the position adjustment item 31 is moved to the lower end of the control screen 30 , the position of the sound source of the dialogue sound 11 becomes closer. Accordingly, the controller 180 may increase the volume of the conversation sound 11 .

In addition, when the position adjustment item 31 moves left and right on the control screen 30 , the position of the corresponding dialogue sound 11 also moves left and right. Accordingly, the controller 180 may move the position of the conversation sound 11 left and right.

As shown in FIG. 7 , it is assumed that the user moves the position adjustment item 31 corresponding to the conversation sound 11 to the upper left corner. In this case, the position of the dialogue sound 11 is shifted further to the left and further away. Accordingly, the controller 180 may move the conversation sound 11 to the left while decreasing the volume.

According to an example, the controller 180 may display the user's location on the sound image of the control screen 30 . For example, when the user's position is in the center of the control screen 30 , the controller 180 may generate an audio output signal as if the guitar sound 12 is heard from behind the user's right side.

In the above description, it has been described on the premise that the sound image is configured in a planar manner, but this is an example and is not limited thereto. According to another example, the sound image may be generated in three dimensions. In this case, the controller 180 may also adjust the height of each audio object.

Although the description has been made on the premise that the control screens 10 and 30 are displayed according to a user's input, this is an example and is not limited thereto. According to another example, the control screens 10 and 30 may be displayed in an OSD (On Screen Display) method. When the object-based audio content is reproduced, the controller 180 may display the control screens 10 and 30 for the currently output audio object on one area of the touch screen 151 without a separate input based on the metadata. can

Referring again to FIG. 2 , the controller 180 may output an audio output signal reflecting the properties adjusted through the control screen through the sound output unit [S150].

For example, as shown in FIG. 7 , when the user moves the position adjustment item 31 corresponding to the conversation sound 11 to the upper left corner, the controller 180 controls the volume of the conversation sound 11 . It can be moved more to the left while decreasing . The controller 180 may change the position and volume of the audio object corresponding to the dialogue sound 11 in the audio output signal including the plurality of audio objects and output the changed. In this case, the volume and position of the dialogue sound 11 are changed in the output audio signal compared to the original object-based audio content.

In the above description, the content including the object-based audio content and the video content has been described on the premise, but the present invention is not limited thereto. The above description may be substantially the same even when there is only object-based audio content, as long as it is not contradictory or difficult to apply.

According to this, by displaying a control screen capable of adjusting the properties of each of the plurality of audio objects on one area of the touch screen 151, the user can conveniently change the properties of each audio object with a simple touch input when playing object-based audio content. can be adjusted In addition, by providing a control screen including a volume control item capable of adjusting the volume of each audio object when playing audio content, the user can easily adjust the volume of each audio object.

In addition, by providing a control screen including a position adjustment item capable of adjusting the position of each audio object during playback of audio content, the user can easily adjust the position of each audio object. In addition, by displaying the sound image of the audio output signal in the position adjustment item, the user can intuitively adjust the position of the audio object on the sound image.

In the above description, it is assumed that a separate control screen for adjusting the properties of the audio object is displayed, but the present invention is not limited thereto. When a predetermined touch input is applied to the audio object, the controller 180 may be set to enable adjustment of a corresponding property.

For example, when the speaker A shown in FIG. 3 is touched and dragged upward, the controller 180 may be set to increase the volume of the corresponding conversation sound 11 . However, this is an example and is not limited thereto. The predetermined touch input corresponds to each property to be adjusted, such as a short touch, a long touch, a multi-touch, a drag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipe touch, a hovering touch, etc. The same, various types of touch or a combination thereof may be set.

8 is a flowchart of a method for controlling an electronic device according to another embodiment of the present invention. FIG. 9 is a diagram for explaining displaying an image object corresponding to an audio object to be distinguished according to an embodiment of the present invention. 10 to 13 are diagrams for explaining adjustment of properties for each corresponding audio object using an image object according to an embodiment of the present invention.

The method for controlling an electronic device according to another embodiment of the present invention may be implemented in the electronic device 100 described with reference to FIG. 1 . Hereinafter, a method for controlling an electronic device according to an embodiment of the present invention and an operation of the electronic device 100 for implementing the method will be described in detail with reference to the necessary drawings.

Referring to FIG. 8 , the controller 180 may extract an image object respectively corresponding to the plurality of audio objects from the video content based on the metadata [S200].

The controller 180 may restore video content from the broadcast stream and display it on the touch screen 151 . Referring to FIG. 9 , the currently displayed image includes various image objects such as trees and chairs in addition to two people (A, B) and others (C) as objects. The controller 180 may extract an image object from the currently displayed image. A method of extracting an object in an image follows a known method, and further detailed description will be omitted.

According to an example, the controller 180 may reflect the metadata in relation to the extraction of the image object. The controller 180 may extract an image object corresponding to the audio object by using information such as the type of each audio object included in the metadata. For example, when the guitar is included as an audio object in the metadata for the audio content part corresponding to the currently displayed image, the guitar may be extracted as the image object. In this case, the controller 180 may increase the accuracy of extraction of the image object by using the position of the guitar sound or the general shape of the guitar.

Referring back to FIG. 8 , the controller 180 may display the extracted image object from the video content displayed on the touch screen to be distinguished from other image objects [S210].

Referring to FIG. 9 , the controller 180 may display image objects A and C corresponding to the currently output audio object in the currently displayed image to be distinguished from other image objects. For example, as shown in FIG. 9 , indicators 41 and 42 indicating image objects corresponding to the audio object may be displayed on the borders of the speaker A and the guitar C, which are image objects, respectively.

However, this is an example and is not limited thereto. If an image object corresponding to the audio object being output can be distinguished, various visual effects such as highlight processing and a specific color may be applied. Through this, the user can intuitively recognize the audio object currently being output.

According to an example, the indicators 41 and 42 may be displayed according to a user's input. For example, as shown in FIG. 3 , when a predetermined touch input is applied, such as when the user touches an area of the touch screen 151 twice while video content is being displayed, the controller 180 controls the indicator 41 , 42) can be displayed.

However, the present invention is not limited thereto, and the predetermined touch input may include a short touch, a long touch, a multi-touch, a drag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipe touch, a hovering touch, etc. It can be set by a touch method or a combination thereof. Also, when there is a specific key input or a specific input through a device such as a mouse or a remote controller in addition to the touch input, the indicators 41 and 42 may be set to be displayed.

Referring back to FIG. 8 , upon receiving a touch input for selecting the extracted image object [S220], the controller 180 displays a control screen for adjusting the properties of the audio object corresponding to the selected image object on the touch screen. It can be displayed in one area of [S230].

Referring to FIG. 10 , a user may select an image object A corresponding to an audio object whose properties are to be adjusted. According to an example, the selection of the image object A may be performed by a touch input through the touch screen 151 . However, this is an example and is not limited thereto. According to another example, the selection may be performed using an interface such as a separate remote control, a motion remote control, or a mouse. Alternatively, the selection may be performed by a separate external device or a mobile terminal.

According to an example, when the image object A is selected, the controller 180 may perform a process to indicate that the image object A is selected. For example, the controller 180 may increase the volume of the audio object corresponding to the selected image object A. Alternatively, the controller 180 may decrease the volume of the audio object corresponding to the image object except for the selected image object A. Alternatively, the controller 180 may display the selected image object A to be distinguished from other objects.

As shown in FIG. 10 , the controller 180 may display indicators 50 and 60 for adjusting the property of a dialogue sound that is an audio object corresponding to the selected image object A. The user may select the corresponding indicators 50 and 60 for volume control or position control.

When the user wants to adjust the volume of the audio object, the user may select the volume control indicator 50 . As shown in FIG. 11 , the controller 180 may display a control screen 51 for adjusting the volume of a dialogue sound that is an audio object corresponding to the selected image object A.

The control screen 51 may include information 52 of an audio object corresponding to the selected image object A. The controller 180 may display the volume control item 53 of the conversation sound 52 currently being output on the control screen 51 . The volume control item 53 may include a handle 54 for adjusting the volume.

The user may adjust the volume of the dialogue sound 52 by controlling the position of the handle 54 . Since the volume control using the volume control item 53 has been described above with reference to FIG. 4 , a detailed description thereof will be omitted.

When the user wants to adjust the position of the audio object, the user may select the position adjustment indicator 60 . As shown in FIG. 12 , the controller 180 may display a control screen 61 for adjusting the position of a dialogue sound that is an audio object corresponding to the selected image object A.

The control screen 61 may include information on an audio object corresponding to the selected image object A. The controller 180 may display the position adjustment item 62 of the conversation sound 52 currently being output on the control screen 61 .

The user may adjust the position of the dialogue sound 52 by controlling the position of the position adjustment item 62 . Since the position adjustment using the position adjustment item 62 has been described above with reference to FIG. 4 , further detailed description thereof will be omitted.

In the above, the case of selecting one image object has been described, but the present invention is not limited thereto. A user may simultaneously or sequentially select a plurality of image objects corresponding to a plurality of audio objects whose properties are to be adjusted. The controller 180 may display a control screen for adjusting the volume or position of the audio object corresponding to the selected image object.

In the above description, it is assumed that the control screens 51 and 61 are displayed according to a user's input, but the present invention is not limited thereto. Referring to FIG. 13 , according to another example, the controller 180 may display a control screen 10 capable of adjusting the properties of an audio object corresponding to an extracted image object regardless of a user input.

According to an example, the controller 180 may display the information 11 and 12 of the audio object so as to correspond to the indicators 41 and 42 displayed on the extracted image objects A and C. For example, referring to FIG. 13 , the information 11 of the audio object corresponding to the indicator 41 displayed on the image object A may be displayed in the same shape. Similarly, the information 12 of the audio object corresponding to the indicator 42 displayed on the image object C may be displayed in the same shape.

The user may adjust the volume of the corresponding audio objects 11 and 12 by using the volume control items 21 and 22 in the control screen 10 . Although only the case of volume control is illustrated in FIG. 13 , the case of position control may be applied substantially the same. Since the volume control using the volume control items 21 and 22 has been described above with reference to FIG. 4 , a detailed description thereof will be omitted.

Accordingly, by displaying the image object corresponding to the audio object to be distinguished, the user can intuitively recognize the image object corresponding to the audio object. In addition, by allowing the property of the audio object corresponding to the selected image object to be adjusted when the video content is reproduced, the user can conveniently adjust the property of the audio object.

14 is a flowchart of a method for controlling an electronic device according to another embodiment of the present invention. 15 to 18 are diagrams for explaining how to remotely adjust an attribute of an audio object through a control device according to an embodiment of the present invention.

The method for controlling an electronic device according to another embodiment of the present invention may be implemented in the electronic device 100 described with reference to FIG. 1 . Hereinafter, a method for controlling an electronic device according to an embodiment of the present invention and an operation of the electronic device 100 for implementing the method will be described in detail with reference to the necessary drawings.

Referring to FIG. 14 , the control unit 180 may communicate with the control device [S300] and share the screen displayed on the touch screen with the control device through the communication unit [S310].

Referring to FIG. 15 , it is illustrated that the electronic device 100 and the control device 200 are communicatively connected. Although a mobile terminal is illustrated as the control device 200 in FIG. 15 , the present invention is not limited thereto. The control device 200 is connected to the electronic device 100 through communication through the communication unit 110 , and any device capable of generating and transmitting a control signal to the electronic device 100 may be applied.

In addition, in the communication connection, any communication method may be applied as long as the screen displayed on the touch screen 151 can be shared with the control device 200 and the control signal can be transmitted through the communication unit 110 , and specific communication It is not limited to the method.

According to an example, the controller 180 may share the screen displayed on the touch screen 151 with the control device 200 . To this end, the controller 180 may transmit the object-based audio content and the corresponding video content to the controller 200 to the controller 200 .

However, this is an example and is not limited thereto. According to another example, the controller 180 may transmit a signal for receiving a broadcast stream including the object-based audio content and the corresponding video content to the control device 200 . Alternatively, according to another example, the controller 180 may transmit only information about the object-based audio content to the control device 200 without sharing the screen.

Referring back to FIG. 14 , the controller 180 may select the extracted image object according to the control signal received from the control device 200 [S320].

As described with reference to FIG. 9 , the controller 180 may extract an image object corresponding to the currently output audio object. For this, in order to avoid duplication of description, further detailed description will be omitted.

Referring to FIG. 16 , the same screen as the screen displayed on the touch screen 151 of FIG. 15 may be displayed on the display 251 of the control device 200 . According to an example, as shown in FIG. 9 , when indicators 41 and 42 for distinguishing extracted image objects are shown, the indicators may also be displayed on the display 251 .

The user may select an image object A' corresponding to an audio object whose properties are to be adjusted on the display 251 of the control device 200 . According to an example, the controller 180 may receive a control signal including the coordinates of the touch input applied to the display 251 .

The controller 180 may select the image object A located at the coordinates on the touch screen 151 corresponding to the coordinates of the touch input included in the received control signal. To this end, according to an example, the controller 180 may match the coordinates of the touch screen 151 and the display 251 when communicating with the control device 200 .

However, this is an example and is not limited thereto. As long as an image object on the touch screen 151 corresponding to the image object selected on the display 251 can be selected, any method may be applied.

Referring back to FIG. 14 , the controller 180 may display a control screen for adjusting the properties of the audio object corresponding to the selected image object on one region of the touch screen [S330].

Referring to FIG. 17 , the controller 180 may display an image object A corresponding to the image object A′ selected on the display 251 to be distinguished from other image objects. To this end, the controller 180 may display the indicator 70 on the edge of the selected image object A, as shown in FIG. 17 .

However, this is an example and is not limited thereto. In order to emphasize the selected image object (A), the controller 180 may perform highlight processing, application of visual effects such as a specific color or contrast adjustment, adjustment of the volume of a corresponding audio object, or application of a specific audio filter. have.

Also, the controller 180 may display the control screen 80 for adjusting the properties of the audio object corresponding to the selected image object A on one area of the touch screen 151 . As shown in FIG. 17 , the control screen 80 may include information 81 of an audio object corresponding to the selected image object A. As shown in FIG. The controller 180 may acquire information on the audio object based on the metadata.

The controller 180 may display a volume control item 82 for the audio object 81 on the control screen 80 . The controller 180 may display a position adjustment item 83 for the audio object 81 on the control screen 80 .

The controller 180 may transmit a control signal for displaying the control screen 80 on the display 251 to the control device 200 . The control unit of the control device 200 may display the control screen 210 on the display 251 as shown in FIG. 18 according to the control signal.

Referring back to FIG. 14 , the controller 180 receives the control signal including information on a touch input applied to the control screen displayed on the display [S340], and audio reflecting properties adjusted according to the control signal An output signal may be output through the sound output unit [S350].

Referring to FIG. 18 , the user receives information 211 of an audio object included in the control screen 210 displayed on the display 251 of the control device 200 , a volume control item 212 , and a position control item 213 . can be used to adjust the properties of the audio object 211 . According to an example, the control screen 210 may be displayed only when the image object A' is touched, as shown in FIG. 18 . However, this is an example and is not limited thereto.

For example, it is assumed that the user moves the handle of the volume control item 212 to the left. The controller of the control device 200 may generate a control signal according to the movement of the handle on the volume control item 212 . According to an example, the control signal may include information on the movement of the handle. In this case, the controller 180 may receive the control signal, and may equally move the handle to the left on the volume control item 82 according to the received control signal. The controller 180 may decrease the volume of the audio object 81 according to the position of the moved handle.

As another example, it is assumed that the user moves the position of the audio object upward on the position adjustment item 213 in the control device 200 . The controller of the control device 200 may generate a control signal according to the movement of the audio object on the position adjustment item 213 . According to an example, the control signal may include information on movement of the audio object. In this case, the controller 180 may receive the control signal and may equally move the position of the audio object upward on the position adjustment item 83 according to the received control signal. The controller 180 may decrease the volume so that the position of the audio object 81 is output as if the position of the audio object 81 has moved away according to the position of the moved audio object.

The controller 180 may output an audio output signal reflecting the adjusted property through the sound output unit 152 . Since this is substantially the same as that described above in FIG. 2 , further detailed description thereof will be omitted.

In the above description, it is assumed that the screen is shared, but the present invention is not limited thereto. When only object-based audio content is shared, the location of the audio object using metadata included in the audio content may be displayed on the display of the control device 200 . When the user selects a specific audio object, the controller of the control device 200 may transmit a control signal including information on the selected audio object to the electronic device 100 . The controller 180 may select an audio object according to the control signal. Since the subsequent adjustment of the properties of the audio object is the same as described above, a detailed description thereof will be omitted.

Accordingly, by allowing the control device sharing the screen to adjust the properties of the audio object for the image object selected, the user can conveniently adjust the properties of the audio object through a separate device.

The present invention described above can be implemented as computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that includes implementation in the form of. Also, the computer may include the controller 180 of the electronic device. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

100: electronic device 110: communication unit
120: input unit 140: sensing unit
150: output unit 160: interface unit
170: memory 180: control unit
190: power supply

Claims (14)

a touch screen displaying video content corresponding to object-based audio content;
sound output unit;
a plurality of audio objects and metadata related to the object-based audio content are restored from the object-based audio content, and an audio output signal including the plurality of audio objects is generated based on the metadata, and the audio a control unit outputting an output signal through the sound output unit; and
a communication unit that communicates with the control device;
including,
The control unit displays a control screen capable of adjusting the properties of each of the plurality of audio objects based on the metadata on one area of the touch screen, and generates an audio output signal reflecting the properties adjusted through the control screen. output through the output unit,
The control unit is
extracting an image object respectively corresponding to the plurality of audio objects from the video content based on the metadata;
Displaying the extracted image object from the video content displayed on the touch screen to be distinguished from other image objects,
When a touch input for selecting the extracted image object is received, a control screen capable of adjusting a property of an audio object corresponding to the selected image object is displayed on an area of the touch screen,
Share the screen displayed on the touch screen with a control device through the communication unit, select the extracted image object according to a control signal received from the control device, and adjust the properties of the audio object corresponding to the selected image object Displaying a control screen in one area of the touch screen,
Electronic device, characterized in that. The method of claim 1,
The metadata includes information related to the generation of the audio output signal including the type, location, sound intensity, and acoustic characteristics according to the location of each of the plurality of audio objects. The method of claim 1,
The property of each of the plurality of audio objects comprises a volume or position of each audio object. 4. The method of claim 3,
The control screen includes a volume control item corresponding to each of the plurality of audio objects,
The electronic device, characterized in that the control unit adjusts the volume of the audio object corresponding to the input applied to the volume control item. 4. The method of claim 3,
The control screen includes a position adjustment item corresponding to each of the plurality of audio objects,
The control unit adjusts the position of the audio object corresponding to the input applied to the position adjustment item. 6. The method of claim 5,
The control screen includes a sound image of the audio output signal based on the metadata,
The control unit displays the position adjustment item on the sound image and adjusts the position of the corresponding audio object according to the input. The method of claim 1,
The electronic device, characterized in that the control screen is displayed in an OSD (On Screen Display) method. delete delete delete delete The method of claim 1,
The control unit receives a control signal including the coordinates of the touch input applied to the display of the control device, and selects an image object located at the coordinates on the touch screen corresponding to the coordinates of the touch input . The method of claim 1,
The control unit receives the control signal including information on a touch input applied to a control screen displayed on a display of the control device, and outputs an audio output signal reflecting the properties adjusted according to the control signal through the sound output unit Electronic device, characterized in that.
delete

Images