KR20220111552A - Electronic apparatus and controlling method thereof - Google Patents

Abstract

An electronic apparatus includes: a communication interface, a display, and a processor that obtains first characteristic information corresponding to a first content when the first content is received from a first device through the communication interface, synchronizes playback timings of the first content and the second content based on the first characteristic information and the second characteristic information when the second characteristic information corresponding to the first content and the second content are received from a second device through the communication interface, and controls the display to display the synchronized first content and the synchronized second content.

Description

ELECTRONIC APPARATUS AND CONTROLLING METHOD THEREOF

The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device for simultaneously displaying contents from different sources and a method for controlling the same.

Broadcasters transmit copyrighted content (eg, a soccer game) to authorized viewers, who can view the copyrighted content.

As the number of situations in which personal broadcasting is conducted such as one-person media increases, there may be a situation in which a personal broadcasting producer combines copyrighted content and personal content and transmits it to new viewers.

Personal broadcast producers can overlay their own personal content on copyrighted content and reprocess it into a single stream. For example, a personal broadcast producer may combine additional personal content such as images acquired through a camera and real-time chatting with existing copyrighted content.

Even if a personal broadcast producer does not infringe copyright when viewing copyrighted content received from a broadcaster, processing or redistributing copyrighted content may infringe copyright.

Accordingly, a personal broadcast producer may transmit only personal content to viewers, excluding copyright content, in order to avoid reprocessing and redistribution of copyright content.

However, viewers were inconvenient in not being able to watch both copyrighted content and personal content.

The present disclosure has been devised to improve the above-described problems, and an object of the present disclosure is to provide an electronic device that receives different contents from separate devices and synchronizes playback times of different contents, and a method for controlling the same.

When first content is received from a first device through a communication interface, a display, and the communication interface, the electronic device according to the present embodiment for achieving the above object obtains first characteristic information corresponding to the first content, , when second characteristic information and second content corresponding to the first content are received from a second device through the communication interface, the first content and the second content are based on the first characteristic information and the second characteristic information and a processor that synchronizes the reproduction times of the two contents, and controls the display to display the first content and the second content with which the reproduction times are synchronized.

Meanwhile, the processor may control the display to display the first content on a first area of the display, and control the display to display the second content on a second area of the display, The reproduction timing of the first content and the second content may be synchronized based on the first characteristic information corresponding to the displayed first content and the second characteristic information corresponding to the displayed second content.

Meanwhile, the first characteristic information may include at least one of first image data of the first content and first audio data of the first content obtained by the electronic device, and the second characteristic information may include the first image data of the first content. It may include at least one of second image data of the first content acquired by the second device or second audio data of the first content.

Meanwhile, when the first characteristic information and the second characteristic information do not match, the processor may synchronize reproduction times of the first content and the second content.

Meanwhile, when the first characteristic information and the second characteristic information do not match, the processor may change a reproduction setting of one of the first content and the second content.

Meanwhile, the processor may change the reproduction setting of one of the first content and the second content by temporarily stopping the reproduction of the contents, decreasing the reproduction speed of the contents, or increasing the reproduction speed of the contents.

Meanwhile, the processor may identify a content having a faster playback time from among the first content and the second content based on the first characteristic information and the second characteristic information, and set the playback setting of the identified content. can be changed

Meanwhile, the electronic device may further include a memory, and the processor may store the identified content in the memory, and pause the reproduction of the identified content or decrease the playback speed of the identified content to identify the identified content. You can change the playback settings of the contents.

Meanwhile, the processor may maintain a reproduction speed of content different from the identified content.

Meanwhile, the processor may control the display to display a user interface (UI) including at least one device that provides content corresponding to the first content, and selects the second device through the UI. Upon receiving a user input for , the communication interface may be controlled to transmit a control signal for requesting the second characteristic information and the second content to the second device.

A method of controlling an electronic device according to an embodiment of the present disclosure includes, when first content is received from a first device, acquiring first characteristic information corresponding to the first content, and accessing the first content from a second device. When the corresponding second characteristic information and the second content are received, synchronizing the playback timings of the first content and the second content based on the first characteristic information and the second characteristic information, and synchronizing the playback timings and displaying the first content and the second content.

Meanwhile, the control method may further include displaying the first content on a first area of the display of the electronic device and displaying the second content on a second area of the display, The synchronizing may include synchronizing playback times of the first content and the second content based on first characteristic information corresponding to the displayed first content and second characteristic information corresponding to the displayed second content.

Meanwhile, the first characteristic information may include at least one of first image data of the first content and first audio data of the first content obtained by the electronic device, and the second characteristic information may include the first image data of the first content. It may include at least one of second image data of the first content acquired by the second device or second audio data of the first content.

Meanwhile, in the synchronizing of the reproduction time, when the first characteristic information and the second characteristic information do not match, the reproduction time of the first content and the second content may be synchronized.

Meanwhile, in the synchronizing of the playback time, when the first characteristic information and the second characteristic information do not match, the playback setting of one of the first content and the second content may be changed.

On the other hand, the synchronizing the playback time may change the playback setting of one of the first content or the second content by pausing the playback of the content, decreasing the playback speed of the content, or increasing the playback speed of the content. have.

Meanwhile, the synchronizing the playback time may include identifying a content having a faster playback time from among the first content and the second content based on the first characteristic information and the second characteristic information, You can change the content playback settings.

Meanwhile, the synchronizing the playback time may store the identified content in a memory, and change the playback setting of the identified content by pausing the playback of the identified content or reducing the playback speed of the identified content. can

Meanwhile, the synchronizing the playback timing may maintain the playback speed of the identified content and other content.

On the other hand, displaying a user interface (UI) including at least one device providing content corresponding to the first content, and when a user input for selecting the second device is received through the UI, the second device The method may further include transmitting a control signal for requesting the second characteristic information and the second content to the second device.

1 is a diagram for describing an electronic device that simultaneously displays first content received from a first device and second content received from a second device.
2 is a diagram for explaining first content and second content, respectively.
3 is a diagram for explaining whether the first content and the second content are synchronized.
4 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.
FIG. 5 is a block diagram illustrating a detailed configuration of the electronic device of FIG. 4 .
6 is a flowchart illustrating a content display operation of an electronic device according to an embodiment of the present disclosure.
7 is a flowchart illustrating a content matching operation of an electronic device according to an embodiment of the present disclosure.
FIG. 8 is a diagram for explaining a UI displayed in the matching operation of FIG. 7 .
9 is a flowchart illustrating an operation of synchronizing first content and second content of an electronic device according to an embodiment of the present disclosure.
10 is a flowchart illustrating an operation of changing a content reproduction setting of an electronic device according to an embodiment of the present disclosure.
11 is a flowchart illustrating an operation of changing a content reproduction setting of an electronic device according to another embodiment of the present disclosure.
12 is a flowchart illustrating an operation of changing a content reproduction setting of an electronic device according to another embodiment of the present disclosure.
13 is a diagram for explaining an operation of changing a content reproduction setting of an electronic device according to various embodiments of the present disclosure;
14 is a diagram for explaining an embodiment in which a playback time of a first content is earlier than a playback time of a second content.
15 is a diagram for explaining an embodiment in which a playback time of a first content is slower than a playback time of a second content.
16 is a system flow diagram for explaining contents synchronization operations of an electronic device, a first device, and a second device.
17 is a flowchart illustrating an operation of acquiring second characteristic information of a second device.
18 is a system flowchart illustrating an operation of acquiring second characteristic information of a second device according to an embodiment of the present disclosure.
19 is a diagram for explaining an operation of identifying a reproduction time point by comparing first characteristic information and second characteristic information, according to an embodiment of the present disclosure.
20 is a system flowchart for explaining an operation of acquiring second characteristic information of a second device according to another embodiment of the present disclosure.
21 is a diagram for explaining an operation of identifying a reproduction time point by comparing first characteristic information and second characteristic information, according to another embodiment of the present disclosure.
22 is a diagram for explaining an operation of identifying a reproduction time point by comparing first characteristic information and second characteristic information, according to another embodiment of the present disclosure;
23 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

The terms used in the embodiments of the present disclosure have been selected as currently widely used general terms as possible while considering the functions in the present disclosure, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding disclosure. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

In the present specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

The expression “at least one of A and/or B” is to be understood as indicating either “A” or “B” or “A and B”.

As used herein, expressions such as "first," "second," "first," or "second," can modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it should be understood that a component may be directly connected to another component or may be connected through another component (eg, a third component).

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It is to be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In the present disclosure, a "module" or "unit" performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of “modules” or a plurality of “units” are integrated into at least one module and implemented with at least one processor (not shown) except for “modules” or “units” that need to be implemented with specific hardware. can be

In this specification, the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

Hereinafter, an embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings.

1 is a diagram for describing an electronic device that simultaneously displays first content received from a first device and second content received from a second device.

Referring to FIG. 1 , a system 1000 may include an electronic device 100 , a first device 200 , and a second device 300 . Here, the electronic device 100 may simultaneously display the first content and the second content. The first content may be content received from the first device 200 , and the second content may be content received from the second device 300 .

Here, the electronic device 100 may refer to various types of electronic devices for viewing content. For example, the electronic device 100 may mean a TV, a smartphone, a tablet, a PC, or the like.

Here, the first device 200 may mean a server that provides the first content. Here, the first content may mean content provided to a plurality of viewers. For example, the first content may mean content provided by a broadcaster or content provided by an OTT service company.

Here, the second device 300 may mean a device that provides personal broadcasting. Here, the second device 300 may refer to a device that generates and transmits a personal broadcast while viewing the first content provided by the first device 200 . Here, the second content may mean personal broadcast content. For example, the second content may mean content provided by a personal channel.

The electronic device 100 may receive the first content from the first device 200 and may receive the second content from the second device 300 . In addition, the first content and the second content may be displayed at the same time.

For example, it is assumed that the first device 200 is a broadcaster server that relays a soccer game, and the first content is soccer game content. The second device 300 may generate the second content while receiving the first content from the first device 200 . Here, the second content may be a relay content of a user of the second device 300 watching the first content. The electronic device 100 may receive official soccer content from the first device 200 and may receive personal relay content from the second device 300 . In addition, the electronic device 100 may simultaneously display the official soccer content and the personal relay content.

2 is a diagram for explaining first content and second content, respectively.

Referring to FIG. 2 , a timeline 201 of the first content according to an exemplary embodiment will be described. The first content may include a plurality of images 201-1, 201-2, and 201-3 and a plurality of audios (not shown). As for the first content, a first image 201-1 may be displayed at 1 second, a second image 201-2 may be displayed at 2 seconds, and a third image 201-3 may be displayed at 3 seconds. .

Also, a timeline 202 of the second content according to an embodiment will be described. The second content may include a plurality of images 202-1, 202-2, 202-3 and a plurality of audios (not shown). As for the second content, a fourth image 202-1 may be displayed in 1 second, a fifth image 202-2 may be displayed in 2 seconds, and a sixth image 202-3 may be displayed in 3 seconds. .

Here, the user of the second device 300 may generate the second content while watching the first content. Accordingly, the timeline 201 of the first content and the timeline 202 of the second content may be synchronized with each other. For example, if the electronic device 100 of FIG. 1 normally completes the synchronization operation, the first image 201-1 and the fourth image 202-1 are simultaneously displayed and the second image 201-2 and a fifth image 202 - 2 may be simultaneously displayed, and a third image 201-3 and a sixth image 202 - 3 may be simultaneously displayed.

Meanwhile, the second content may additionally include audio information in addition to image information. For example, the audio 202-4 corresponding to the fourth image 202-1 may be “one person dribbles”. The audio 202 - 5 corresponding to the fifth image 202 - 2 may be "two people compete to see". The audio 202 - 6 corresponding to the sixth image 202 - 3 may be "three people compete for a ball".

3 is a diagram for explaining whether the first content and the second content are synchronized.

Referring to FIG. 3 , according to an embodiment 305 , the first content and the second content may be in a normally synchronized state. Here, the electronic device 100 may simultaneously display the first image 201-1 of the first content and the fourth image 202-1 of the second content. Accordingly, the user of the electronic device 100 may view the second content suitable for the content of the displayed first content.

According to another embodiment 310, the first content and the second content may be out of synchronization. Here, the electronic device 100 may display the first content received from the first device 200 and the second content received from the second device 300 without synchronization. Here, the sync may not match due to the delay of the first content or the second content. For example, while a first image 201-1 of a first content in which one soccer player dribbles a ball is displayed, the electronic device 100 displays a second image 201-1 that is not suitable for the first image 201-1. A fifth image 202 - 2 of content may be displayed. Accordingly, the user of the electronic device 100 may feel that the displayed first content and the second content are not synchronized.

4 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 4 , the electronic device 100 may include a communication interface 110 , a display 120 , and a processor 130 .

Electronic devices according to various embodiments of the present specification may include, for example, a smartphone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, a PMP ( portable multimedia player), an MP3 player, a medical device, a camera, or a wearable device. A wearable device may be an accessory (e.g., watch, ring, bracelet, anklet, necklace, eyewear, contact lens, or head-mounted-device (HMD)), a textile or clothing integral (e.g. electronic garment); It may include at least one of a body-worn (eg, skin pad or tattoo) or bioimplantable circuit In some embodiments, the electronic device may include, for example, a television, a digital video disk (DVD) player, Audio, refrigerator, air conditioner, vacuum cleaner, oven, microwave oven, washing machine, air purifier, set-top box, home automation control panel, security control panel, media box (e.g. Samsung HomeSyncTM, Apple TVTM, or Google TVTM), game console (eg, XboxTM, PlayStationTM), an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame.

The communication interface 110 is configured to communicate with various types of external devices according to various types of communication methods. The communication interface 110 includes a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module. Here, each communication module may be implemented in the form of at least one hardware chip.

The communication interface 110 may communicate with a Wi-Fi module and a Bluetooth module using a Wi-Fi method and a Bluetooth method, respectively. In the case of using a Wi-Fi module or a Bluetooth module, various types of connection information such as an SSID and a session key are first transmitted and received, and various types of information can be transmitted/received after communication connection using this.

The infrared communication module communicates according to the infrared data association (IrDA) technology, which wirelessly transmits data in a short distance using infrared that is between visible light and millimeter wave.

In addition to the above-described communication method, the wireless communication module includes Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), LTE Advanced (LTE-A), 4th Generation (4G), 5G It may include at least one communication chip that performs communication according to various wireless communication standards such as (5th Generation).

In addition, the communication interface 110 is at least one of a wired communication module for performing communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, an optical fiber cable, or a UWB (Ultra Wide-Band) module, etc. may include.

The display 120 may be implemented as various types of displays, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), and the like. The display 120 may also include a driving circuit, a backlight unit, and the like, which may be implemented in the form of an a-si TFT, a low temperature poly silicon (LTPS) TFT, or an organic TFT (OTFT). Meanwhile, the display 120 may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display, or the like.

Also, according to an embodiment of the present disclosure, the display 120 may include a bezel housing the display panel as well as a display panel for outputting an image. In particular, according to an embodiment of the present disclosure, the bezel may include a touch sensor (not shown) for detecting user interaction.

The processor 130 may perform an overall control operation of the electronic device. Specifically, the processor 130 functions to control the overall operation of the electronic device.

The processor 130 may be implemented as a digital signal processor (DSP), a microprocessor (microprocessor), or a time controller (TCON) for processing a digital signal, but is not limited thereto, and a central processing unit ( central processing unit (CPU), micro controller unit (MPU), micro processing unit (MPU), controller, application processor (AP), graphics-processing unit (GPU) or communication processor (CP)), may include one or more of an ARM processor, or may be defined by a corresponding term In addition, the processor 130 is a SoC (System on Chip) or LSI (large scale integration) with a built-in processing algorithm. It may be implemented in the form of a field programmable gate array (FPGA), and the processor 130 may perform various functions by executing computer executable instructions stored in a memory.

When the first content is received from the first device 200 through the communication interface 110 , the processor 130 may acquire first characteristic information corresponding to the first content, and may obtain the first characteristic information through the communication interface 110 . When the second feature information and the second content corresponding to the first content are received from the second device 300 , the playback timing of the first content and the second content may be synchronized based on the first feature information and the second feature information. and the display 120 may be controlled to display the synchronized first content and the synchronized second content.

Here, the first device 200 may mean a content providing server that provides copyrighted content to viewers who have legitimate rights.

According to an embodiment, the first device 200 may mean a broadcasting station server. And, the first content may mean copyright content provided by a broadcasting station server. Here, the first content may mean content provided in real time from the first device 200 . For example, the first content may mean soccer game content provided to authorized viewers.

According to another embodiment, the first device 200 may mean an over-the-top (OTT) server. And, the first content may mean copyright content provided by the OTT server. Here, the first content may mean content provided in response to a request of a viewer. For example, the first content may mean movie content directly selected by a viewer.

Meanwhile, the second device 300 may refer to a device of a personal content creator that provides personal content related to copyright content provided by the first device 200 or a server that provides personal content.

The second device 300 may receive the first content directly from the first device 200 and may generate second content related to the first content. For example, the second device 300 may start a personal broadcast while watching the soccer game content received from the first device 200 . In addition, the second device 300 may transmit personal content related to personal broadcasting to a user (or a viewer).

The processor 130 may control the communication interface 110 to receive the first content from the first device 200 and receive the second content from the second device 300 . The processor 130 may receive the first content and the second content from separate devices (or sources), respectively. Meanwhile, the first content may be described as the main content and the second content may be described as the sub content.

Here, the processor 130 may acquire first characteristic information corresponding to the first content based on the first content received from the first device 200 . In addition, the processor 130 may acquire second characteristic information corresponding to the first content from the second device 300 . Here, both the first characteristic information received from the first device 200 and the second characteristic information received from the second device 300 may be information obtained based on the first content.

Here, the feature information may refer to information that can specify a reproduction time point. Accordingly, the feature information may refer to one of image data and audio data obtained from content. The processor 130 may specify a playback time point based on data obtained from content.

Here, the reproduction time may mean a reproduction time of the currently displayed frame, and may mean information indicating which frame among a plurality of frames is being reproduced.

As an example, the content may be received only in real time and may be content that cannot be changed at a playback time (eg, a soccer game relayed in real time). Here, the reproduction time point may mean time information of a precedence relation. Also, the playback time may mean absolute time information (eg, 13:20:15 standard time).

As another example, it may be content (eg, movie content provided based on a user's request) in which the entire content is already stored and the playback time can be changed. Here, the reproduction time may mean a reproduction time (eg, 15 minutes and 30 seconds out of 60 minutes) of the currently reproduced frame in the reproduction time of the entire content.

Meanwhile, the processor 130 may determine only whether the playback timings between the first content and the second content match, without individually specifying the playback timing of the content. Specifically, the processor 130 may compare the first characteristic information and the second characteristic information, compare the data pattern included in the first characteristic information and the second characteristic information, how quickly the first content than the second content ( or slow) can be identified.

Here, the first characteristic information may include first data obtained from the first content. Here, the first data may include at least one of first image data and first audio data obtained based on the first content.

Here, the second characteristic information may include second data obtained from the first content and frame information of the second content corresponding to the second data. Here, the second data may include at least one of second image data and second audio data obtained based on the first content.

The second device 300 may transmit the second data obtained based on the first content, not the first content, to the electronic device 100 in order to avoid copyright infringement. Here, since the second data does not mean the original image, copyright infringement can be avoided. However, there is a need for the second device 300 to inform viewers at which playback time point of the first content the second content corresponds. This is because viewers have to match the sync of the first content and the second content. Accordingly, the second device 300 may obtain information obtained by matching each frame (or some reference frames) included in the second content with the second data obtained from the first content as the second characteristic information. .

For example, it is assumed that the second data are pixel values of image data and are #112233, #223344, #334455, #445566, and #556677. Here, the second device 300 generates frame #01 of the second content while the frame of the first content corresponding to #112233 is provided, and while the frame of the first content corresponding to #223344 is provided, the second device 300 is provided. Frame #02 of the content is generated, frame #03 of the second content is generated while the frame of the first content corresponding to #334455 is provided, and frame #03 of the second content is generated while the frame of the first content corresponding to #445566 is provided. It is assumed that frame #04 of the content is generated and frame #05 of the second content is generated while the frame of the first content corresponding to #556677 is provided. The second device 300 includes the second data #112233, #223344, #334455, #445566, and #556677 of the first content and #01, #02, #03, #04, # which are frame information of the second content. 05 may be matched to obtain the second characteristic information.

The processor 130 may identify which playback timing of the first content corresponds to the playback timing of the second content based on the second data included in the second characteristic information and frame information of the second content.

For example, if frame #03 of the second content is displayed on the display 120 , the processor 130 may identify that it should correspond to #334455 of the first content. When the pixel value corresponding to the display frame of the first content received through the first device 200 is #334455, the processor 130 determines that the playback timings of the first content and the second content match. can be identified However, if the pixel value corresponding to the display frame of the first content received through the first device 200 is not #334455, the processor 130 identifies that the playback timing of the first content and the second content do not match, , it may be determined that synchronization of the playback time is necessary.

The playback time synchronization operation of the present disclosure is to match the synchronization of the first content and the second content. Here, the information as a reference for matching the sync may be characteristic information obtained from the first content.

Specifically, the processor 130 may display the first content and the second content at the same time, and in order to determine whether the playback timing of the first content and the second content match, the first content indicating the playback timing of the first content The characteristic information and the second characteristic information indicating the playback time of the second content may be displayed. Here, the comparison criterion may be first data corresponding to the first content included in the first characteristic information and second data corresponding to the first content included in the second characteristic information.

Meanwhile, the second characteristic information may be information used to specify a reproduction time (second reproduction time) of the second content during an operation of synchronizing the reproduction times of the first content and the second content. Specifically, the second characteristic information may include data corresponding to the first content and frame information of the second content matching the data.

An embodiment of synchronizing the first content and the second content will be described in detail with reference to FIGS. 14 to 23 . In addition, embodiments related to the first characteristic information and the second characteristic information will be described in detail with reference to FIGS. 19 and 21 .

Meanwhile, the processor 130 may control the display 120 to display the first content on the first area of the display 120 , and the display 120 to display the second content on the second area of the display 120 . ) can be controlled, and playback times of the first content and the second content can be synchronized based on the first characteristic information corresponding to the displayed first content and the second characteristic information corresponding to the displayed second content.

The display 120 may include a plurality of areas. Specifically, the display 120 may include a first area and a second area. The processor 130 may display the first content on the first area of the display 120 and display the second content on the second area of the display 120 . According to an embodiment, the first area and the second area may be areas in which the entire display area is independently separated. The electronic device 100 of FIG. 1 may divide the display into a left area and a right area and display each content in a separate area. According to another exemplary embodiment, the first area and the second area may be areas in which at least some areas overlap. For example, all or part of the second area may be included in the first area, and all or part of the first area may be included in the second area.

Here, the processor 130 may obtain first characteristic information corresponding to the first content currently displayed in the first area, and obtain second characteristic information corresponding to the second content currently displayed in the second area. . Here, the feature information may be information corresponding to each frame. Accordingly, the characteristic information may also be different depending on the frame of the currently reproduced content.

Here, the processor 130 may obtain the first characteristic information (first data) corresponding to the frame of the displayed first content and the second characteristic information (the second data) corresponding to the frame of the displayed second content. have. In addition, the processor 130 may synchronize the playback timing of the first content and the second content based on the acquired first data and the second data. A detailed description of the synchronization operation will be described later with reference to FIGS. 9 to 12 .

Meanwhile, the first characteristic information may include at least one of first image data of the first content obtained by the electronic device 100 or first audio data of the first content, and the second characteristic information may include the second device ( 300) may include at least one of the second image data of the first content and the second audio data of the first content.

The processor 130 may receive the first content from the first device 200 and acquire at least one of first image data and first audio data corresponding to the first content based on the received first content. .

In addition, the second device 300 receives the first content from the first device 200 , and generates at least one of second image data or second audio data corresponding to the first content based on the received first content. can be obtained

Here, the first image data and the second image data may mean pixel values corresponding to at least one frame among a plurality of frames included in the first content. The first audio data and the second audio data may be a sound level value or a sound waveform corresponding to at least one frame among a plurality of frames included in the first content.

Meanwhile, the above-described image data and audio data may be converted by a predetermined hash function. Since the image data itself or the audio data itself has a large data size, the data may be transformed for a comparison operation.

Meanwhile, when the first characteristic information and the second characteristic information do not match, the processor 130 may synchronize the reproduction timing of the first content and the second content.

Here, when the information (first data) corresponding to the frame of the displayed first content does not match the information (the second data) corresponding to the frame of the displayed second content, the processor 130 determines the first content and the second content. 2 It can be identified that the sync of the content is inconsistent. In addition, the processor 130 may synchronize playback times of the first content and the second content.

Meanwhile, when the first characteristic information and the second characteristic information do not match, the processor 130 may change the reproduction setting of one of the first content and the second content.

Here, if the first data included in the first characteristic information and the second data included in the second characteristic information do not match, the processor 130 changes the reproduction setting of one of the first content and the second content, , the playback times of the first content and the second content may be synchronized (or matched).

Meanwhile, the processor 130 may change the reproduction setting of one of the first content and the second content by pausing the reproduction of the contents, decreasing the reproduction speed of the contents, or increasing the reproduction speed of the contents.

Here, the operation of changing the reproduction setting may mean one of an operation of pausing the reproduction of content, an operation of increasing the reproduction speed of the content, or an operation of decreasing the reproduction speed of the content.

Meanwhile, the processor 130 may identify a content having a faster playback time from among the first content and the second content based on the first characteristic information and the second characteristic information, and may change the playback setting of the identified content. .

Here, the processor 130 may compare the first playback timing of the first content with the second playback timing of the second content. In addition, the processor 130 may identify a faster reproduction time among the first reproduction time and the second reproduction time, and identify the content corresponding to the identified reproduction time. In addition, the processor 130 may change the playback setting of the content having a faster playback time.

The reason for changing the playback setting of the content having a faster playback time is that the electronic device 100 itself provides the first content and second 2 This is because the playback time of the contents can be synchronized.

The easiest way to synchronize the playback timings of the first content and the second content is to request the first device 200 or the second device 300 for content based on the changed playback timing, respectively. For example, the electronic device 100 may request a frame corresponding to 12 seconds instead of a frame corresponding to 10 seconds, and receive a frame corresponding to 12 seconds.

However, it is possible to request content corresponding to the changed playback time from each device providing the content only when each device stores the content corresponding to the entire playback timing. For example, the first device 200 or the second device 300 must store all 60 minutes of movie content data to transmit the content corresponding to the changed playback time. If a service function for changing a playing time or a sports game broadcast in real time is not separately provided, it may be difficult for the processor 130 to synchronize the playback timing of the first content and the second content.

Meanwhile, the electronic device 100 may further include a memory 140 , and the processor 130 stores the identified content (the first content or the second content having a faster playback time) in the memory 140 . Also, the playback setting of the identified content may be changed by pausing the playback of the identified content or decreasing the playback speed of the identified content.

This may mean that the processor 130 stores the identified content in a specific space of the memory 140 . Here, the specific space may mean a non-transitory storage space, and may mean a space stored until a predetermined event occurs.

In general, data displayed as streaming is stored in specific hardware as soon as it is received by the electronic device 100 , but may not be continuously stored in the memory 140 . For example, the processor 130 may delete the displayed streaming data after displaying the streaming data on the display 120 .

However, the electronic device 100 according to an embodiment of the present disclosure may store streaming data corresponding to the identified content in the memory 140 . And, the stored streaming data may be continuously stored in the memory 140 until a predetermined event is identified.

Here, the predetermined event may refer to an event in which playback times of the first content and the second content are synchronized. This is because when the playback timings of the first content and the second content are synchronized, it is no longer necessary to separately and continuously store the content in the memory 140 .

On the other hand, the processor 130 may receive and store the content having a faster playback time until a predetermined event occurs. Here, the processor 130 may pause or decrease the playback speed of the content having a fast playback time. By stopping or delaying the playback of the content having a faster playback time, the processor 130 may match the playback timing of the first content and the second content.

Meanwhile, the processor 130 may maintain the playback speed of the identified content and other content.

Here, the processor 130 may obtain a difference between the reproduction time points of the first content and the second content, and may change the reproduction setting of the identified content based on the obtained difference between the reproduction time points. When the difference between the playback timings is identified, the processor 130 may change the playback speed of the first content and the playback speed of the second content differently.

Here, it is assumed that the playback timing of the first content is earlier than the playback timing of the second content.

Here, the processor 130 may maintain the playback speed of the second content while temporarily stopping the playback of the first content. For example, assuming that the playback time difference between the first content and the second content is 1 second, the processor 130 pauses the first content for 1 second and reproduces the second content at the normal speed (1x speed). The playback timing can be matched.

Also, the processor 130 may maintain the playback speed of the second content while decreasing the playback speed of the first content. For example, assuming that the playback timing difference between the first content and the second content is 1 second, the processor 130 plays the first content at 0.5x speed for 2 seconds and the second content at the normal speed (1x speed) for 2 seconds. It can be played back to match the playback timing.

Meanwhile, the processor 130 may control the display 120 to display a user interface (UI) including at least one device that provides content corresponding to the first content, and the second device 300 through the UI. When a user input for selecting ) is received, the communication interface 110 may be controlled to transmit a control signal requesting the second characteristic information and the second content to the second device 300 .

Here, when a first user input for selecting a first content from among a plurality of content is received, the processor 130 may request the first content corresponding to the first user input from the first device 200 . Here, the first user input may be a user input for changing a specific channel number or requesting specific content.

Also, when a second user input requesting content (sub content) related to the first content (main content) is received, the processor 130 may identify at least one sub content corresponding to the first content. In addition, the processor 130 may display a UI including information corresponding to the plurality of identified sub-contents. Here, the UI may be the UI 802 of FIG. 8 . Here, the second user input may mean a user input for additionally requesting a separate sub content in addition to the first content.

In addition, when a third user input for selecting one sub-content (second content) from among the plurality of sub-contents displayed on the UI 802 is received, the processor 130 generates the second content and the second content corresponding to the third user input. 2 It is possible to generate a control signal requesting feature information. In addition, the processor 130 may transmit the generated control signal to the second device 300 . Here, the third user input may mean a user input of one sub content among the plurality of sub content.

Meanwhile, the processor 130 may identify whether the reproduction time points of the first content and the second content coincide with each other for each threshold time. This is because, depending on the situation, a delay may occur in the middle. Accordingly, the processor 130 may compare the first characteristic information and the second characteristic information at every threshold time to determine whether the reproduction times of the first content and the second content are synchronized, and perform a synchronization operation.

The electronic device 100 according to various embodiments of the present disclosure may display the first content (main content) and the second content (sub content) at the same time while synchronizing the playback time. The viewer may not feel the inconvenience of inconsistency in playback timing while simultaneously watching sub content in addition to the main content.

Meanwhile, although only a simple configuration constituting the electronic device 100 has been illustrated and described above, various configurations may be additionally provided in implementation. This will be described below with reference to FIG. 5 .

FIG. 5 is a block diagram illustrating a detailed configuration of the electronic device of FIG. 4 .

Referring to FIG. 5 , the electronic device 100 includes a communication interface 110 , a display 120 , a processor 130 , a memory 140 , an operation interface 150 , an input/output interface 160 , a microphone 170 and The speaker 180 may be configured.

Meanwhile, redundant descriptions of the same operations as described above among the operations of the communication interface 110 , the display 120 , and the processor 130 will be omitted.

The memory 140 is implemented as an internal memory such as a ROM (eg, electrically erasable programmable read-only memory (EEPROM)) included in the processor 130 , a RAM, or the like, or with the processor 130 . It may be implemented as a separate memory. In this case, the memory 140 may be implemented in the form of a memory embedded in the electronic device 100 or may be implemented in the form of a memory detachable from the electronic device 100 according to the purpose of data storage. For example, data for driving the electronic device 100 is stored in a memory embedded in the electronic device 100 , and data for an extended function of the electronic device 100 is detachable from the electronic device 100 . It can be stored in any available memory.

Meanwhile, in the case of a memory embedded in the electronic device 100 , a volatile memory (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (non-volatile memory) ( Examples: one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (such as NAND flash or NOR flash, etc.) ), a hard drive, or a solid state drive (SSD), and in the case of a removable memory in the electronic device 100, a memory card (eg, a compact flash (CF), SD ( secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to the USB port (e.g., USB memory) and the like.

The manipulation interface 150 may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or may be implemented as a touch screen capable of performing the above-described display function and manipulation input function together. Here, the button may be various types of buttons such as a mechanical button, a touch pad, a wheel, etc. formed in an arbitrary area such as the front, side, or rear of the exterior of the main body of the electronic device 100 .

Input/output interface 160 is HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt (Thunderbolt), VGA (Video Graphics Array) port, The interface may be any one of an RGB port, a D-subminiature (D-SUB), and a digital visual interface (DVI).

The input/output interface 160 may input/output at least one of audio and video signals.

Depending on the implementation, the input/output interface 160 may include a port for inputting and outputting only an audio signal and a port for inputting and outputting only a video signal as separate ports, or may be implemented as a single port for inputting and outputting both an audio signal and a video signal.

The electronic device 100 may further include a microphone 170 . A microphone is a component for receiving a user's voice or other sound and converting it into audio data.

The microphone 170 may receive a user's voice in an activated state. For example, the microphone 170 may be integrally formed in the upper side, the front direction, the side direction, or the like of the electronic device 100 . The microphone 170 includes a microphone for collecting analog user voice, an amplifier circuit for amplifying the collected user voice, an A/D conversion circuit for sampling the amplified user voice and converting it into a digital signal, and a noise component from the converted digital signal. It may include various configurations such as a filter circuit that removes the

The electronic device 100 may include a speaker 180 . The speaker 180 may be a component that outputs various types of audio data processed in the input/output interface, as well as various notification sounds or voice messages.

6 is a flowchart illustrating a content display operation of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 6 , the electronic device 100 may display the first content (S605). Here, the first content may mean content received from the first device 200 .

Also, the electronic device 100 may match the first content and the second content ( S610 ). When the first content is received, the electronic device 100 may identify a plurality of content related to the first content. In addition, the electronic device 100 may display a UI including a plurality of identified contents. Then, when a user input for selecting one content (second content) from among a plurality of content is received through the displayed UI, the electronic device 100 displays one content (second content) and the first content corresponding to the user input. can match. Here, the matching operation may refer to an operation in which the user directly determines the content to be displayed together with the first content.

For example, when the user selects a soccer game (first content), the electronic device 100 may identify a plurality of personal broadcast content related to the soccer game (first content). In addition, the electronic device 100 may identify one content (second content) corresponding to a user input among a plurality of personal broadcast content.

Also, the electronic device 100 may synchronize the playback timings of the first content and the second content ( S615 ). Here, since the electronic device 100 receives the first content from the first device 200 and receives the second content from the second device 300 , the sync between the first content and the second content may not match. Accordingly, the electronic device 100 may perform a synchronization operation in order to synchronize.

Also, when the synchronization operation is completed, the electronic device 100 may display the first content and the second content (S620). Here, the electronic device 100 may display the first content on the first area of the display 120 and display the second content on the second area of the display 120 .

7 is a flowchart illustrating a content matching operation of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 7 , step S610 of FIG. 6 includes displaying a plurality of contents matching the first content ( S705 ), receiving a user input for selecting one content from among the plurality of contents ( S710 ), and It may include matching the second content corresponding to the user input with the first content (S715).

FIG. 8 is a diagram for explaining a UI displayed in the matching operation of FIG. 7 .

Referring to FIG. 8 , the electronic device 100 simultaneously displays an image 801 corresponding to the first content and a UI 802 including a plurality of content matching the first content on the display 120 in step S705 . can do. Here, the image 801 may be displayed on the first area of the display 120 , and the UI 802 may be displayed on the second area of the display 120 .

9 is a flowchart illustrating an operation of synchronizing first content and second content of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 9 , the electronic device 100 may obtain first characteristic information and second characteristic information ( S905 ). Also, the electronic device 100 may identify whether the first characteristic information and the second characteristic information match ( S910 ).

Here, when the first characteristic information and the second characteristic information match (S910-Y), the electronic device 100 may display the first content and the second content (S930).

On the other hand, if the first characteristic information and the second characteristic information do not match (S910-N), the electronic device 100 may identify whether there is a content whose playback time can be changed among the first content and the second content. There is (S915). Here, changing the playback timing may mean changing the timing of the currently played frame. For example, changing the playback time may mean changing from 2 seconds to 4 seconds in the timeline of the content. When the playback time is changed, it may mean that the displayed image and output audio are different. The playback timing of recorded broadcast content or OTT-related content may be changeable, but the playback timing of live content may not be changeable. Also, as an exception, even for live content, the device providing the content may separately provide a function of changing the playback time.

Here, if at least one of the first content and the second content, which can change the playback time, exists (S915-Y), the electronic device 100 may change the playback setting of the content whose playback time can be changed (S920). Then, the electronic device 100 may display the first content and the second content (S930). A detailed operation of step S920 will be described later with reference to FIG. 10 .

Meanwhile, if there is no content capable of changing the playback time among the first content and the second content (S915-N), the electronic device 100 may change the playback setting of the content having a faster playback time (S925). Then, the electronic device 100 may display the first content and the second content (S930). A detailed operation of step S925 will be described later with reference to FIGS. 11 and 12 .

10 is a flowchart illustrating an operation of changing a content reproduction setting of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 10 , if the first characteristic information and the second characteristic information do not match ( S910 -N in FIG. 9 ), the electronic device 100 may identify whether the first playback time of the first content can be changed. (S1005). Here, if it is identified that the first playback time of the first content can be changed (S1005-Y), the electronic device 100 may change the first playback time of the first content (S1010). Then, the electronic device 100 may display the first content and the second content (S1015).

Meanwhile, if it is identified that the first playback time of the first content cannot be changed (S1005-N), the electronic device 100 may identify whether the second playback time of the second content can be changed (S1020) . Here, if it is identified that the second reproduction time of the second content can be changed (S1020-Y), the electronic device 100 may change the second reproduction time of the second content (S1025). Then, the electronic device 100 may display the first content and the second content (S1015).

On the other hand, if it is identified that the second playback time of the second content cannot be changed (S1020-N), the electronic device 100 may change the playback setting of the content having an earlier playback time (S1030). Then, the electronic device 100 may display the first content and the second content (S1015). Here, step S1030 may correspond to step S925 of FIG. 9 .

11 is a flowchart illustrating an operation of changing a content reproduction setting of an electronic device according to another embodiment of the present disclosure.

Referring to FIG. 11 , if the content that can change the playback time is not identified ( S915-N in FIG. 9 ), the electronic device 100 determines that the first playback time of the first content is higher than the second playback time of the second content. It can be identified whether it is fast (S1105).

Here, if it is identified that the first reproduction time of the first content is earlier than the second reproduction time of the second content (S1105-Y), the electronic device 100 may store the first content in the memory 140 (S1110) . Then, the electronic device 100 may pause the reproduction of the first content and continue to display the second content (S1115). Then, the electronic device 100 may identify whether the first characteristic information and the second characteristic information match (S1120). Here, if the first characteristic information and the second characteristic information match (S1120-Y), the electronic device 100 may display the first content and the second content (S1125). In addition, the electronic device 100 may delete content (first content) stored in the memory 140 . Here, if the first characteristic information and the second characteristic information do not match (S1120-N), the electronic device 100 repeatedly stops the reproduction of the first content and continuously displays the second content ( S1115 ). can do.

Meanwhile, while the first content is paused in step S1115 , the first content may be continuously received from the first device 200 . If the first characteristic information and the second characteristic information match, the electronic device 100 may identify that the sync of the first content and the second content match. In addition, the electronic device 100 may immediately display the first content and the second content as soon as it is received. However, if the first characteristic information and the second characteristic information do not match, the electronic device 100 may identify that the syncs of the first content and the second content do not match. In addition, the electronic device 100 may pause the reproduction of the first content determined to have a fast reproduction time in order to synchronize (to match). However, since the first content is continuously being received here, the electronic device 100 may store the first content in the nonvolatile area of the memory 140 . Here, the electronic device 100 may pause the reproduction of the first content until the first characteristic information and the second characteristic information match.

Meanwhile, if it is identified that the first reproduction time of the first content is slower than the second reproduction time of the second content (S1105-N), the second content may be stored in the memory 140 (S1130). Then, the electronic device 100 may pause the reproduction of the second content and continue to display the first content ( S1135 ). Then, the electronic device 100 may identify whether the first characteristic information and the second characteristic information match ( S1140 ). Here, when the first characteristic information and the second characteristic information match (S1140-Y), the electronic device 100 may display the second content and the first content (S1125). In addition, the electronic device 100 may delete content (second content) stored in the memory 140 . Here, if the first characteristic information and the second characteristic information do not match ( S1140 -N ), the electronic device 100 repeatedly performs step S1135 of pausing the reproduction of the second content and continuously displaying the first content. can do.

Meanwhile, while the second content is paused in step S1135 , the second content may be continuously received from the second device 300 . If the first characteristic information and the second characteristic information match, the electronic device 100 may identify that the sync of the first content and the second content match. In addition, the electronic device 100 may immediately display the first content and the second content as soon as it is received. However, if the first characteristic information and the second characteristic information do not match, the electronic device 100 may identify that the syncs of the first content and the second content do not match. In addition, the electronic device 100 may pause the reproduction of the second content determined that the reproduction time is fast in order to synchronize (to match). However, since the second content is continuously being received here, the electronic device 100 may store the second content in the nonvolatile area of the memory 140 . Here, the electronic device 100 may pause the reproduction of the second content until the first characteristic information and the second characteristic information match.

12 is a flowchart illustrating an operation of changing a content reproduction setting of an electronic device according to another embodiment of the present disclosure.

12 , steps S1205, S1210, S1220, S1225, S1226, S1230, and S1240 may correspond to steps S1105, S1110, S1120, S1125, S1126, S1130, and S1140 of FIG. 11 . Therefore, redundant description is omitted.

Meanwhile, step S1215 of FIG. 12 may be performed instead of step S1115 of FIG. 11 , and step S1235 of FIG. 12 may be performed instead of step S1135 of FIG. 11 .

Specifically, after step S1210 , the electronic device 100 may display the first content by reducing the playback speed and maintain the playback speed of the second content to be displayed. For example, the electronic device 100 may reproduce the first content at 0.8x speed and reproduce the second content at 1x speed, which is the existing double speed.

Also, after step S1230 , the electronic device 100 may display the second content by reducing the playback speed, and maintain the playback speed of the first content to be displayed. For example, the electronic device 100 may reproduce the second content at 0.8x speed and reproduce the first content at 1x speed, which is the existing double speed.

Unlike FIG. 11 , the electronic device 100 of the embodiment of FIG. 12 may decrease the playback speed without pausing the content. Therefore, from the user's point of view, the operation of synchronizing the sync may feel natural.

13 is a diagram for explaining an operation of changing a content reproduction setting of an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 13 , a table 1305 may include playback setting operations of the electronic device 100 according to four embodiments. According to the first embodiment, it is possible to change the playback time of the first content and change the playback timing of the second content. Here, the electronic device 100 may change the reproduction setting of at least one of the first content and the second content. Here, the operation of changing the reproduction setting may mean a pause in reproduction, an increase in the reproduction speed, or a decrease in the reproduction speed.

Also, according to the second embodiment, it is possible to change the playback time of the first content, but it may not be possible to change the playback time of the second content. Here, the electronic device 100 may change the reproduction setting of the first content that can change the reproduction time.

Also, according to the third exemplary embodiment, it may be impossible to change the playback time of the first content, and it may be possible to change the playback timing of the second content. Here, the electronic device 100 may change the reproduction setting of the second content that can change the reproduction time.

Also, according to the fourth embodiment, it may be impossible to change the playback time of the first content and it may not be possible to change the playback timing of the second content. Here, the electronic device 100 may store the content having a faster playback time in the memory 140 and then change the playback setting of the stored content.

14 is a diagram for explaining an embodiment in which a playback time of a first content is earlier than a playback time of a second content.

Referring to FIG. 14 , according to an embodiment 1405 , a first playback time of the first content may be earlier than a second playback time of the second content. This is because a delay may occur in generation and transmission of the second content.

It is assumed that the first reproduction time of the first content is earlier than the second reproduction time of the second content by 1 second.

Here, the electronic device 100 may simultaneously display the first image 201-1 of the first content and the loading image 1410 of the second content. After one second has elapsed, the electronic device 100 may simultaneously display the second image 201 - 2 of the first content and the fourth image 202-1 of the second content. After one second has elapsed, the electronic device 100 may simultaneously display the third image 201-3 of the first content and the fifth image 202-2 of the second content.

Since the electronic device 100 reproduces the first reproduction time of the first content faster than the second reproduction time of the second content by one second, the user may feel that the synchronization is out of sync by an interval corresponding to one second.

15 is a diagram for explaining an embodiment in which a playback time of a first content is slower than a playback time of a second content.

Referring to FIG. 15 , according to another embodiment 1505 , the second playback time of the second content may be earlier than the first playback time of the first content. This is because a delay may occur in the transmission of the first content.

It is assumed that the second reproduction time of the second content is earlier than the first reproduction time of the first content by 1 second.

Here, the electronic device 100 may simultaneously display the first image 201-1 of the first content and the fifth image 202-2 of the second content. After one second has elapsed, the electronic device 100 may simultaneously display the second image 201 - 2 of the first content and the sixth image 202 - 3 of the second content. After one second has elapsed, the electronic device 100 may simultaneously display the third image 201-3 of the first content and the seventh image 202-4 of the second content.

Since the electronic device 100 reproduces the second reproduction time of the second content faster by 1 second than the first reproduction time of the first content, the user may feel that the synchronization is out of sync by an interval corresponding to one second.

16 is a system flow diagram for explaining contents synchronization operations of an electronic device, a first device, and a second device.

Referring to FIG. 16 , the first device 200 may store the first content ( S1605 ). Then, the first device 200 may transmit the first content to the second device 300 (S1610). Then, the first device 200 may transmit the first content to the electronic device 100 (S1615).

Here, the electronic device 100 may obtain first characteristic information corresponding to the first content (S1620).

Meanwhile, after receiving the first content, the second device 300 may generate second content related to the first content ( S1625 ). In addition, the second device 300 may acquire second characteristic information corresponding to the first content based on the first content received from the first device 200 ( S1630 ). Then, the second device 300 may transmit the second characteristic information and the second content to the electronic device 100 (S1635). Here, the reason why the second device 300 transmits the second characteristic information corresponding to the first content instead of the first content may be due to a copyright problem. The second device 300 may receive the first content from the first device 200 , but may not have the right to distribute the first content to others without permission. Accordingly, the second device 300 cannot transmit the first content to the electronic device 100 . However, for the synchronization operation, the electronic device 100 must determine how the second content is synchronized with the first content. Accordingly, the electronic device 100 may receive the second characteristic information corresponding to the first content instead of the first content from the second device 300 .

The electronic device 100 may synchronize the playback timing of the first content and the second content based on the first characteristic information acquired in step S1620 and the second characteristic information acquired in step S1635 ( S1640 ). Then, the electronic device 100 may display the first content and the second content (S1645).

17 is a flowchart illustrating an operation of acquiring second characteristic information of a second device.

Referring to FIG. 17 , the second device 300 may receive the first content from the first device 200 ( S1705 ). Then, the second device 300 may generate second content related to the first content (S1710). Then, the second device 300 may acquire second characteristic information corresponding to the first content based on the first content received from the first device 200 ( S1715 ). Then, the second device 300 may transmit the second content and the second characteristic information to the electronic device 100 ( S1720 ).

According to an embodiment, the second device 300 may directly receive the first content and acquire second characteristic information corresponding to the first content. For example, the second device 300 may receive the first content from the first device 200 through the communication interface. Here, the first characteristic information may include at least one of a broadcast type, a viewing channel, and a content time stamp.

According to another embodiment, the second device 300 may acquire the second characteristic information corresponding to the first content without receiving the first content. For example, the second device 300 may acquire information about the first content being reproduced through a camera or a microphone. Specifically, the second device 300 may acquire audio data of the first content being reproduced in a separate device through the microphone of the second device 300 . Also, the second device 300 may acquire image data of the first content being reproduced in a separate device through the camera of the second device 300 . Accordingly, the second device 300 may acquire the second characteristic information corresponding to the first content without directly receiving the first content.

18 is a system flowchart illustrating an operation of acquiring second characteristic information of a second device according to an embodiment of the present disclosure.

Referring to FIG. 18 , the first device 200 may store the first content (S1805). Then, the first device 200 may transmit the first content to the second device 300 (S1810). Then, the first device 200 may transmit the first content to the electronic device 100 (S1815).

Here, the electronic device 100 may acquire the first image data of the first content as the first characteristic information based on the first content received from the first device 200 ( S1820 ).

Meanwhile, the second device 300 may acquire second image data of the first content based on the first content received from the first device 200 ( S1825 ). Then, the second device 300 may generate second content related to the first content (S1830). Then, the second device 300 may obtain the second characteristic information by matching the second image data of the first content and the second content (S1835). Then, the second device 300 may transmit the second characteristic information and the second content to the electronic device 100 ( S1840 ).

The electronic device 100 may synchronize the playback timing of the first content and the second content based on the first characteristic information acquired in step S1820 and the second characteristic information acquired in step S1840 ( S1845 ). Then, the electronic device 100 may display the first content and the second content (S1850).

19 is a diagram for explaining an operation of identifying a reproduction time point by comparing first characteristic information and second characteristic information, according to an embodiment of the present disclosure.

Referring to FIG. 19 , a table 1905 may include first characteristic information obtained from the electronic device 100 and second characteristic information received from the second device 300 . Here, the first characteristic information and the second characteristic information may include values corresponding to image data. Here, the value corresponding to the image data may mean a pixel value. It is assumed that pixel values of the frame unit (eg, 1 second) of the first content included in the first feature information are #223344, #334455, #445566, #556677, and #667788. Also, it is assumed that pixel values of the frame unit (eg, 1 second) of the first content included in the second characteristic information are #112233, #223344, #334455, #445566, and #556677. Here, the second characteristic information is information obtained based on the first content, but may be information representing a reproduction time of the second content. The second characteristic information is information obtained by the second device 300 that generates the second content, and the second characteristic information may include information related to a reproduction time of the second content.

Specifically, the first characteristic information may include first data (pixel values) corresponding to the first content. In addition, the second characteristic information may include second data (pixel values) corresponding to the first content and frame information of the second content corresponding to the second data. The second device 300 may obtain the second characteristic information by matching each frame of the second content with a pixel value of the first content.

For example, it is assumed that the second data are pixel values of image data and are #112233, #223344, #334455, #445566, and #556677. Here, the second device 300 generates frame #01 of the second content while the frame of the first content corresponding to #112233 is provided, and while the frame of the first content corresponding to #223344 is provided, the second device 300 is provided. Frame #02 of the content is generated, frame #03 of the second content is generated while the frame of the first content corresponding to #334455 is provided, and frame #03 of the second content is generated while the frame of the first content corresponding to #445566 is provided. Frame #04 of the content may be generated, and frame #05 of the second content may be generated while the frame of the first content corresponding to #556677 is provided.

Although both the first characteristic information and the second characteristic information obtained by the electronic device 100 relate to the first content, the subject from which the characteristic information is extracted may be different. Even if the subject is different, the same feature information may be extracted when the same extraction method is used. However, some syncs may not match due to the occurrence of a delay.

The electronic device 100 may synchronize by comparing the first characteristic information and the second characteristic information. Referring to the table 1905 , the electronic device 100 may obtain a pixel value change pattern based on the first characteristic information and the second characteristic information, and first content based on the acquired pixel value change pattern. and a difference in playback timing of the second content may be identified. The electronic device 100 may identify that the first content is reproduced 1 second faster than the second content based on the table 1905 .

20 is a system flowchart for explaining an operation of acquiring second characteristic information of a second device according to another embodiment of the present disclosure.

Referring to FIG. 20 , the first device 200 may store the first content (S2005). Then, the first device 200 may transmit the first content to the second device 300 (S2010). Then, the first device 200 may transmit the first content to the electronic device 100 (S2015).

Here, the electronic device 100 may acquire the first audio data of the first content as the first characteristic information based on the first content received from the first device 200 ( S2020 ).

Meanwhile, the second device 300 may acquire second audio data of the first content based on the first content received from the first device 200 ( S2025 ). Then, the second device 300 may generate second content related to the first content (S2030). Then, the second device 300 may obtain the second characteristic information by matching the second audio data of the first content and the second content ( S2035 ). Then, the second device 300 may transmit the second characteristic information and the second content to the electronic device 100 (S2040).

The electronic device 100 may synchronize the playback timing of the first content and the second content based on the first characteristic information acquired in step S2020 and the second characteristic information acquired in step S2040 ( S2045 ). Then, the electronic device 100 may display the first content and the second content (S2050).

21 is a diagram for explaining an operation of identifying a reproduction time point by comparing first characteristic information and second characteristic information, according to another embodiment of the present disclosure.

Referring to FIG. 21 , a table 2105 may include first characteristic information obtained from the electronic device 100 and second characteristic information received from the second device 300 . Here, the first characteristic information and the second characteristic information may include values corresponding to audio data. Here, the value corresponding to the audio data may mean a sound level value. It is assumed that the sound level values of the frame unit (eg, 1 second) of the first content included in the first characteristic information are 20 dB, 30 dB, 40 dB, 50 dB, and 60 dB. In addition, it is assumed that the sound level values of the frame unit (eg, 1 second) of the first content included in the second characteristic information are 10 dB, 20 dB, 30 dB, 40 dB, and 50 dB. Here, the second characteristic information is information obtained based on the first content, but may be information representing a reproduction time of the second content. The second characteristic information is information obtained by the second device 300 that generates the second content, and the second characteristic information may include information related to a reproduction time of the second content.

Specifically, the first characteristic information may include first data (a sound volume value) corresponding to the first content. In addition, the second characteristic information may include second data (a loudness value) corresponding to the first content and frame information of the second content corresponding to the second data. The second device 300 may obtain the second characteristic information by matching each frame of the second content with the loudness value of the first content.

For example, it is assumed that the second data is a sound level value of image data and is 10dB, 20dB, 30dB, 40dB, or 50dB. Here, the second device 300 generates frame #01 of the second content while the frame of the first content corresponding to 10 dB is provided, and generates the frame #01 of the second content while the frame of the first content corresponding to 20 dB is provided. Frame #02 is generated, frame #03 of the second content is generated while the frame of the first content corresponding to 30dB is provided, and frame #04 of the second content is provided while the frame of the first content corresponding to 40dB is provided. A frame may be generated, and frame #05 of the second content may be generated while the frame of the first content corresponding to 50 dB is provided.

Although both the first characteristic information and the second characteristic information obtained by the electronic device 100 relate to the first content, the subject from which the characteristic information is extracted may be different. Even if the subject is different, the same feature information may be extracted when the same extraction method is used. However, some syncs may not match due to the occurrence of a delay.

The electronic device 100 may synchronize by comparing the first characteristic information and the second characteristic information. Referring to table 2105 , the electronic device 100 may obtain a change pattern of a sound level value based on the first characteristic information and the second characteristic information, and based on the obtained change pattern of the sound level value, the electronic device 100 may It is possible to identify a difference between the playback timings of the first content and the second content. The electronic device 100 may identify that the first content is reproduced 1 second faster than the second content based on the table 2105 .

22 is a diagram for explaining an operation of identifying a reproduction time point by comparing first characteristic information and second characteristic information, according to another embodiment of the present disclosure;

Referring to FIG. 22 , a graph 2201 may be a sound waveform of the first content included in the first characteristic information. The graph 2202 may be a sound waveform of the first content included in the second characteristic information.

The electronic device 100 may identify a difference in reproduction timing between the first content and the second content based on the first feature information and the second feature information. Based on the graph 2201 and the graph 2202, the overall waveform may identify that the graph 2201 is fast. Accordingly, the electronic device 100 may identify that the first content is being reproduced 1 second earlier than the second content.

23 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 23 , in the method of controlling the electronic device 100 , when the first content is received from the first device 200 , obtaining first characteristic information corresponding to the first content ( S2305 ), the second device When the second characteristic information and the second content corresponding to the first content are received from the 300 , the step of synchronizing the reproduction timing of the first content and the second content based on the first characteristic information and the second characteristic information ( S2310 ) ) and displaying the synchronized first content and the synchronized second content (S2315).

Meanwhile, the control method may further include displaying the first content on the first area of the display 120 of the electronic device 100 and displaying the second content on the second area of the display 120 , , the step of synchronizing the playback time (S2310) is to synchronize the playback timing of the first content and the second content based on the first characteristic information corresponding to the displayed first content and the second characteristic information corresponding to the displayed second content. can

Meanwhile, the first characteristic information may include at least one of first image data of the first content obtained by the electronic device 100 or first audio data of the first content, and the second characteristic information may include the second device ( 300) may include at least one of the second image data of the first content and the second audio data of the first content.

Meanwhile, in the step of synchronizing the playback timing ( S2310 ), when the first characteristic information and the second characteristic information do not match, the playback timing of the first content and the second content may be synchronized.

Meanwhile, in the step of synchronizing the playback time ( S2310 ), if the first characteristic information and the second characteristic information do not match, the playback setting of one of the first content and the second content may be changed.

Meanwhile, in the step of synchronizing the playback time ( S2310 ), the playback setting of one of the first content or the second content may be changed by pausing the playback of the content, decreasing the playback speed of the content, or increasing the playback speed of the content. have.

Meanwhile, in the step of synchronizing the playback timing ( S2310 ), it is possible to identify a content having a faster playback timing from among the first content and the second content based on the first characteristic information and the second characteristic information, and to reproduce the identified content. You can change the settings.

On the other hand, in the step of synchronizing the playback time (S2310), the identified content may be stored in the memory 140, and the playback setting of the identified content may be set by pausing the playback of the identified content or reducing the playback speed of the identified content. can be changed

Meanwhile, in the step of synchronizing the playback time ( S2310 ), the playback speed of the identified content and other content may be maintained.

On the other hand, when a user input for selecting the second device 300 is received through the step of displaying a UI (User Interface) including at least one device providing content corresponding to the first content, and the second device 300 is received, the second The method may further include transmitting a control signal requesting the feature information and the second content to the second device 300 .

Meanwhile, the method of controlling the electronic device shown in FIG. 23 may be executed on the electronic device having the configuration of FIG. 4 or FIG. 5 , and may also be executed on the electronic device having other configurations.

Meanwhile, the above-described methods according to various embodiments of the present disclosure may be implemented in the form of an application that can be installed in an existing electronic device.

In addition, the above-described methods according to various embodiments of the present disclosure may be implemented only by software upgrade or hardware upgrade of an existing electronic device.

In addition, various embodiments of the present disclosure described above may be performed through an embedded server provided in an electronic device, or an external server of at least one of an electronic device and a display device.

Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage media readable by a machine (eg, a computer). can The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include the electronic device according to the disclosed embodiments. When the instruction is executed by the processor, the processor may perform a function corresponding to the instruction by using other components directly or under the control of the processor. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

In addition, according to an embodiment of the present disclosure, the method according to the various embodiments described above may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

In addition, each of the components (eg, modules or programs) according to the above-described various embodiments may be composed of a single or a plurality of entities, and some sub-components of the above-described corresponding sub-components may be omitted, or other sub-components may be omitted. Components may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, repetitively or heuristically executed, or at least some operations may be executed in a different order, omitted, or other operations may be added. can

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and it is common in the technical field pertaining to the present disclosure without departing from the gist of the present disclosure as claimed in the claims. Various modifications may be made by those having the knowledge of

100: electronic device
110: communication interface
120: display
130: processor

Claims (20)

In an electronic device,
communication interface;
display; and
When the first content is received from the first device through the communication interface, acquiring first characteristic information corresponding to the first content,
When second characteristic information and second content corresponding to the first content are received from a second device through the communication interface, the first content and the second content are based on the first characteristic information and the second characteristic information Synchronize the playback time of content,
The electronic device including a; According to claim 1,
The processor is
controlling the display to display the first content on a first area of the display;
controlling the display to display the second content on a second area of the display;
An electronic device for synchronizing the reproduction timing of the first content and the second content based on first characteristic information corresponding to the displayed first content and second characteristic information corresponding to the displayed second content. According to claim 1,
The first characteristic information is
and at least one of first image data of the first content acquired by the electronic device and first audio data of the first content,
The second characteristic information,
The electronic device comprising at least one of second image data of the first content and second audio data of the first content acquired by the second device. According to claim 1,
The processor is
When the first characteristic information and the second characteristic information do not match, the electronic device synchronizes playback times of the first content and the second content. 3. The method of claim 2,
The processor is
If the first characteristic information and the second characteristic information do not match, change a reproduction setting of one of the first content and the second content. 6. The method of claim 5,
The processor is
An electronic device for changing a playback setting of one of the first content and the second content by pausing the playback of the content, decreasing the playback speed of the content, or increasing the playback speed of the content. 6. The method of claim 5,
The processor is
Identifying a content having a faster playback time from among the first content and the second content based on the first characteristic information and the second characteristic information,
An electronic device for changing a reproduction setting of the identified content. 8. The method of claim 7,
memory; further comprising
The processor is
storing the identified content in the memory;
and changing a playback setting of the identified content by pausing the playback of the identified content or decreasing a playback speed of the identified content. 9. The method of claim 8,
The processor is
An electronic device that maintains a playback speed of content different from the identified content. According to claim 1,
The processor is
controlling the display to display a UI (User Interface) including at least one device that provides content corresponding to the first content;
When a user input for selecting the second device is received through the UI, the electronic device controls the communication interface to transmit a control signal for requesting the second characteristic information and the second content to the second device. . A method for controlling an electronic device, comprising:
when the first content is received from the first device, obtaining first characteristic information corresponding to the first content;
When the second characteristic information and the second content corresponding to the first content are received from the second device, the playback time of the first content and the second content is determined based on the first characteristic information and the second characteristic information. synchronizing; and
Displaying the first content and the second content in which the playback time is synchronized; including, a control method. 12. The method of claim 11,
displaying the first content on a first area of a display of the electronic device;
displaying the second content on a second area of the display; further comprising,
The step of synchronizing the playback time includes:
A control method for synchronizing reproduction times of the first content and the second content based on first characteristic information corresponding to the displayed first content and second characteristic information corresponding to the displayed second content. 12. The method of claim 11,
The first characteristic information is
and at least one of first image data of the first content acquired by the electronic device and first audio data of the first content,
The second characteristic information,
A control method comprising at least one of second image data of the first content and second audio data of the first content obtained by the second device. 12. The method of claim 11,
The step of synchronizing the playback time includes:
When the first characteristic information and the second characteristic information do not match, the playback time of the first content and the second content is synchronized. 13. The method of claim 12,
The step of synchronizing the playback time includes:
If the first characteristic information and the second characteristic information do not match, changing a reproduction setting of one of the first content and the second content. 16. The method of claim 15,
The step of synchronizing the playback time includes:
A control method for changing a playback setting of one of the first content and the second content by pausing the playback of the content, decreasing the playback speed of the content, or increasing the playback speed of the content. 16. The method of claim 15,
The step of synchronizing the playback time includes:
Identifying a content having a faster playback time from among the first content and the second content based on the first characteristic information and the second characteristic information,
A control method for changing a reproduction setting of the identified content. 18. The method of claim 17,
The step of synchronizing the playback time includes:
storing the identified content in a memory;
The control method of changing the playback setting of the identified content by pausing the playback of the identified content or reducing the playback speed of the identified content. 19. The method of claim 18,
The step of synchronizing the playback time includes:
A control method for maintaining a playback speed of content different from the identified content. 12. The method of claim 11,
displaying a user interface (UI) including at least one device providing content corresponding to the first content; and
When a user input for selecting the second device is received through the UI, transmitting a control signal for requesting the second characteristic information and the second content to the second device; .

Images