KR102012580B1 - Apparatus and method for multimedia contents handling - Google Patents

Abstract

The present invention relates to an apparatus for processing multimedia content and a method of controlling the same. In order to achieve the above object, the control method of the multimedia content processing device according to the present invention includes the integrated multimedia content including at least one sub-content from a content providing server. Receiving; Determining the number of sub-contents included in the integrated multimedia content by using the information included in the header of the received integrated multimedia content; Extracting each sub content from the integrated multimedia content by the determined number of sub contents; And synchronizing each sub content using time information included in each extracted sub content, and controlling the synchronized sub content to be played together.

Description

Apparatus for controlling multimedia content and its control method {APPARATUS AND METHOD FOR MULTIMEDIA CONTENTS HANDLING}

The present invention relates to an apparatus for processing multimedia content and a method of controlling the same, and more particularly, to an apparatus for receiving and displaying multimedia content through a specific channel according to a user selection and a method of controlling the same.

Recently, various types of broadcast contents services are provided, not limited to analog broadcasting.

However, in the related art, as a broadcast station transmits only an image photographed by one camera, a viewer cannot simultaneously view images photographed at various angles or at various positions.

In particular, in the case of a sports in which multiple players participate, such a service is not provided in the related art even though there are demands of viewers to check and view the appearance of players at various positions.

For example, in a baseball game, some viewers would like to see the pitcher pitching and the first or third runner at the same time. Only the pitcher and the runner can simultaneously show the pitcher and the runner at the same time. In this case, the camera angle or the angle of view satisfies the above conditions. There is a problem, which is still insufficient to meet the needs of viewers.

In order to solve this problem, although a broadcast station transmits images captured by a plurality of cameras through separate broadcast channels, the terminal may simultaneously display respective broadcast contents received through the respective broadcast channels on the screen. Occupying multiple broadcast channels for one broadcast is a difficult task in reality.

In addition, time synchronization between broadcast contents received through each individual broadcasting channel should be minimized to minimize inconvenience or confusion for viewers. Due to various reasons, the synchronization state of each broadcast content may be out of sync when received by the actual terminal.

Patent Registration No. 10-0974638

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and its object is to receive multimedia content through one channel and simultaneously display images taken at various angles or images taken at various angles for viewers to view. The present invention provides an apparatus for processing multimedia content and a control method thereof.

In order to achieve the above object, the multimedia content processing device according to the present invention comprises: a receiving unit for receiving integrated multimedia content including at least one sub content from a content providing server through one content receiving channel; A determination unit which determines the number of sub contents included in the integrated multimedia content by using information included in the header of the integrated multimedia content received by the receiving unit; An extraction unit for extracting each sub content from the integrated multimedia content by the number of sub contents determined by the determination unit; And a controller for synchronizing each sub content using time information included in each sub content extracted by the extractor, and controlling the synchronized sub content to be played together.

In addition, to achieve the above object, a control method of a multimedia content processing apparatus according to the present invention comprises the steps of: receiving integrated multimedia content including at least one sub-content from a content providing server; Determining the number of sub-contents included in the integrated multimedia content by using the information included in the header of the received integrated multimedia content; Extracting each sub content from the integrated multimedia content by the determined number of sub contents; Synchronizing each sub content using time information included in each extracted sub content, and controlling the synchronized sub content to be played together.

As described above, according to the present invention, a camera image photographing different subjects or a camera image photographing at different angles can be simultaneously viewed at the same photographing time point.

In particular, since a plurality of camera images photographing different subjects are received through one broadcasting channel (content receiving channel), the time taken for the playback time of the plurality of camera images is compared when compared with the case of receiving them through different channels. It can be shortened.

For example, when each camera image is received through a different communication channel, since each camera image is divided into packets and received through different paths, a considerable time delay may occur, but through the same communication channel as in the present invention. Receiving a plurality of camera image packets can significantly solve the time delay problem.

In particular, the plurality of sub-contents received through the corresponding broadcast channel each include time information for synchronization, and the time-synchronization of each sub-content is performed by the receiving terminal (ie, the multimedia content processing apparatus) using the time information for synchronization. As a result, each camera image packet is received through a different path, thereby solving the problem of synchronization failure.

In the processing of a plurality of sub-contents, by using a decoding module or a decoding thread, a plurality of sub-contents can be simultaneously viewed through one broadcast channel even in relatively low hardware.

1 is a schematic structural diagram of an entire system including a multimedia content processing device according to an embodiment of the present invention;
FIG. 2 is a functional block diagram of FIG. 1;
3 is a control flowchart of a multimedia content processing apparatus according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, each embodiment according to the present invention is only one example to help understanding of the present invention, and the present invention is not limited to these embodiments. In particular, the present invention may be composed of a combination of at least one or more of individual configurations, individual functions, or individual steps included in each embodiment.

In particular, for convenience, some claims in the claims include alphabets such as '(a)', but these alphabets do not prescribe the order of each step.

In addition, each signal referred to in each embodiment according to the present invention may refer to one signal transmitted by one connection or the like, but may also mean a series of signal groups transmitted for the purpose of performing a specific function described below. . That is, in each embodiment, a plurality of signals transmitted at predetermined time intervals or transmitted after receiving a response signal from the counterpart device may be represented by one signal name for convenience.

An overall communication system including a multimedia content processing device 100 according to an embodiment of the present invention is shown in FIG. 1.

In the drawing, the content providing server 200 performs a function of providing integrated multimedia contents according to a request of the multimedia content processing apparatus 100. For example, the content providing server 200 provides a broadcasting server or an individual customer that provides a broadcast image captured in real time. It may correspond to a video on demand (VOD) server that extracts and provides pre-stored or recorded content according to a request.

In the following description of the present embodiment, it is assumed that the content providing server 200 receives the captured images from the plurality of cameras 310, 320, and 330 and then provides them to the multimedia content processing apparatus 100.

That is, in FIG. 1, each of the cameras 310, 320, 330 is installed at the same place (for example, a sports stadium) and photographs, and transmits the photographed video signal according to the photographing to the content providing server 200. The providing server 200 receives the captured video signals from each of the cameras 310, 320, 330, performs synchronization processing on the respective captured video signals, integrates them, and transmits them to the multimedia content processing apparatus 100 corresponding to the viewer terminal. It performs the function.

In detail, the content providing server 200 includes predetermined time information in the captured image signals received from the plurality of cameras 310, 320, and 330 to generate respective sub contents, and multiplexes the sub contents into one integrated multimedia content. After generation, the multimedia content may be provided to the multimedia content processing device 100.

For example, the content providing server 200 generates time information for synchronization based on a point in time at which a specific audio signal is detected from each captured image signal, or based on a specific captured image extracted from the captured image signal. Each sub content may be generated by matching the information with the corresponding captured image signal.

In this case, the sub content may be, for example, an MPEG transport stream (ie, an MPEG transport stream, or TS for short) format (hereinafter referred to as a TS file). The MPEG transport stream uses a packet including a header and a payload as its basic data units. Since the structure itself corresponds to a known technology, a detailed description thereof will be omitted.

As described above, a detailed process of generating the sub content by the content providing server 200 will be described below.

For example, the first specific audio signal is detected in the fourth frame of the captured video signal received by the first camera 310 and the first in the sixth frame of the captured video signal received by the second camera 320. When the first specific audio signal is detected, the content providing server 200 determines whether the TS packet including the fourth frame of the captured video signal of the first camera 310 and the captured video signal of the second camera 320 are included. In the TS packet including the sixth frame of the frame, the synchronization time information of '1' may be included in each case. As another example, the second to sixth frame among the frames of the captured video signal received from the first camera 310 may be included. A specific audio signal is detected and a second specific audio signal is detected in the eighth frame among the frames of the captured video signal received from the camera 320. In this case, the content providing server 200 includes a TS packet including a sixth frame of the frame of the captured video signal of the camera 310 and an eighth frame of the frame of the captured video signal of the camera 320. The TS packet may include synchronization time information of '2'.

Subsequently, the content providing server 200 generates integrated multimedia contents including respective sub-contents which have been synchronized. The integrated multimedia contents may correspond to a video signal or video data that can be viewed on one viewing channel.

That is, if the sub content is a file of a TS format as mentioned above, the integrated multimedia content may be referred to as an integrated TS file composed of a combination of these TS files. It can be transmitted in a streaming manner.

Here, the term “file” does not necessarily include all the images from the beginning to the end of the image, but may be a file for each section generated in real time by the captured image signal received from each camera 310, 320, 330.

After generating the integrated multimedia content as described above, the content providing server 200 transmits the generated integrated multimedia content to the multimedia content processing device 100, for example, the content providing server 200 is a multimedia content processing device 100 Integrated multimedia content may be transmitted after connecting a communication channel capable of mutual communication with each other, or may be integrated in a manner of transmitting by radio waves as in a conventional one-way broadcasting station without connecting an interconnection channel with the multimedia content processing apparatus 100. Multimedia content may also be transmitted.

As such, the integrated TS file transmitted by the content providing server 200 to the multimedia content processing apparatus 100 may be transmitted in succession in packet units including a header and a payload. In this case, specific information in the first header for session connection may be transmitted. The multimedia content processing apparatus 100 may be informed that the stream is a stream for transmitting a plurality of TS files.

That is, the content providing server 200 generates the integrated TS file by including information on how many TS files are included in the packet for the session (header and payload), and each individual TS thereafter. The process may include attaching the file sequentially.

For example, if the header and payload of the first integrated TS file stream for a session connection specify that three programs (sub-contents) are included in the integrated TS file, then the content provisioning server 200 may subsequently Three respective TS files (sub content) are sequentially sent to the packet (header + payload), and this process (the process of sending three sub contents sequentially or the whole process) may be repeated.

On the other hand, the multimedia content processing device 100 performs the function of processing the integrated multimedia content received from the content providing server 200 as described above, as shown in the drawing, the set-top box 101 and the television ( 102 or may be configured as a portable wireless communication terminal 100.

Specific functional blocks of the multimedia content processing device 100 are illustrated in FIG. 2.

As shown in the drawing, the multimedia content processing apparatus 100 may include a display unit 150, a receiver 110, a determiner 120, an extractor 130, and a controller 140.

The display unit 150 displays each multimedia content received from the content providing server 200 and may include a display element such as a liquid crystal display (LCD) with low power consumption.

The receiving unit 110 performs a function of receiving integrated multimedia content including at least one sub content from the content providing server 200 through one content receiving channel.

Here, the content reception channel is selected by the user to watch a specific broadcast or specific multimedia content. For example, the content reception channel may correspond to a broadcast channel having a specific number or an Internet address consisting of IP: port.

In the following description of the present embodiment, the content receiving channel includes an IP address and a specific port of the content providing server 200 as an example. In this case, the receiving unit 110 integrates integrated multimedia content including at least one sub-content. To connect to and listen to a specific IP: port.

In addition, although the integrated multimedia content received by the receiver 110 may be configured in various forms, in the present embodiment, it is an example of an integrated TS file including at least one sub-content of a TS format.

That is, the receiver 110 may receive the integrated TS file in a streaming manner through one content reception channel (for example, one broadcast channel).

In this case, the integrated TS file may include a packet (header + payload) for session connection and a packet (header + payload) including actual sub content, as described above. It may be included in the payload of the packet for.

The determination unit 120 performs a function of determining the number of sub contents included in the integrated multimedia content by using the information included in the header of the integrated multimedia content received by the receiver 110.

As described above, the content providing server 200 may generate and provide, for example, integrated multimedia content including a plurality of sub contents, and the determination unit 120 of the multimedia content processing apparatus 100 may provide such integrated multimedia content. It is to check how many sub-contents are included by using the information included in the header of the.

For example, if PID (packet ID) is 0 in the header of a packet for session connection, the determination unit 120 determines how many TS files (i.e., how many sub-contents) to transmit to the next payload. The determination unit 120 may determine how many sub-contents are included in the integrated multimedia content, that is, how many sub-contents are included by reading a value specified in the corresponding payload. .

The extractor 130 extracts each sub content from the integrated multimedia content by the number of sub contents determined by the determiner 120.

For example, when the determination unit 120 determines that the number of sub contents is three, the extraction unit 130 sequentially extracts three sub contents from the integrated multimedia contents. In this case, the preset packet size or format size may be referred to. Can be.

The controller 140 performs a function of controlling each sub content to be reproduced together after synchronizing each sub content using time information included in each sub content extracted by the extractor 130. .

For example, when each sub content is voice content, the controller 140 may output the synchronized sub content together through a speaker (not shown).

As another example, when each sub content is a content including an image, the controller 140 controls to arrange and display the synchronized sub content according to a preset algorithm on the screen.

In this case, the controller 140 may generate the decoding modules as many as the number of sub contents determined by the determination unit 120, and time-synchronize them while continuously assigning each sub content to the generated decoding modules in the order.

Specifically, the controller 140 controls each decoding module to wait until all of the time information included in the sub content allocated to each decoding module matches, and the time information included in the sub content allocated to the decoding module, respectively. When all match, the sub content decoded by each decoding module may be controlled to be displayed in an area on the screen allocated to each decoding module.

For example, the decoding module is composed of three (first decoding module, second decoding, third decoding module), and the time information included in the sub content allocated to the first decoding module and the second decoding module is all ' 3 'and the time information included in the sub content allocated to the third decoding module is' 1', the controller 140 allows the first decoding module and the second decoding module to maintain a standby state, and the third decoding. When the time information included in the sub content allocated to the module becomes '3' (that is, when the time information included in the sub content allocated to all the decoding modules becomes the same), each sub content decoded by each decoding module is added. It can be controlled to be displayed in the area on the screen assigned to each decoding module.

Arranging / displaying according to a preset algorithm may mean arranging based on a default order when there is no user request, and arranging and displaying according to the request when there is a user request.

That is, in the state where the sub content by the first decoding module, the sub content by the second decoding module, and the sub content by the third decoding module are displayed on the display 150, the order may be changed according to a user selection. Each sub content may be overlapped and displayed according to the layer order in which the sub contents are set.

Accordingly, a user who views the content using the multimedia content processing apparatus 100 may simultaneously view images captured by the cameras 310, 320, and 330 at the same time.

Here, the decoding module corresponds to a specific function in a kind of software program that processes multimedia content, and the generation of the decoding module may be performed in the same manner as a function call. The process of assigning sub content to each decoding module may be based on a variable value of the function. The sub content data value may be allocated.

That is, the control unit 140 may be composed of a manager module for managing the decoding module and the decoding module integrated, the manager module controls the operation / standby of each decoding module, each decoding module decodes / It is to play and display.

As another example, the controller 140 generates decoding threads as many as the number of sub contents determined by the determination unit 120, and time-synchronizes the sub-contents continuously with the generated decoding threads in the order thereof. Each decoding thread may be controlled such that each time-synchronized sub-content is placed and displayed on the screen.

Here, a thread is distinguished from a process, and can be referred to as an execution unit that separates execution control from a process. It is a basic unit that uses a processor and is a control flow that can execute instructions independently. Can be.

In the case of generating a decoding thread as many as the number of sub-contents, it can be referred to as a multi-threaded structure. In this case, since the redundancy of resource generation and management is minimized, the execution performance can be improved and the processing can be performed quickly. There is an effect that the function of the controller 140 can be implemented. This is the same when using the above-described functions (modules), and also in the case of a multi-threaded structure, as described above, a separate manager program for controlling the operation / standby of each decoding thread may be included in the controller 140. Of course.

Of course, a separate player may be driven to decode / play each sub-content so that it is displayed on the display 150. Thus, the controller 140 drives only one player and a separate decoding module / for each sub-content. Creating a thread and decoding / reproducing is preferable when considering the speed of the above-described processing.

Hereinafter, a control process of the multimedia content processing apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. 3.

First, in the present embodiment, the content providing server 200 transmits the integrated multimedia contents including the plurality of sub contents, and the integrated multimedia contents are provided in the integrated TS format and are provided in a streaming manner. Yes.

The multimedia content processing apparatus 100 receives the integrated multimedia content from the content providing server 200 through one content receiving channel (step S1).

The multimedia content processing apparatus 100 checks the number of sub contents included in the integrated multimedia content on the basis of the information included in the header of the integrated multimedia content (step S3).

Subsequently, the multimedia content processing apparatus 100 calls or generates a decoding module as many as the number of identified sub contents (step S5).

After the decoding module is called or generated, the multimedia content processing apparatus 100 matches specific sub content for each generated decoding module (step S7).

For example, if the integrated multimedia content includes a first TS packet (first sub content), a second TS packet (second sub content), and a third TS packet (third sub content), the first TS packet. Is matched with the first decoding module, the second TS packet is matched with the second decoding module, and the third TS packet is matched with the third decoding module.

Referring to FIG. 1, the first TS packet may correspond to an image packet captured by the first camera 310 (but as described above, time information for synchronization is included by the content providing server 200). The second TS packet may correspond to an image packet captured by the second camera 320, and the third TS packet may correspond to an image packet photographed by the third camera 330.

Thereafter, the multimedia content processing apparatus 100 delivers each sub content continuously received from the content providing server 200 to the matched decoding module (step S9).

That is, the multimedia content processing apparatus 100 continuously transmits the first TS packet photographed and transmitted by the first camera 310 to the first decoding module, and is photographed and transmitted by the second camera 320. 2 TS packets are continuously delivered to the second decoding module, and the third TS packets photographed and transmitted by the third camera 330 are continuously delivered to the third decoding module.

Subsequently, the multimedia content processing apparatus 100 extracts synchronization time information included in the sub content delivered to each decoding module (step S11). That is, synchronization time information is extracted from headers of the first TS packet, the second TS packet, and the third TS packet.

The multimedia content processing apparatus 100 performs a time synchronization process for each sub content based on the extracted time information (step S13), for example, at least until the time of each sub content delivered to each decoding module coincides. Some decoding modules can be kept idle.

When the time synchronization process is completed, that is, when the time of each sub content delivered to each decoding module coincides, the multimedia content processing apparatus 100 displays and reproduces the time synchronized sub content by the corresponding decoding module. 150 can be displayed (step S15).

The time synchronization process of step S13 described above may be repeated for each preset time, for example, at periodic time intervals.

That is, the multimedia content processing apparatus 100 may not perform the time synchronization process only for the first time, but may perform the time synchronization process repeatedly by periodically.

On the other hand, the process of performing each of the above-described embodiments can be performed by a program or an application stored in a predetermined recording medium (for example, computer readable). Here, the recording medium includes both an electronic recording medium such as a random access memory (RAM), a magnetic recording medium such as a hard disk, an optical recording medium such as a compact disk (CD), and the like.

In this case, the program stored in the recording medium may be executed on hardware such as a computer or a smartphone to perform the above-described embodiments. In particular, at least one of the functional blocks of the multimedia content processing apparatus according to the present invention described above may be implemented by such a program or application.

In addition, this invention is not limited to the above-mentioned specific Example, It can implement in a various deformation | transformation and correction within the range which does not deviate from the summary of this invention. It will be apparent that such variations and modifications are included in the present invention as long as they belong to the appended claims.

100: multimedia content processing device
200: content providing server
310,320,330: Camera
110: receiving unit 120: determining unit
130: extraction unit 140: control unit
150: display unit

Claims (14)

(a) receiving integrated multimedia content including at least one sub-content from a content providing server;
(b) determining the number of sub-contents included in the integrated multimedia content by using the information included in the header of the integrated multimedia content received in step (a);
(c) extracting each sub content from the integrated multimedia content by the number of sub contents determined in the step (b);
(d) synchronizing each sub content using time information included in each sub content extracted in step (c), and then placing and synchronizing each sub content together according to a preset algorithm on the screen. Controlling to be displayed;
In step (d),
(d4) generating decoding threads by the number of sub contents determined in the step (b);
(d5) time-synchronizing continuously assigning each sub-content extracted in the step (c) to the decoding thread generated in the step (d4) in the order thereof;
and (d6) controlling each sub-content which is time-synchronized in step (d5) to be arranged and displayed according to a preset algorithm on a screen. delete delete delete delete The method of claim 1,
In the step (a), the control method of the content processing device, characterized in that for receiving the integrated multimedia content including at least one sub-content from the content providing server via one content receiving channel. A computer-readable recording medium having recorded thereon a program for executing the method of claim 1. A computer program stored in a computer readable recording medium in combination with hardware to carry out the method of claim 1. A receiving unit for receiving integrated multimedia content including at least one sub content from a content providing server through a content receiving channel;
A determination unit which determines the number of sub contents included in the integrated multimedia content by using information included in the header of the integrated multimedia content received by the receiving unit;
An extraction unit for extracting each sub content from the integrated multimedia content by the number of sub contents determined by the determination unit;
A control unit for synchronizing each sub content by using time information included in each sub content extracted by the extracting unit, and controlling the arranged sub-contents to be arranged and displayed according to a predetermined algorithm on the screen while the synchronized sub content is played together Including,
The control unit generates decoding threads as many as the number of sub-contents determined by the determination unit, and time-synchronizes each sub-content extracted by the extraction unit to the generated decoding threads in a sequential order. And controlling each decoding thread such that each sub content is arranged and displayed on a screen. delete delete delete delete The method of claim 9,
And the receiving unit receives integrated multimedia content including at least one sub content from the content providing server through one content receiving channel.

Images