KR20120057034A - System and method for adaptively streaming of images having multi component - Google Patents

Abstract

PURPOSE: A system for adaptively streaming a moving picture with multi components and a method thereof are provided to efficiently transceive a series of image streams without modifying a separate encoding method wherein the series of image streams are needed in a terminal device. CONSTITUTION: A server device(100) stores multi component content. The server device extracts image reference information for decoding the multi component content. A terminal device receives an image component from the server device based on the extracted image reference information. The terminal device decodes and displays the received image component on a screen.

Description

Adaptive streaming system and method for multi-element video {SYSTEM AND METHOD FOR ADAPTIVELY STREAMING OF IMAGES HAVING MULTI COMPONENT}

The present invention relates to a technique for transmitting and receiving encoded video having multiple elements. The present invention relates to a video stream from a user's video selection and a reference information structure of a video stream composed of a combination of time information and a video stream in order to construct a video selected by a user. Disclosed is a technical idea of receiving necessary video stream information and a corresponding stream according to each time.

Recently, with the development of digital imaging technology, interest in 3D imaging technology in which spatial and temporal elements are applied has increased. Three-dimensional video is in the spotlight as the next generation of imaging technology along with high-performance video. The 3D image may provide the user with a three-dimensional image at any point in time.

In general, a multiview video coding (MVC) method or a scalable video coding (SVC) method may send only a difference value between scenes based on similarities between spatially and temporally close scenes.

The MVC scheme is a future three-dimensional TV that enables the interaction between the system and the user so that the user can view a scene at a desired time point for images captured at the same time from cameras disposed at spatially different positions. It is a new type of video encoding standard to support free viewpoint video and 3D security system.

In general, a multi-view video sequence has a proportional amount of data to be processed compared to a conventional single-view video sequence in proportion to the number of cameras used and the size of the image. In order to process these multi-view video images, two international video coding standards organizations (MPEG, VCEG) are required to efficiently encode the multi-view video images having different characteristics from those of single-view video images. International standardization is underway to meet the requirements for encoding multi-view video images.

A terminal receiving an image composed of multiple elements may play an image based on all or some elements constituting the image according to its type.

In the case of a streaming scheme in which only a specific video element is played by a receiving terminal and a video element is sequentially received by a server, a signal exchange for which video element is to be transmitted and received first is necessary.

For example, in the case of a multi-view image, the image is composed of view 1 and view 2, and if the user selects view 1, the image element required at view 1 is sent first in terms of transmission efficiency, and other image elements Needs to transmit and receive only when another time point is selected later.

If each view is composed of independent video streams, changes between views can be treated like virtual channel changes.However, when transmitting and receiving a multi-view video encoded by MVC in a conventional manner, regardless of the user's selection time Only when video elements are received at all times can a video stream at a desired time be decoded.

This is a problem that the advantages of the MVC method for the purpose of encoding efficiency of the multi-view video stream is canceled during the streaming process.

In an adaptive streaming system of a video according to an embodiment of the present invention, a server apparatus for storing multi-element content and extracting image reference information for decoding from the multi-element content, and corresponding to the extracted image reference information, And a terminal device for receiving an image element from the server device, decoding the received image element, and displaying the image element on a screen.

In an adaptive streaming method of a video according to an embodiment of the present invention, the method may further include storing multi-element content, extracting image reference information for decoding from the multi-element content, and corresponding to the extracted image reference information. The method may include receiving an image element from a device, and decoding and displaying the received image element on a screen.

According to an embodiment of the present invention, it is possible to efficiently transmit the necessary video elements according to the user's selection.

According to an embodiment of the present invention, a series of video streams required by a terminal device can be efficiently transmitted and received without modification of a separate encoding scheme.

1 is a block diagram illustrating a server device in an adaptive streaming system according to an embodiment of the present invention.
2 is a block diagram illustrating a terminal device in an adaptive streaming system according to an embodiment of the present invention.
3 is a diagram illustrating image reference information according to a series of image elements selected by a user according to an embodiment of the present invention.
4 and 5 are flowcharts illustrating an operation of a streaming receiver for receiving a series of video elements according to a user's selection.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

In describing the present invention, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. The terminologies used herein are terms used to properly represent preferred embodiments of the present invention, which may vary depending on the user, the intent of the operator, or the practice of the field to which the present invention belongs. Therefore, the definitions of the terms should be made based on the contents throughout the specification. Like reference numerals in the drawings denote like elements.

1 is a block diagram illustrating a server device in an adaptive streaming system according to an embodiment of the present invention.

The server device 100 according to an embodiment of the present invention may store multi-element content, extract image reference information for decoding from the multi-element content, and provide the same to a terminal device.

To this end, the server device 100 according to an embodiment of the present invention may include a multi-element content storage unit 110, an image reference information extraction unit 120, and a streaming transmitter 130.

Not only one video stream but also multi-element content composed of various image elements such as multi-view may be stored and maintained in the content storage unit 110.

The image reference information extractor 120 may extract image reference information for decoding one scene from the multi-element content.

The image reference information may be variously implemented according to an encoding scheme, and when encoding of a multiview video coding (MVC) scheme is used, the reference information may be configured as a reference mapping table between I, B, and P for each view point and time.

The streaming transmitter 130 may extract and transmit only necessary video elements for a series of video elements.

2 is a block diagram illustrating a terminal device in an adaptive streaming system according to an embodiment of the present invention.

The terminal device 200 according to an embodiment of the present invention may receive an image element from the server device in response to the extracted image reference information, and decode the received image element to display it on the screen.

To this end, the terminal device 200 according to an embodiment of the present invention may include a streaming receiver 210, a decoder 220, and a screen unit 230.

The streaming receiver 210 may receive an image element corresponding to the user's selection based on the image reference information.

To this end, the streaming receiver 210 may receive and confirm the image reference information extracted by the image reference information extractor 120. For example, the streaming receiver 210 may receive the necessary stream while increasing the discrete time value from the initial video playback time to the final video playback time in order to analyze the received video reference information.

In this case, the video element may be received by extracting and transmitting a video element required by the streaming transmitter 130 of FIG. 1.

The decoder 220 may decode the received series of image elements and convert the image elements into necessary images, and the screen unit 230 may display the converted images to the user through the display apparatus.

The user may check the image displayed on the display device to check whether the image is selected by the user.

As a result, according to an embodiment of the present invention, it is possible to efficiently transmit the required video elements according to the user's selection, and can efficiently transmit and receive a series of video streams required by the terminal device without modification of a separate encoding scheme.

Therefore, by using the adaptive streaming system of the video to which the server device and the terminal device according to an embodiment of the present invention is applied, it is possible to provide a fast and accurate quality service with less resources. In particular, by quickly processing the decoding of the streamed video, it is possible to provide a stable system.

3 is a diagram illustrating image reference information according to a series of image elements selected by a user according to an embodiment of the present invention.

As in the case of the MVC encoding method, the image reference information according to a series of image elements selected by the user may be displayed as shown in FIG. 3.

Referring to FIG. 3, a set of time information (timestamps 302, 303, 304) and stream sets 305, 306 corresponding to time information (timestamps 302, 303, 304) according to a user's selection Vn 301. , 307 may be configured as a mapping table.

As shown in FIG. 3, when the stream set Ri 305 is received at time t1 302, the terminal device according to an embodiment of the present invention may decode an image corresponding to Vn 301.

Since the configuration of generating the image reference information can be flexibly selected according to the encoding scheme and the method of configuring the series stream, it is obvious that various methods can be applied.

4 and 5 are flowcharts illustrating an operation of a streaming receiver for receiving a series of video elements according to a user's selection.

In this specification, a basic implementation method is presented, and an extension method that can be implemented by an expert in a related field is not separately specified.

According to the present invention, the user can view a series of image elements displayed on the screen according to one user selection.

First, referring to FIG. 4, a user may select one image to acquire an image.

For example, when the image is a multi-view image, the user may select a specific point in time when the image is selected.

Accordingly, the terminal device according to an embodiment of the present invention may receive an image selection signal corresponding to the selection input from the user (step 401).

Next, the streaming receiver according to an embodiment of the present invention may request image reference information on an image corresponding to the image selection signal from the server device (step 402), and receive it (step 403).

For example, in response to the request for the image reference information, the server device may extract image reference information for decoding one scene from the multi-element content through the image reference information extracting unit, and transmit the image reference information to the streaming receiving unit.

In addition, in response to the request for the video reference information, the streaming transmitter may extract only the necessary video elements of the series of video elements and transmit the extracted video elements to the streaming receiver of the terminal device.

Next, the streaming receiver may process a series of operations for playing back an image (step 404). A series of operations for the streaming receiver to play the video will be described in detail with reference to FIG. 5.

Referring to FIG. 5, the streaming receiver according to an embodiment of the present invention may receive a necessary stream while increasing a discrete time value from an initial video playback time to a final video playback time in order to analyze the received video reference information. .

First, the streaming receiver according to an embodiment of the present invention may obtain the fastest ti after the current time t (501).

Next, the streaming receiver according to an embodiment of the present invention may acquire a set Ri of streams corresponding to the obtained ti (502).

In addition, the streaming receiver according to an embodiment of the present invention may request and receive the corresponding stream from the streaming transmitter according to the acquired Ri and ti (step 503).

The streaming receiver according to an embodiment of the present invention may transfer the received video stream to the decoder to request decoding (step 504).

According to an embodiment of the present invention, the streaming receiver may determine whether the current time exceeds the last video playback time (step 505), and if so, terminates the processing, and if not, branches to step 501 again. have.

That is, the streaming receiver according to an embodiment of the present invention may repeatedly perform the decoding process for decoding the steps 501 to 504 until the end of the video.

According to an embodiment of the present invention, not only the necessary video elements according to the user's selection can be efficiently transmitted, but also a series of video streams required by the terminal device can be efficiently transmitted and received without modification of a separate encoding scheme.

The adaptive streaming method of a video according to an embodiment of the present invention stores multi-element content, extracts video reference information for decoding from the multi-element content, and corresponds to the extracted video reference information from a server device. An image element may be received, and the received image element may be decoded and displayed on a screen.

The adaptive streaming method of a video according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

100: server device
110: multi-element content storage
120: image reference information extraction unit
130: streaming transmitter
200: terminal device
210: streaming receiver
220: decryption unit
230: screen portion

Claims (1)

A server device for storing multi-element content and extracting image reference information for decoding from the multi-element content; And
In response to the extracted image reference information, a terminal device that receives an image element from the server device, decodes the received image element and displays it on a screen.
Adaptive streaming system of a video comprising a.

Images