KR20150029461A - Method of synchronizing tiled image in a streaming service providing system of panoramic image - Google Patents

Abstract

The present invention relates to a method of synchronizing a tiled image in a streaming service providing system of a panoramic image. The method includes: a) inputting a panoramic image photographed from a camera in a server; b) classifying the panoramic image input in the server by a time period; c) tiling the panoramic image by the photographed time period to divide the panoramic image in a block unit; d) compressing the tile image to store the compressed tile image in the server; and e) providing the tile image and the reference signal to at least one client terminal. The step e) includes: e1) decoding the tile image and the reference image with respect to a specific frame; e2) storing the decoded data in a data buffer to wait; e3) determining whether a data buffer corresponding to all reference images and time image is filled with a specific frame; e4) outputting the tile image and the reference image synchronized on a screen when the all data buffers are filled; and e5) initializing a data buffer.

Description

Technical Field [0001] The present invention relates to a method for synchronizing a tiling image in a streaming service providing system of a panorama image,

The present invention relates to a system for providing a streaming service of a panoramic image, and more particularly, to a method of synchronizing tiled images when transmitting a tiled image for efficient streaming.

UHD TV is one of the next-generation broadcasts after HDTV. This will satisfy the quality needs of consumers by providing theater-quality ultra-high-quality service at home (equivalent to 7680 × 4320 video resolution, 16 times the screen size of HD) and multi-channel (~ 22.2ch) Can aim for TV.

Generally, a service providing a panoramic image photographed by a camera is divided into a still image, which is a static method, and a moving image, which is a dynamic method. However, in order to display an image at a high resolution, the capacity of a file to be provided must be increased. If such a high-capacity file is transmitted, the bandwidth of the network becomes large, and at the same time, the load on the transmission server is increased.

Therefore, the streaming service using the tiling technique which smoothes the bandwidth of the network while reducing the capacity of the moving image and provides images similar to the still image of high resolution is proposed. This realizes a realistic image service by enabling users to interactively provide scenes desired by viewers in a smart device providing HD images such as smart phones and smart pads using UHD-level panoramic images by using image tiling technology .

Specifically, the interactive video service for a smart device based on a multi-view panorama video captures video content such as performance, sports broadcast, and education as a 360-degree panoramic video, stores it on a server, It is an interactive video service that enables zoom in / out and panning according to the view.

However, there is a problem in the synchronization method between the tiling images in the streaming service providing system of the panorama image using the tiling technology.

Korean Patent Publication No. 2005-0091278 Korean Patent Publication No. 2006-0050348

It is an object of the present invention to provide a method of synchronizing a tiled image in a system for providing a streaming service of a panoramic image using a tiling technology.

According to another aspect of the present invention, there is provided a method of synchronizing a tiled image performed by a system for providing a streaming service of a panorama image, the method comprising: a) inputting a panorama image photographed by a camera to a server; b) C) dividing each panorama image classified by the photographed time period into tiles and dividing the panorama images into blocks, d) compressing each tile image and storing the tile image in the server E1) decoding the tile image and reference image for a particular frame; e2) decoding the tile image and reference image for a particular frame; and e) Storing the decoded data in a data buffer and waiting; e3) Determining a data buffer corresponding to the video Were filled, e4) includes the steps of initializing the stage, e5) data buffer for all of the data buffer is full jyeoteumyeon, screen outputs the synchronized said tile image and the reference image.

At this time, it is preferable that each of the reference image and the tile image has a separate decoding stream, and each decoding stream has a unique ID.

In addition, it is preferable that each of the decoded streams can be used in multiple ways, and that they can be individually controlled.

It is preferable that the tile image is synchronized with the reference image every frame.

Wherein the step (e) further comprises, when the client terminal requests to provide the panoramic image photographed in the specific time zone and the specific time zone within the specific time zone, An image of a specific time zone and a specific screen region within the specific time zone may be provided to the client terminal.

In this case, provision of a specific screen region image within the specific time zone and the specific time zone from the server to the client terminal may include separating a low resolution full screen reference image of the specific time period and a tile image of the specific screen region, Wherein the reference image is arranged and displayed as a background screen on the client terminal, the tile image is synthesized with the reference image displayed as a background screen on the client terminal, and the composite image is displayed on the client terminal as a panorama image And allowing the composite image to be texture mapped to a sphere to enable it to be displayed.

As described above, according to the present invention, an efficient streaming system for transmitting a panorama image for providing a wide viewing area to a client in a tile image unit, It is possible to freely access a tile area that the client is looking at to a plurality of clients in a given area, as well as transmitting a visual area of a certain size to a client in a panoramic image coming in via an input through a camera.

In particular, by providing a synchronization method between tiled images in a streaming service providing system of a panorama image using a tiling technology, an accurate screen can be smoothly provided to a client.

1 is an example of a system for providing a streaming service of a panoramic image,
FIG. 2 is a flowchart illustrating a method of providing a streaming service of a panoramic image performed in FIG.
3 is a conceptual diagram for explaining an example of a tiling video,
4 is a flowchart illustrating a method of providing a tile image to a client terminal in a server,
5 is a flowchart for explaining a tiling image synchronization method according to a preferred embodiment of the present invention.
6 is a conceptual diagram for explaining a matching between a data buffer and a moving picture in a system for providing a streaming service of a panoramic image,
Figs. 7A to 7D and Fig. 8 are exemplary diagrams illustrating tiling image synchronization according to successive frames.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it should be understood that the following embodiments are provided so that those skilled in the art will be able to fully understand the present invention, and that various modifications may be made without departing from the scope of the present invention. It is not.

FIG. 1 is an example of a system for providing a streaming service of a panoramic image, and FIG. 2 is a flowchart illustrating a method of providing a streaming service of the panoramic image performed in FIG.

1, a system for providing a streaming service of a panoramic image includes a camera 100 in which a panoramic image is captured, a server 200, a client terminal 300 that receives a tile image provided by the server 200, . The server 200 includes an image input unit 210 for inputting panoramic images photographed by the camera 100 in the form of files, a first input unit 210 for inputting the panoramic images input through the image input unit 210, A second image conversion unit 230 for generating a tile image in which each panorama image classified in each of the time zones photographed by the first image conversion unit 220 is tiled and divided into blocks, A storage unit 240 for compressing and storing the tile image generated by the second image conversion unit 230 and an image providing unit 240 for providing the tile image compressed and stored in the storage unit 240 to at least one client terminal 300 250).

A method of providing a streaming service of a panorama image performed in a system for providing a streaming service of a panorama image includes a panorama image input step (S100) in which a panorama image photographed by a camera is input to a server in a file form, A panoramic image temporal classifying step S200 of classifying the panoramic image by time zone in which the panoramic image is photographed, a tile image generating step S300 of tiling each panaural image classified by the photographed time zone, A tile image storing step (S400) of storing compressed images in a server, and a tile image providing step (S500) in which a tile image stored in a server is provided to at least one client terminal.

Here, the tile image providing step according to step S500 is a step of providing the tile image to the server 200 when the client terminal 300 requests to provide the panorama image in which the specific time zone and the specific point of view within the specific time zone are photographed The client terminal 300 is provided with an image of a specific time zone and a specific view point of view within a specific time zone requested by the client terminal among the stored tile images.

Specifically, the server 200 provides the client terminal 300 with a specific point of view image within a specific time zone and a specific time zone. This is because a low-resolution full-screen image and a specific screen region the method comprising the steps of: separating a tile image of a low-resolution point-of-view image from a tile image of a low-resolution full-screen image; And a step in which the composite image is texture mapped to a sphere so that the composite image is displayed on the client terminal as a panorama image.

In such a system, if a signal requesting only a certain region of the entire panorama image from the client terminal 300 is inputted to the server 200, the server 200 transmits the image of a certain region to the client terminal 300. The client terminal 300 provides a viewpoint moving function. When each client terminal 300 receives a tile image of a desired portion from the server and wants to view an image of a new region, the client terminal 300 transmits information indicating that the viewing region has changed To the server (200). Then, the client 200 transmits the image of the area requested by the client terminal 300 among the tiled time zones in the server 200. In other words, the present invention provides a limited viewing area for multiple clients and a viewpoint movement in the area, which is advantageous in that the viewer can control the movement of the viewpoint by transmitting the image as much as the viewing area reproduced when the image is transmitted to the client have.

The tile image refers to an image obtained by separating an image into small units as shown in FIG. Thus, the purpose of tiling the image is to minimize the delay time in image reproduction according to the network speed. So the advantage of tiling is that it can speed up the video playback on the client. That is, in order to reduce the compression burden of the server, the entire panorama image is not compressed and transmitted to the client, but the panorama image is provided to the client by tiling only a limited area of the predetermined size.

4, when a client receives streaming data, it is necessary to receive the entire panorama image and the tiled panorama image, and to match the time axis.

Hereinafter, a tiling moving picture synchronization method performed in a system for providing a streaming service of a panorama image will be described with reference to FIGS. 5 to 8. FIG.

FIG. 5 is a flow chart for explaining a tiling moving picture synchronizing method according to a preferred embodiment of the present invention, and FIG. 6 is a conceptual diagram for explaining matching of a data buffer and a moving picture in a system for providing a streaming service of a panorama image.

Referring to FIG. 6, a tiled moving picture for a region of interest (ROI) is placed on a reference moving picture background, and each reference moving picture and a tiled moving picture have a data buffer for picture output. This is because the reference video and the tiling video reproduction screen must be synchronized with each other when the screen is output to the user terminal.

Here, each of the reference moving image and the tiled moving image has a separate decoding stream for video synchronization. Each decoding stream retains a unique ID, is available in multiple as far as it is possible to identify, and is individually controllable.

The video synchronization is based on the frame of the reference video having a low resolution, and the tiling video is synchronized with the reference video every frame. That is, when the decoding stream of the reference moving picture reaches a specific frame, the data is stored in the buffer and then put in a standby state. When the decoding stream of each tiling moving picture reaches the corresponding frame of the reference moving picture, decoding of another tiling moving picture is waited.

Referring to FIG. 5, the synchronization method includes decoding (S510) moving picture data for a specific frame, storing the decoded data in a data buffer and waiting (S520) A step S530 of determining whether the data buffer for storing the synchronized image is filled up, a step S540 for outputting a synchronized image and a data buffer initializing step S550. This synchronization method will be described in more detail with reference to Figs. 7A to 7D.

First, video data is decoded for a specific frame (S510). For example, as shown in FIG. 7A, for 94 frames after 93 frames, each reference moving picture and tile moving pictures 1 to 4 are decoded with separate decoding streams. At this time, the data buffer before decoding the moving picture data is in an initial state and all are empty.

Next, the decoded data is stored in the data buffer and waits (S520). For example, as shown in FIG. 7B, when the decoding stream of the reference moving picture and the tile moving picture 1 reaches the corresponding 94 frames, the moving picture is stored in the data buffers 0 and 1, and the decoded streams of the remaining tile moving pictures 2, Waits until it reaches 94 frames. At this time, if the status of the data buffer is checked and the data buffers are not all filled up, the display waits without outputting the screen.

Next, it is determined whether the data buffers corresponding to all the reference moving pictures and the tile moving pictures are filled in the specific frame (S530). If all data buffers are filled, a screen of the synchronized image is output (S540). However, if all the data buffers have not been filled yet, continue to store the video in the data buffer. For example, as shown in FIG. 7C, if the reference moving image and the tile videos 1, 2, 3, and 4 reach 94 frames and the data buffers 0 to 4 are all filled, the entire moving image is displayed on the screen. Therefore, the entire screen is output depending on the speed of the reference moving picture.

Next, the data buffer is initialized to output a screen for the next frame (S550). That is, as shown in FIG. 7D, all of the data buffers 0 to 5 are initialized in order to output the screen for 95 frames, so that the data buffers are all emptied.

The synchronization between the reference video and the tile video is adjusted based on each frame of the continuous video. FIG. 8 shows such a tiling moving picture synchronization according to a continuous frame. The data buffer at the top of the drawing shows a state in which each data buffer is filled in accordance with a frame. The middle of the drawing shows a synchronization step according to the frame. Shows the screen output according to the frame.

Referring to FIG. 8, when the screen output for 93 frames is completed, the data buffer is initialized. Then, tiling images for 94 frames are filled into the data buffer, and when all the data buffers are filled, an image for 94 frames is output to the screen. The data buffer is then initialized, and the same procedure is performed for 95 frames.

100: camera 200: server
210: image input unit 220: first image conversion unit
230: second image conversion unit 240:
250: Image providing unit 300: Client terminal

Claims (6)

A method for synchronizing a tiling image performed in a streaming service providing system of a panorama image,
a) a panorama image photographed by a camera is input to a server; b) the panorama image input to the server is classified according to a time zone in which the panorama image is photographed; c) each panorama image classified by the photographed time zone is tiled And d) separating each tile image into a plurality of blocks, and storing the compressed tile image in the server, and e) providing the tile image and the reference image to at least one client terminal,
In the step e)
e1) decoding the tile image and reference image for a particular frame;
e2) storing the decoded data in a data buffer and waiting;
e3) determining whether a data buffer corresponding to all reference images and tile images in a specific frame is filled;
e4) outputting the synchronized tile image and the reference image when all the data buffers are filled;
and e5) initializing the data buffer. 2. The method of claim 1, wherein each reference and tile image has a separate decoding stream, each decoding stream having a unique ID. 3. The method as claimed in claim 2, wherein each of the decoded streams can be used in multiple ways and can be individually controlled. The method as claimed in claim 1, wherein the tile image is synchronized with a reference image having a low resolution, and the tile image is synchronized with a reference image every frame. The method of claim 1, wherein the step (e)
When the client terminal requests to provide the panoramic image photographed in the specific time zone and the specific screen region within the specific time zone,
Wherein the image of each specific tile image stored in the server is provided to the client terminal in the specific time zone requested by the client terminal and in a specific screen area within the specific time zone. 6. The method of claim 5,
Wherein the providing of the specific screen area image within the specific time zone and the specific time zone from the server to the client terminal,
Separating the low resolution full screen reference image in the specific time period and the tile image in the specific screen area;
Displaying the reference image as a background screen on the client terminal;
Synthesizing the tile image with the reference image displayed as a background screen on the client terminal; And
Wherein the composite image is texture mapped to a sphere so that the composite image is displayed as a panorama image on the client terminal.

Images