KR20020070015A - Method and apparatus for compressing and broadcasting the spherical or panoramic moving pictures - Google Patents

Abstract

PURPOSE: A method and a device for compressing or transmitting 360° panoramic or sphere motion pictures are provided to supply broadcasting motion pictures transmitted from at least two or more cameras after three dimensional re-construction through the rotation, expansion or reduction of the pictures by 360°. CONSTITUTION: A method for compressing or transmitting 360° panoramic or sphere motion pictures includes the steps of photographing a site with at least two cameras and transmitting the photographed images by a panoramic or sphere motion picture encoding system, synthesizing respective frames of the photographed and transmitted images into a panoramic or sphere frame by the IBR(Image Based Rendering) technique in the motion picture encoding system, compressing the synthesized frame into motion pictures, and transmitting a predetermined portion of the encoded panoramic or sphere motion pictures.

Description

METHOD AND APPARATUS FOR COMPRESSING AND BROADCASTING THE SPHERICAL OR PANORAMIC MOVING PICTURES}

The present invention relates to devices and methods in the field of broadcast transmission, such as digital TV, Internet broadcasting, IMT-2000, and in particular, each video input from two or more relay cameras is three-dimensionally utilized by using IBR (Image Based Rendering) technology. The present invention relates to a broadcast relay device and a method by which a viewer can rotate a video of a relay site by 360 degrees up, down, left, and right to view, enlarge, reduce, and search for a desired video.

The spread of digital broadcasting technology and internet broadcasting has made it possible to provide interactive broadcasting that allows viewers to participate differently from conventional broadcasting. Accordingly, viewers can participate in broadcasting by searching for various information relayed through the broadcasting.

However, such a broadcast relay has a disadvantage in that it transmits only the point where the broadcast camera is photographing. No matter how many cameras are installed, viewers will only see what the broadcast camera is shooting.

In other words, many viewers try to look at various, realistic screens that do not appear on the screen by looking at 360 degrees up, down, left, and right around the camera that the camera is not relaying in the existing broadcast, but this has not been possible until now. In addition, it is technically impossible to remotely control the camera to relay the object desired by the viewer because of the millions of viewers.

Therefore, by reconstructing the broadcast broadcast screen continuously transmitted using a conventional broadcast relay camera in three dimensions and transmitting it to the viewer, if the viewer only relays the desired object or content will be a true two-way broadcast.

The present invention has been made in view of this point, and an object of the present invention is to process a broadcast video transmitted from two or more cameras and provide a system that looks as if a relay site is viewed in a three-dimensional virtual space. Accordingly, viewers enjoy broadcasting while rotating, expanding, and contracting the relay site 360 degrees in the vertical direction.

Another object of the present invention is to provide an apparatus and method for transmitting the three-dimensional reconstructed virtual space to the viewer of the viewer.

It is still another object of the present invention to provide a viewer device in which a viewer looks at a desired object or content by moving, expanding, and reducing a desired area in a virtual space reconstructed three-dimensionally.

1 is a schematic diagram of a configuration of an apparatus according to the present invention,

2 is a view showing the configuration of a panoramic or spherical video encoding system of the apparatus according to the present invention;

FIG. 3 is a view illustrating photographing an object with two fisheye lens cameras according to the present invention; FIG.

FIG. 4 is a view incorporating two frames photographed in FIG. 3 into one spherical frame;

5 is a view illustrating photographing an object with four general cameras in the apparatus according to the present invention;

FIG. 6 is a view showing a combination of four frames photographed in FIG. 5 into one panoramic frame;

FIG. 7 is a diagram illustrating transmitting only a portion desired by a viewer among frames integrated into one frame in FIGS. 4 and 6;

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a diagram illustrating a method and an apparatus for vertically, horizontally and vertically 360 degree panoramic or spherical video compression and transmission of the present invention. In FIG. 1, the three-dimensional stereoscopic broadcast apparatus 100 is connected to a panoramic or spherical video encoding system 200 through the Internet, a broadcast transmission network, and a wireless communication 400. The broadcast camera 300 photographs the relay site and transmits the broadcast scene to the panoramic or spherical video encoding system 200 in a general manner.

Broadcasting camera 300 is installed in the field to relay or record the 360-degree panoramic or spherical video of the present invention up and down, encoding the panoramic or spherical video over the wireless network or broadcasting network, or the Internet To the system 200. In this case, two spherical video cameras 300a employing a fisheye lens are used, and two or more general cameras 300b are used for a panoramic video.

The viewer 100 for up, down, left, right, or right 360 ° panoramic or spherical video broadcasting viewing of the present invention is a computer, a digital TV, an IMT-2000, or the like of the viewer to watch the broadcast. The request for changing the degree, etc., is transmitted to the panoramic or spherical video encoding system 200 through the specially designed protocol 400.

Next, referring to FIG. 2, a method of reprocessing a general image transmitted as a panoramic or spherical video to a viewer of the viewer will be described.

First, an image 300 captured at a broadcast relay site or a recording site is transmitted to the panoramic or spherical video encoding system 200.

The transmitted video is transmitted in real time by 24 frames per second, and each of these frames coming from each camera is a panoramic or spherical frame for each frame by the composition server 210 for each of the panoramic or spherical video frames. Will be synthesized.

The synthesized panoramic or spherical frame is compressed by the panoramic or spherical video encoding server 220 into a panoramic or spherical video of 24 frames per second.

The video reconstructed into a panoramic or spherical video is twice as large as a video transmitted through a single camera, and there are many problems in transmitting it to a viewer's viewer in a general manner. .

Since the portion that the viewer actually watches is about the portion of one of the processed panoramic or spherical videos taken by the camera, a server 230 for transmitting only a specific portion to the viewer's viewer 100 is required according to the viewer's request. Done. The server 230 which transmits the video according to the viewer's request transmits the video to the viewer 100 of the viewer.

Next, a method of three-dimensionally reprocessing the planar image photographed by the camera will be described with reference to FIGS. 3 to 6.

First, FIGS. 3 and 4 illustrate a method of generating a spherical image using a fisheye lens camera, and FIGS. 5 and 6 describe a method of generating a panoramic image using a general lens camera.

When the user of the present invention captures an object using a camera using a fisheye lens, a spherical surface of about 185 degrees is recorded for each camera 310a or 310b, and the captured camera image is transmitted 24 frames per second. Or to be recorded (FIG. 3).

Each image photographed by the fisheye lens camera is transmitted to the panoramic or spherical video encoding system 200 of the present invention by 24 frames per second for each camera, and the panoramic or spherical video encoding system 200 The frames coming from the camera are synthesized and reconstructed into a new spherical frame (FIG. 4).

When the user of the present invention shoots an object using a camera using a general lens, each camera (320a, 320b, 320c, 320d) to shoot a plane of the appropriate angle, so the camera image is taken 24 seconds per second It will be transmitted or recorded frame by frame.

Each image captured by the general lens camera is transmitted to the encoding system 200 of the present invention by 24 frames per second for each camera, and the panoramic or spherical video encoding system 200 synthesizes the frames from each camera. To reconstruct the new panoramic frame (FIG. 6).

Next, a description will be given of a method for transmitting a viewer's desired image from a reconstructed panoramic frame or spherical frame using the apparatus and method of the present invention.

Since the frame reconstructed into a panoramic or spherical frame is much larger than a normal frame, it is difficult to transmit such a frame to a viewer as it is. Therefore, a method of selecting and transmitting a screen in the panoramic or spherical video encoding system 200, or preferably transmitting only a screen that appears when the viewer performs a desired screen manipulation such as 360 degree rotation, zoom in and zoom out in the viewer. To solve this problem.

In FIG. 7, the viewer performs a viewer operation such as zooming in, zooming out, rotating 360 degrees up, down, left, and right, and the corresponding screen coordinate information corresponding to the result of the manipulation is displayed on the panoramic or spherical video transmission server 230a or 230b. Sent via a protocol that synchronizes the coordinate system and time. According to the transmitted information, the panoramic or spherical video transmission system transmits the video frame to the viewers through the Internet, IMT-2000, or a general broadcasting network by transmitting a frame corresponding to the viewer 100. will be.

In the above-described embodiment, the viewer 100 has been described for being installed in a computer, a digital TV, an IMT-2000 or the like, but a general TV currently used as the viewer 100 may be used, in which case the viewer Only a specific screen sent from the panoramic or spherical video encoding system 200 can be viewed.

In addition, in the process of synthesizing the transmitted captured image, it is possible to insert an advertisement in a specific portion of a frame to be synthesized or to insert an advertisement in a specific video.

As described above, according to the present invention, it is possible to perform a real-time 360-degree real-time relay in all areas of video transmission, such as Internet broadcasting, digital TV, IMT-2000, so that viewers feel as if they are in the field Will have an effect.

In addition, in military and security services that must always shoot 360 degrees of the top, bottom, left and right of the camera, there is an effect that can easily navigate and shoot in real time.

In addition, more than tens of thousands of people, such as Internet broadcasting and digital TV, can simultaneously access and show exceptional effects in exhibition broadcasts, sports broadcasts, and religious gatherings where they want to see things they want. .

In addition, when providing traffic information to the IMT-2000 using the present invention, it is possible to simultaneously provide various traffic information desired by the user to tens of thousands or more people without moving or rotating the traffic camera.

In addition, even when the existing TV viewer is used, there is an effect that it is possible to watch the image from an angle where the camera is not installed.

Claims (6)

In the method for re-processing the image taken by at least two cameras to a panoramic or spherical video, Photographing the scene with at least two cameras, and transmitting the photographed image to a panoramic or spherical video encoding system; Synthesizing each frame of the captured image into a panoramic or spherical frame using IBR technology in the video encoding system; Compressing the synthesized frame and encoding the video into a video; And A method for compressing and transmitting up, down, left, and right 360 degree panoramic or spherical video, characterized by transmitting only a specific portion of an encoded panoramic or spherical video. The method of claim 1, further comprising, prior to transmitting only the specific portion, the viewer directly selecting the specific portion through a protocol for transmitting a viewer's request to the parametric or spherical video encoding system. How to. The method of claim 1 or 2, wherein the particular portion includes an advertisement. The method of claim 2, wherein the protocol is a protocol for synchronizing time with an image coordinate system to transmit only a desired portion at the request of a viewer. In the device for receiving the images taken by at least two cameras to compress and transmit them to a panoramic or spherical moving image, An image input unit configured to receive an image photographed by at least two cameras; An image synthesizer for synthesizing the photographed image transmitted from the image input unit into a panoramic or spherical image for each frame using an IBR technique; A panoramic or spherical video encoding server that compresses the frame synthesized by the image synthesis unit and encodes the video into a video; And And a panoramic or spherical video transmission unit for transmitting only a specific portion of the encoded panoramic or old video. 6. The apparatus of claim 5, wherein the panoramic or spherical video transmitting unit receives the request of the viewer through a protocol prior to the transmission and transmits the specific portion requested.