KR20190053565A - Live stitching method using stitching device - Google Patents

Abstract

The present invention relates to a live broadcasting stitching method using a stitching device, comprising the following steps: receiving, by the stitching device, a plurality of captured images from a plurality of cameras in real time, and grouping the received captured images by classifying the captured images into a left eye image and a right eye image; extracting a plurality of feature points with respect to each of the grouped captured images; comparing the plurality of feature points extracted from the plurality of captured images in the group to match corresponding feature points; extracting a homography formula using the matched feature points; and performing a stitching process to generate one 360 degree 3D stereoscopic image for each group using the extracted homography formula and the feature points. According to the present invention, small 3D virtual reality content and augmented reality content with improved stereoscopic effect can be generated and provided, and high-quality 360 degree 3D stereoscopic images without distortion can be provided. In addition, the feature points and the homography formula are extracted and then correction is performed in units of frames to be stitched, and thus the speed of stitching can be improved.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a stitching method using a stitching apparatus,

The present invention relates to a live stitching method using a stitching apparatus, and more particularly, to a live stitching method using a stitching apparatus for stitching a 360-degree 3D stereoscopic image using a plurality of shot images received from a plurality of cameras will be.

Virtual Reality (VR) is a generic term used to refer to a technique that allows a user to interact as if it were a real reality in a virtual space created by a computer. In general, a HMD (Head Mount Display) device with two convex lenses is used This type of virtual reality experience represents VR because it sees the image.

As part of a method to maximize the characteristics of HMD images with immersive feeling, I recently shot a 360 degree camera using multiple cameras and attached the images together to give an astonishing experience of first person as if the viewer is in the shooting scene A new technology is being developed, specifically called '360 VR'. 360 degree VR shows the 360 degree image of the first person through the HMD, so it can be used as a new image content method because it gives a vivid sense of presence as if it is there without going to the shooting place.

In other words, although virtual reality has been mainly dominated in the game field up to now, research on VR has been increasing in the field of image as 360 degree VR image attracts people's attention. Particularly, the interest of VR live broadcasting is increasing because the characteristics of VR immersion and presence can be maximized in live broadcast.

In order to live 360 degree VR video, 360 degree video of several times of general video is wirelessly transmitted along with the operation of stitching while shooting with a camera. Therefore, it is necessary to develop efficient stitching method and high-performance computing and transmission Speed-related technology development is required.

The technology which is the background of the present invention is disclosed in Korean Patent No. 10-0614004 (published on Aug. 21, 2006).

SUMMARY OF THE INVENTION The present invention provides a live stitching method using a stitching apparatus for stitching a 360-degree 3D stereoscopic image using a plurality of photographed images received from a plurality of cameras.

According to an aspect of the present invention, there is provided a live stitching method using a stitching apparatus, wherein the stitching apparatus receives a plurality of photographed images from a plurality of cameras in real time, A plurality of feature points extracted from a plurality of photographed images in the group, and comparing the plurality of feature points extracted from the plurality of photographed images in the group, A step of extracting a homography formula using the matched feature points, and a step of generating a 360-degree 3D stereoscopic image for each group using the extracted homography formula and the feature points, .

The step of stitching may include combining the common parts of the plurality of photographed images through equirectangular projection in a fish-eye lens by combining the 360-degree 3D stereoscopic left-eye image and the one 360-degree 3D stereoscopic left- A stereoscopic right eye image can be generated.

When a plurality of photographed images are received in real time after the feature points and the homography formula are extracted, only the common partial frames are corrected in a plurality of photographed images received using the extracted feature points and a homography formula And generating 360-degree 3D stereoscopic images for the left eye image and the right eye image, respectively.

The 360-degree 3D stereoscopic image may further include a step of adjusting resolution by changing an image size using a bicubic or bilinear algorithm.

According to the present invention, small 3D virtual reality contents and augmented reality contents with improved stereoscopic effect can be generated and provided, and high-quality 360-degree 3D stereoscopic images without distortion can be provided.

In addition, since the feature points and the homography formula are extracted and then corrected in units of frames to be stitched, the speed of stitching can be improved.

1 is an exemplary view for explaining a live stitching method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as " comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention.

Hereinafter, a method of performing live stitching using a stitching apparatus according to an embodiment of the present invention will be described in detail with reference to FIG.

1 is an exemplary view for explaining a live stitching method according to an embodiment of the present invention.

First, the stitching apparatus receives a plurality of photographed images from a plurality of cameras in real time. For example, the stitching device can receive images from at least eight cameras. Each camera is connected to a respective Capture Card, and each of the Capture Card Filters ) Can be connected to a sample grabber filter to transmit 8 images at the same time to the stitching device without delay.

The stitching apparatus divides the received plurality of photographed images into a left eye image and a right eye image and groups them. In other words, the image corresponding to the left eye and the image corresponding to the right eye can be grouped into two groups.

Next, the stinging apparatus extracts a plurality of feature points for each of the plurality of grouped photographed images.

As shown in FIG. 1 (a), the stinging apparatus can extract a plurality of feature points from each photographed image.

As shown in FIG. 1B, the stitching apparatus compares a plurality of minutiae extracted by the photographed image for each group, and matches the minutiae matching each other.

Next, the stitching apparatus extracts the homography from the matched feature points. That is, the stitching apparatus calculates a homography equation, which is a geometric association between adjacent images, by matching the same feature points.

Then, the stitching apparatus synthesizes the common parts of the plurality of photographed images through Equirectangular Projection in the fisheye lens using the matched feature points and the homography formula.

As shown in FIG. 1 (c), the stitching apparatus stitches a common portion of two images so that one image can be generated by matching identical feature points in two images.

That is, the stitching apparatus generates a 360-degree 3D stereoscopic left-eye image and a 360-degree 3D stereoscopic right-eye image using the extracted homography and the extracted feature points.

Meanwhile, when the feature points and the homography formula are extracted in the above process and then a plurality of photographed images are received in real time, the stitching device extracts a plurality of received photographed images using the extracted feature points and the homography formula Dimensional stereoscopic image can be generated for each of the left eye image and the right eye image by correcting only the common partial frame.

That is, if the distance between the plurality of cameras is unchanged, it is possible to obtain all the same. Therefore, it is possible to quickly generate a 360-degree 3D stereoscopic image by correcting only the common partial frame through the step of stitching in the received image.

When the stitch device generates a 360-degree 3D stereoscopic image, the resolution can be adjusted by changing the image size using bicubic or bilinear algorithm.

On the other hand, audio can be input from one capture card (3840x2160 360 images), and each 3840x2160 image is made up of 4 3840x2160 360 images through the stitching device. You can create one 3840x2160 image at the bottom. The resulting 3840x2160 top-bottom image can be compressed by the H.264 codec and matched with audio.

As described above, according to the embodiment of the present invention, it is possible to generate and provide small 3D virtual reality contents and augmented reality contents with improved stereoscopic effect, and to provide high quality 360 degree 3D stereoscopic images without distortion.

In addition, since the feature points and the homography formula are extracted and then corrected in units of frames to be stitched, the speed of stitching can be improved.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (4)

In a live stitching method using a stitching apparatus,
The stitching apparatus includes a step of receiving a plurality of photographed images from a plurality of cameras in real time, dividing the received photographed images into a left eye image and a right eye image,
Extracting a plurality of feature points for each of the plurality of grouped shot images,
Comparing the plurality of feature points extracted from the plurality of photographed images in the group and matching the matching feature points,
A homography extraction using the matched minutiae points, and
And stitching to generate one 360 degree 3D stereoscopic image for each group using the extracted homography formula and the minutiae points. The method according to claim 1,
Wherein said stitching comprises:
And a 360 ° 3D stereoscopic left eye image and a 360 ° 3D stereoscopic right eye image are generated by combining common portions of the plurality of photographed images through Equirectangular Projection in a fisheye lens of the photographed images of the respective groups Live stitching method. 3. The method of claim 2,
When a plurality of photographed images are received in real time after the feature points and the homography formula are extracted, only the common partial frames are corrected in a plurality of photographed images received using the extracted feature points and a homography formula And generating 360-degree 3D stereoscopic images for the left eye image and the right eye image, respectively. The method of claim 3,
And generating the 360-degree 3D stereoscopic image, further comprising adjusting resolution by changing an image size using a bicubic or bilinear algorithm.

Images