CN102692806B - Collection and the formation method of free view-point four-dimentional space video sequence - Google Patents

Abstract

A kind of collection and formation method of free view-point four-dimentional space video sequence, belong to technology of video compressing encoding field.The N number of camera gathered by being arranged in frame realizes, and it is the circle of r that camera is arranged as a radius, and the angle between adjacent two cameras is θ; Subject forms circle that a radius is R and is concentric circles with the circle that camera is formed; Camera to subject shooting, obtains the flat two-dimensional images frame with free view-point feature simultaneously; Formation method is, through the mapping of plane picture to cylindrical picture, increases radian dimension information, and then is spliced by multiple images be connected according to the lap of image, form 360 degree of Cylindrical panoramic images; Cylinder panorama video sequence carrys out the depth information of prognostic chart picture frame by disparity computation and depth perception principle parallax, thus spatially adds depth information, final formation free view-point four-dimentional space video sequence.

Description

Collection and the formation method of free view-point four-dimentional space video sequence

Technical field

The present invention relates to a kind of collection and formation method of free view-point four-dimentional space video sequence, belong to technology of video compressing encoding field.

Background technology

Along with the fast development of infotech, various forms of three-dimensional video-frequency process and application have poured in the life of people, as panorama Circamara, flat and stereo TV and film, three-dimensional medical image imaging and transmission, robot controlling three-dimensional imaging, virtual reality system etc.Meanwhile, three-dimensional video-frequency Processing Algorithm and systematic study have become one of study hotspot with fastest developing speed in the world in recent years.

At present, both at home and abroad to the research of free view-point three-dimensional video-frequency from space is formed nothing more than two kinds of forms: cylinder/sphere two-dimensional video and flat and stereo video.Flat and stereo video refers to the plane three-dimensional video sequence with depth information, is made up of, is expressed as (x, y, v) multiple video sequences with certain parallax.It utilizes the motion of multiple video sequence, parallax and depth prediction and enhancing algorithm, within the scope of plane angle, carry out three-dimensional video-frequency process.Cylinder/sphere two-dimensional video is then be treated to basis with planar video, carries out the two-dimensional video process of cylinder/sphere thus reaches full-view stereo effect.Wherein, cylinder two-dimensional video is 360 degree of panoramic videos that planar video sequence obtains after the expansion of cylinder radian dimension space, is expressed as (x, y, α).

Flat and stereo video is the space three-dimensional structure video with depth information based on plane, and cylinder two-dimensional video is the space three-dimensional structure video with radian information based on cylinder.Although both can both realize free view-point Stereoscopic Video Presentation to a certain degree, there is deficiency.Flat and stereo video does not have radian information, cannot show 360 degree of omnibearing stereo videos, and cylinder two-dimensional video does not have depth information, cannot show the 3 D video based on cylinder.Therefore free view-point four-dimentional space video concept has just been had.The two-dimensional video in cylinder space combines with the 3 D video of plane space by free view-point four-dimentional space video, and common composition has the free view-point four-dimentional space video system of space quaternary structure.This system has four dimensional data structures in spatial dimension, is expressed as the four-dimentional space structure (x, y, v, α) of planimetric coordinates (x, y), depth information dimension v and radian Information Dimension α.Radian dimension carries out cylinder expansion to flat and stereo video, and depth dimension carries out Depth Expansion to cylinder two-dimensional video, spatially constitutes and both has depth information, has again the video sequence of radian information.It inherits the comprehensive characteristic in cylinder space and the 3D vision characteristic of plane angle simultaneously, Stereoscopic Video Presentation can be carried out in cylinder space, compensate for the spatial defects of flat and stereo and cylinder two-dimensional video, from truly reappearing and showing 360 degree of omnibearing stereo scenes real world.

Due to the expansion that free view-point four-dimentional space video processing technique is to existing flat and stereo and cylinder two-dimensional video treatment technology, be intended to the real-time process realizing three-dimensional video-frequency at any angle in cylinder 360 degree of panorama environment, this is the newer technical field that current flat and stereo video and cylinder two-dimensional video system cannot match in excellence or beauty.A lot of research is all the Design spreading of algorithm around free view-point four-dimentional space Video processing and system in the world.Accordingly, this is just for video acquisition proposes new requirement, and the card namely how tested for algorithm research and the system of free view-point four-dimentional space Video processing provides required test code streams.At present, in the world cylinder two-dimensional video and flat and stereo video are both provided to official's test code streams of standard, but because the theoretical research of free view-point four-dimentional space video does not also launch on a large scale, so associated video cleanup standard official organization does not also have corresponding video test stream to provide.

Summary of the invention

For the deficiencies in the prior art, the present invention proposes a kind of collection and formation method of free view-point four-dimentional space video sequence.

The acquisition method of free view-point four-dimentional space video sequence, realized by the N number of camera be arranged in frame, it is the circle of r that N number of camera is arranged as a radius, and the angle between adjacent two cameras is identical, is; Subject forms circle that a radius is R and is concentric circles with the circle that camera is formed; N number of camera to subject shooting, obtains N number of flat two-dimensional images frame with free view-point feature simultaneously.

The formation method of free view-point four-dimentional space video sequence, camera collection to picture frame be the flat two-dimensional images frame with free view-point feature, to generate the video sequence of free view-point four-dimentional space, then must first through the mapping of plane picture to cylindrical picture, increase radian dimension information, and then according to the lap of image, multiple images be connected are spliced, form 360 degree of Cylindrical panoramic images; Cylinder panorama video sequence carrys out the depth information of prognostic chart picture frame by disparity computation and depth perception principle parallax, thus spatially adds depth information, final formation free view-point four-dimentional space video sequence.

The selection of radius r, angle theta and radius R is formed with following relation to free view-point four-dimentional space video sequence: the mapping relations from plane picture pixel A (x, y) to cylindrical picture pixel A ' (x ', y ') are:

$x^{\′}=(R-r\cos \frac{\mathrm{\θ}}{2}),{\tan }^{-1}\frac{x}{(R-r\cos \frac{\mathrm{\θ}}{2})},y^{\′}=y\cos \left({\tan }^{-1}\frac{x}{(R-r\cos \frac{\mathrm{\θ}}{2})}\right)$

With the pass of parallax p and depth coefficient v be:

$v=\frac{(R-r\cos \frac{\mathrm{\θ}}{2})p}{(p-2r\sin \frac{\mathrm{\θ}}{2})}.$

Fig. 1 and Fig. 2 is that in the present invention, free view-point four-dimentional space video sequence acquisition system top view and vertical view can carry out multiple views sampling to 360 degree of panoramas by this acquisition system simultaneously respectively.The image collected, after plane to the mapping of cylinder, adds radian dimension information, then carries out video-splicing according to overlapping region, thus define cylinder two-dimensional panoramic video sequence.Cylinder two-dimensional panoramic video sequence carrys out the depth information of prognostic chart picture frame by disparity computation and depth perception principle parallax, spatially adds depth information, final formation free view-point four-dimentional space video sequence.

The invention has the beneficial effects as follows:

As the expansion on space structure of cylinder two-dimensional video and plane three-dimensional video-frequency and enhancing, four-dimentional space Video processing can provide unprecedented stereoscopic vision to experience, its electronic product obtains development at full speed by function, performance, real-time, recreational, dirigibility, convenience, reliability etc., apply ubiquitous, the output value is limitless.Carry out the research of the algorithm of free view-point four-dimentional space Video processing and the design of system, the cycle tests accordingly with free view-point four-dimentional space feature must be needed to carry out the checking of algorithm and system.Because Video processing standardization body does not also launch broad scale research, so do not have corresponding test code streams to free view-point four-dimentional space Video processing in the world.The present invention proposes a kind of special Video Image Collecting System Based, can carry out multiple views sampling simultaneously, thus obtain multiple views omni-directional video acquisition sequence by this system to 360 degree of panoramas.This is just follow-up providing the foundation to the algorithm of free view-point four-dimentional space Video processing and the checking of system.

Accompanying drawing explanation

Fig. 1 is free view-point four-dimentional space video sequence acquisition system top view in the present invention.

Fig. 2 is free view-point four-dimentional space video sequence acquisition system side view in the present invention.

In figure, 1-1-(N) is N number of camera, the concentrically ringed radius r that 2-1, camera form, angle theta between 2-2, adjacent camera, the concentrically ringed radius R that in 2-3, shoot on location, shot object forms, 3, blind area, 4-1-(N) is the visual angle of corresponding 1-1-(N) camera, 5, overlay region

Embodiment

Below in conjunction with drawings and Examples, the present invention is further described.

Embodiment:

A kind of collection and formation method of free view-point four-dimentional space video sequence, as shown in Figure 1 to Figure 2.

The acquisition method of free view-point four-dimentional space video sequence, realized by the N number of camera be arranged in frame, it is the circle of r that N number of camera is arranged as a radius, and the angle between adjacent two cameras is identical, is; Subject forms circle that a radius is R and is concentric circles with the circle that camera is formed; N number of camera to subject shooting, obtains N number of flat two-dimensional images frame with free view-point feature simultaneously.

The formation method of free view-point four-dimentional space video sequence, camera collection to picture frame be the flat two-dimensional images frame with free view-point feature, to generate the video sequence of free view-point four-dimentional space, then must first through the mapping of plane picture to cylindrical picture, increase radian dimension information, and then according to the lap of image, multiple images be connected are spliced, form 360 degree of Cylindrical panoramic images; Cylinder panorama video sequence carrys out the depth information of prognostic chart picture frame by disparity computation and depth perception principle parallax, thus spatially adds depth information, final formation free view-point four-dimentional space video sequence.

The selection of radius r, angle theta and radius R is formed with following relation to free view-point four-dimentional space video sequence: the mapping relations from plane picture pixel A (x, y) to cylindrical picture pixel A ' (x ', y ') are:

$x^{\′}=(R-r\cos \frac{\mathrm{\θ}}{2})\·{\tan }^{-1}\frac{x}{(R-r\cos \frac{\mathrm{\θ}}{2})},y^{\′}=y\cos \left({\tan }^{-1}\frac{x}{(R-r\cos \frac{\mathrm{\θ}}{2})}\right)$

With the pass of parallax p and depth coefficient v be:

$v=\frac{(R-r\cos \frac{\mathrm{\θ}}{2})p}{(p-2r\sin \frac{\mathrm{\θ}}{2})}.$

Claims (2)

1. collection and the method for being formationed of a free view-point four-dimentional space video sequence, is characterized in that, by the N number of camera realization be arranged in frame, it is the circle of r that N number of camera is arranged as a radius, and the angle between adjacent two cameras is identical, is θ; Subject forms circle that a radius is R and is concentric circles with the circle that camera is formed; The circle that subject surrounds is cylindrical, and the circle that camera is formed is inner circle; N number of camera to subject shooting, obtains N number of flat two-dimensional images frame with free view-point feature simultaneously; Camera collection to picture frame be the flat two-dimensional images frame with free view-point feature, to generate free view-point four-dimentional space video sequence, then must first through the mapping of flat two-dimensional images frame to cylindrical picture frame, increase radian dimension information, and then splice according to the cylindrical picture frame that multiple are connected by the lap of cylindrical picture frame, form cylinder panorama video sequence; Cylinder panorama video sequence predicts the depth information of cylinder panorama video sequence by disparity computation and depth perception principle parallax, thus spatially adds depth dimension information, final formation free view-point four-dimentional space video sequence; Described free view-point four-dimentional space video sequence has the four-dimentional space structure (x, y, v, α) being expressed as flat two-dimensional images frame (x, y), depth dimension information v and radian dimension information α in spatial dimension.

2. collection and the formation method of a kind of free view-point four-dimentional space video sequence according to claim 1, it is characterized in that, the selection of the radius R of the radius r of described inner circle, angle theta and described cylindrical is formed with following relation to free view-point four-dimentional space video sequence: from flat two-dimensional images frame (x, y) mapping relations to cylindrical picture frame (x ', y ') are:

$x^{\′}=(R-r\cos \frac{\mathrm{\θ}}{2})\·{\tan }^{-1}\frac{x}{(R-r\cos \frac{\mathrm{\θ}}{2})},y^{\′}=y\cos \left({\tan }^{-1}\frac{x}{(R-r\cos \frac{\mathrm{\θ}}{2})}\right)$

The mapping relations of parallax p and depth dimension information v are:

$v=\frac{(R-r\cos \frac{\mathrm{\θ}}{2})p}{(p-2r\sin \frac{\mathrm{\θ}}{2})}.$