CN106954060B - A kind of three-dimensional video-frequency generation method and auxiliary filming apparatus towards smart phone - Google Patents
A kind of three-dimensional video-frequency generation method and auxiliary filming apparatus towards smart phone Download PDFInfo
- Publication number
- CN106954060B CN106954060B CN201710333255.3A CN201710333255A CN106954060B CN 106954060 B CN106954060 B CN 106954060B CN 201710333255 A CN201710333255 A CN 201710333255A CN 106954060 B CN106954060 B CN 106954060B
- Authority
- CN
- China
- Prior art keywords
- smart phone
- plane mirror
- frequency
- dimensional video
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/246—Calibration of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
- H04M1/72439—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for image or video messaging
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
Abstract
The invention discloses a kind of three-dimensional video-frequency generation method and auxiliary filming apparatus towards smart phone.It includes five steps that three-dimensional video-frequency of the present invention, which generates scheme: three-dimensional video-frequency assists filming apparatus building and installation；Three-dimensional video-frequency assists filming apparatus parameter automatic Calibration；Perspective geometry transformation matrix calculates and the estimation of grey level compensation mapping function；Video data capture；Three-dimensional video-frequency generates.It includes: stainless steel plane mirror, plane mirror fixed frame, bearing attachment part, bracket holder, mobile phone connecting component, fastener that three-dimensional video-frequency, which assists filming apparatus,.The method of the present invention can avoid the asynchronous caused left and right picture problem of disharmony of two cameras, and three-dimensional video-frequency of the present invention auxiliary filming apparatus have many advantages, such as it is of simple structure and low cost, easy to carry and easy to use, facilitate ordinary consumer and obtain the stronger three-dimensional video-frequency of feeling of immersion using smart phone, enriches smart phone visual form, promotes shooting and ornamental enjoyment.
Description
Technical field
The present invention relates to three-dimensional video-frequency imaging fields, particularly relate to a kind of three-dimensional video-frequency generation method towards smart phone
And auxiliary filming apparatus.
Background technique
Vision is the important information source of human perception external environment, therefore image and video are in amusement, education, social activity etc.
It plays a significant role in various application scenarios.However, normal image and video are substantially the two-dimensional projections to three-dimensional world, at
It is lost depth information of scene as during, therefore normal image and video are difficult to bring feeling on the spot in person to observer.
In recent years, three-dimensional (stereoscopy) video imaging technique is rapidly developed, and significantly enhances the three-dimensional sense of scene and true
True feeling.The left and right view with parallax can be presented in three-dimensional video-frequency to observer's images of left and right eyes, the two views pass through observer
Complete scene depth clue can be provided for observer after brain synthesis.Currently, people can be by virtual reality glasses
Ancillary equipments such as (virtual reality glasses) watch the three-dimensional video-frequency recorded in advance on smart phone, for
Unprecedented feeling of immersion and splendid video display is brought to appreciate experience in family.
Existing stereo-picture and stereo video acquisition technology can substantially be divided into following four classes: using computer vision and
Image processing method extracts the depth information of scene from common single camera video, and generate has parallax on this basis
Left and right view；Side by side and the fixed video camera in relative position is shot simultaneously using two；Using bicamera into
Row stereo video shooting；It is shot using single camera combination auxiliary device.These technological means be individually present it is at high cost,
The problems such as structure is complicated, operating difficulties is unable to satisfy ordinary consumer using the demand of common smart mobile phone shooting three-dimensional video-frequency.
Stereo video shooting in the prior art is at high cost, structure is complicated, operating difficulties aiming at the problem that, there has been no have at present
The solution of effect.
Summary of the invention
In view of this, the purpose of the embodiment of the present invention is to propose a kind of three-dimensional video-frequency generation method towards smart phone
And auxiliary filming apparatus, ordinary consumer can be made to shoot three-dimensional video-frequency using common smart mobile phone, cost is reduced, simplify knot
Structure and convenient for operation.
Based on above-mentioned purpose, the three-dimensional video-frequency generation method towards smart phone that the embodiment of the invention provides a kind of, packet
It includes:
Building three-dimensional video-frequency auxiliary filming apparatus is simultaneously installed on smart phone；
Demarcate the installation parameter of three-dimensional video-frequency auxiliary filming apparatus；
Perspective geometry transformation matrix and grey level compensation mapping function are calculated using smart phone；
Video data is shot using smart phone；
Three-dimensional video-frequency is converted by the video data of shooting.
In some embodiments, the three-dimensional video-frequency auxiliary filming apparatus includes the stainless steel for obtaining the scene virtual image
Plane mirror, plane and the smart phone back side are perpendicular where stainless steel plane mirror, plane where stainless steel plane mirror and intelligent hand
The machine back side intersects at stainless steel plane mirror inward flange, and stainless steel plane mirror inward flange and smart phone side are perpendicular；Meanwhile no
Rust steel plane mirror is fixed by plane mirror fixed frame, and plane mirror fixed frame is connected to bracket holder by bearing attachment part, branch
Frame fixed frame is connected by mobile phone connecting component with smart phone, between plane mirror fixed frame and bearing attachment part, bracket
Relative position and angle are fixed or adjusted between fixed frame and mobile phone connecting component with fastener.
In some embodiments, the installation parameter of the calibration three-dimensional video-frequency auxiliary filming apparatus includes:
Image is shot using smart phone, wherein the corresponding photographed scene of the image includes more rich texture, color letter
Breath, and cannot be the single scene of the colors such as metope, desktop；
Multiple sparse features points are extracted from the image and carry out Feature Points Matching, generate Feature Points Matching set；
Multiplicating randomly selects two pairs of characteristic points and matches, and generates the determining plane mirror normal direction of features described above point matching
Amount estimation；
Determine plane mirror normal vector as installation parameter according to the intermediate value of each component of multiple normal estimations.
In some embodiments, described to extract multiple sparse features points from the image and carry out Feature Points Matching, it is raw
Include: at Feature Points Matching set
Algorithm is detected and described according to image local feature, and multiple sparse features points and each sparse are extracted from the image
The description vectors of characteristic point；
Other characteristic points in each sparse features point and the image are carried out than test feature matching process according to distance
Matching generates matching characteristic point pair；
It include the Feature Points Matching set of at least 100 characteristic points pair to foundation according to multiple sparse features points.
In some embodiments, the multiplicating randomly selects two pairs of characteristic points and matches, and generates features described above
Point matches determining plane mirror normal vector
Determine that randomly selecting two pairs of characteristic points carries out matched number according to the quantity of characteristic point pair；
It randomly selects two pairs of characteristic points every time to match, and is taken the photograph according to the image coordinate and smart phone of two pairs of characteristic points
As the focal length of head generates the determining normal vector of features described above point matching.
In some embodiments, described to calculate perspective geometry transformation matrix and grey level compensation mapping letter using smart phone
Number includes:
Image transformation matrix is generated according to the normal vector of the length of video frame, width and plane mirror；
The consistency of all characteristic points pair in Feature Points Matching set is obtained according to the focal length of smart phone camera
Index；
It collects coincident indicator and is greater than all characteristic points pair for preassigning threshold value, establish validity feature point pairing set
It closes；
The matching of validity feature point is obtained to the image grayscale of characteristic point position each in set, and according to cubic polynomial song
The image grayscale of line fitting algorithm and each characteristic point position is fitted grey level compensation mapping function.
In some embodiments, when the shooting video data using smart phone, mobile phone transverse sides and horizontal plane
Angle is no more than 30 degree.
In some embodiments, described to convert three-dimensional video-frequency for the video data of shooting and include:
Video data is decomposed by frame, and geometric correction is carried out using image transformation matrix to every frame image and cuts sky
White region；
It is same size left view and right view by every frame image segmentation, and left and right mirror surface transformation is carried out to right view；
Color compensating is carried out to each pixel in right view according to grey level compensation mapping；
According to each frame left view of stereoscopic video observation mode adaptive processes and right view, and by each frame left view with
Right view merges into stereo video frame, then all stereo video frames are merged into three-dimensional video-frequency.
In some embodiments, the stereoscopic video observation mode includes at least one of: VR glasses, red blue eye
Mirror, naked eye；It is described according to each frame left view of stereoscopic video observation mode adaptive processes and right view, for according to three-dimensional video-frequency
The specified video resolution of view mode and aspect ratio carry out adaptive processes to each frame left view and right view.
Based on above-mentioned purpose, the embodiment of the invention provides a kind of, and the three-dimensional video-frequency auxiliary shooting towards smart phone is filled
It sets, including the stainless steel plane mirror for obtaining the scene virtual image, plane where stainless steel plane mirror is mutually hung down with the smart phone back side
Directly, plane where stainless steel plane mirror and the smart phone back side intersect at stainless steel plane mirror inward flange, in stainless steel plane mirror
Edge and smart phone side are perpendicular；Meanwhile stainless steel plane mirror is fixed by plane mirror fixed frame, plane mirror fixed frame passes through
Bearing attachment part is connected to bracket holder, and bracket holder is connected by mobile phone connecting component with smart phone, plane
It fixes or adjusts between mirror fixed frame and bearing attachment part, between bracket holder and mobile phone connecting component with fastener and is opposite
Position and angle.
From the above it can be seen that the three-dimensional video-frequency generation method provided in an embodiment of the present invention towards smart phone is only
Need a camera that can shoot three-dimensional video-frequency, thus can avoid two cameras it is asynchronous caused by left and right picture is uncoordinated asks
Topic, and three-dimensional video-frequency of the present invention auxiliary filming apparatus is with of simple structure and low cost, easy to carry and easy to use etc. excellent
Point.Therefore, the method for the present invention and device can help ordinary consumer to obtain the stronger stereopsis of feeling of immersion using smart phone
Frequently, it enriches smart phone visual form, promote shooting and ornamental enjoyment.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the flow chart of the three-dimensional video-frequency generation method provided by the invention towards smart phone；
Fig. 2 is that the three-dimensional video-frequency generation method neutrality volumetric video provided by the invention towards smart phone assists filming apparatus
Structure chart；
Fig. 3 is the imaging schematic diagram of the three-dimensional video-frequency generation method provided by the invention towards smart phone；
Fig. 4 is plane mirror in one embodiment of the three-dimensional video-frequency generation method provided by the invention towards smart phone
The estimation distribution map of normal vector；
Fig. 5 is in one embodiment of the three-dimensional video-frequency generation method provided by the invention towards smart phone, and gray scale is mended
Repay the fitting result figure of function；
Fig. 6 is to install additional vertical in one embodiment of the three-dimensional video-frequency generation method provided by the invention towards smart phone
Volumetric video assists the video frame and Geometry rectification result figure of the smart phone shooting of filming apparatus；
Fig. 7 is in one embodiment of the three-dimensional video-frequency generation method provided by the invention towards smart phone, and geometry is rectified
The right view after the left view of acquisition, the right view and color compensating without color compensating is cut in positive video；
Fig. 8 is to be suitble to naked in one embodiment of the three-dimensional video-frequency generation method provided by the invention towards smart phone
The Cross-eye three-dimensional video-frequency of eye observation and the suitable left and right three-dimensional video-frequency screenshot using the viewing of VR glasses.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, and reference
The embodiment of the present invention is further described in attached drawing.
It should be noted that all statements for using " first " and " second " are for differentiation two in the embodiment of the present invention
The non-equal entity of a same names or non-equal parameter, it is seen that " first " " second " only for the convenience of statement, does not answer
It is interpreted as the restriction to the embodiment of the present invention, subsequent embodiment no longer illustrates this one by one.
In one embodiment of the invention, shown in fig. 1 is the three-dimensional video-frequency provided by the invention towards smart phone
The flow diagram of one embodiment of generation method.
As shown in Figure 1, the three-dimensional video-frequency generation method towards smart phone includes:
Step S101, building three-dimensional video-frequency auxiliary filming apparatus are simultaneously installed on smart phone；
Step S103, the installation parameter of calibration three-dimensional video-frequency auxiliary filming apparatus；
Step S105 calculates perspective geometry transformation matrix and grey level compensation mapping function using smart phone；
Step S107 shoots video data using smart phone；
The video data of shooting is converted three-dimensional video-frequency by step S109.
In some embodiments, the three-dimensional video-frequency auxiliary filming apparatus includes the stainless steel for obtaining the scene virtual image
Plane mirror 1,1 place plane of stainless steel plane mirror and 0 back side of smart phone are perpendicular, 1 place plane of stainless steel plane mirror and intelligence
0 back side of energy mobile phone intersects at 1 inward flange of stainless steel plane mirror, and 1 inward flange of stainless steel plane mirror mutually hangs down with 0 side of smart phone
Directly；Meanwhile stainless steel plane mirror 1 is fixed by plane mirror fixed frame 2, plane mirror fixed frame 2 is connected to by bearing attachment part 3
Bracket holder 4, bracket holder 4 are connected by mobile phone connecting component 5 with smart phone 0, plane mirror fixed frame 2 and bracket
Between connecting component 3, relative position and angle are fixed or adjusted between bracket holder 4 and mobile phone connecting component 5 with fastener 6
Degree.
In some embodiments, the installation parameter of the calibration three-dimensional video-frequency auxiliary filming apparatus includes:
Image is shot using smart phone, wherein the corresponding photographed scene of the image includes more rich texture, color letter
Breath, and cannot be the single scene of the colors such as metope, desktop；
Multiple sparse features points are extracted from the image and carry out Feature Points Matching, generate Feature Points Matching set；
Multiplicating randomly selects two pairs of Feature Points Matchings, generates the determining plane mirror normal vector of features described above point matching；
Determine plane mirror normal vector as installation parameter according to the intermediate value of multiple each components of Feature Points Matching normal estimation.
In some embodiments, described to extract multiple sparse features points from the image and carry out Feature Points Matching, it is raw
Include: at Feature Points Matching set
Algorithm is detected and described according to image local feature, and multiple sparse features points and each sparse are extracted from the image
The description vectors of characteristic point；
Other characteristic points in each sparse features point and the image are carried out than test feature matching process according to distance
Matching generates matching characteristic point pair；
It include the Feature Points Matching set of at least 100 characteristic points pair to foundation according to multiple sparse features points.
In some embodiments, the multiplicating randomly selects two pairs of characteristic points and matches, and generates features described above
Point matches determining plane mirror normal vector
Determine that randomly selecting two pairs of characteristic points carries out matched number according to the quantity of characteristic point pair；
It randomly selects two pairs of characteristic points every time to match, and is taken the photograph according to the image coordinate and smart phone of two pairs of characteristic points
As the focal length of head generates the determining normal vector of features described above point matching.
In some embodiments, described to calculate perspective geometry transformation matrix and grey level compensation mapping letter using smart phone
Number includes:
Image transformation matrix is generated according to the normal vector of the length of video frame, width and plane mirror；
The consistency of all characteristic points pair in Feature Points Matching set is obtained according to the focal length of smart phone camera
Index；
It collects coincident indicator and is greater than all characteristic points pair for preassigning threshold value, establish validity feature point pairing set
It closes；
The matching of validity feature point is obtained to the image grayscale of characteristic point position each in set, and according to cubic polynomial song
The image grayscale of line fitting algorithm and each characteristic point position is fitted grey level compensation mapping function.
In some embodiments, when the shooting video data using smart phone, mobile phone transverse sides and horizontal plane
Angle is no more than 30 degree.
In some embodiments, described to convert three-dimensional video-frequency for the video data of shooting and include:
Video data is decomposed by frame, and geometric correction is carried out using image transformation matrix to every frame image and cuts sky
White region；
It is same size left view and right view by every frame image segmentation, and left and right mirror surface transformation is carried out to right view；
Color compensating is carried out to each pixel in right view according to grey level compensation mapping；
According to each frame left view of stereoscopic video observation mode adaptive processes and right view, and by each frame left view with
Right view merges into stereo video frame, then all stereo video frames are merged into three-dimensional video-frequency.
In some embodiments, the stereoscopic video observation mode includes at least one of: VR glasses, red blue eye
Mirror, naked eye；It is described according to each frame left view of stereoscopic video observation mode adaptive processes and right view, for according to three-dimensional video-frequency
The specified video resolution of view mode and aspect ratio carry out adaptive processes to each frame left view and right view.
From the above it can be seen that the three-dimensional video-frequency generation method provided in an embodiment of the present invention towards smart phone is only
Need a camera that can shoot three-dimensional video-frequency, thus can avoid two cameras it is asynchronous caused by left and right picture is uncoordinated asks
Topic, and three-dimensional video-frequency of the present invention auxiliary filming apparatus is with of simple structure and low cost, easy to carry and easy to use etc. excellent
Point.Therefore, the method for the present invention and device can help ordinary consumer to obtain the stronger stereopsis of feeling of immersion using smart phone
Frequently, it enriches smart phone visual form, promote shooting and ornamental enjoyment.
In another embodiment of the present invention, the three-dimensional video-frequency generation method towards smart phone includes:
Step S101, building three-dimensional video-frequency auxiliary filming apparatus are simultaneously installed on smart phone.
Three-dimensional video-frequency auxiliary filming apparatus structure chart of the embodiment of the present invention towards smart phone is as shown in Figure 2, comprising: no
Become rusty steel plane mirror 1, plane mirror fixed frame 2, bearing attachment part 3, bracket holder 4, mobile phone connecting component 5, fastener 6.Its
In, stainless steel plane mirror 1 is used to obtain the virtual image of scene；Plane mirror fixed frame 2 is for fixing stainless steel plane mirror 1；Bracket connects
Relay part 3 is for connecting plane mirror fixed frame 2 and bracket holder 4；Bracket holder 4 is for fixing bearing attachment part 3；Hand
Machine connecting component 5 is used to three-dimensional video-frequency auxiliary filming apparatus being fixed on smart phone 0；Fastener 6 is solid for fixing, adjusting
Determine relative position and the angle of frame 4 and connecting component 5.
Mobile phone used in the embodiment of the present invention is iphone 4s mobile phone, and the specific method is as follows: firstly, by three-dimensional video-frequency
Filming apparatus is assisted to be fixed on mobile phone back by mobile phone connecting component；Then, by each fastener adjust and fixed pan mirror,
Bearing attachment part and bracket holder；Plane where ensuring plane mirror is substantially vertical with mobile phone back, and plane mirror edge to the greatest extent may be used
Mobile phone back can be bonded, distance of the intersecting lens of plane mirror and mobile phone back apart from mobile phone back camera center is maintained at 1~3
Millimeter or so, and plane mirror is substantially vertical with mobile phone long side with the intersecting lens of mobile phone back.By above-mentioned connection, fixation and adjust
Section method finally ensures to observe the essentially identical scene real image of range size and the virtual image simultaneously from mobile phone.
Step S103, the installation parameter of calibration three-dimensional video-frequency auxiliary filming apparatus.
Fig. 3 shows the stereo video shooting imaging schematic diagram towards smart phone of the embodiment of the present invention.Due to auxiliary
Helping the installation of filming apparatus, there may be error, X-direction (i.e. the longitudinal direction of mobile phone) nothings of plane mirror normal vector n and camera
Method is completely coincident, therefore needs to assist three-dimensional video-frequency the plane mirror normal vector parameter of filming apparatus before shooting video data
It is demarcated.Calibration process only needs to execute once, multiple video data can be shot after the completion of calibration, until artificially to stereopsis
It is just needed again in the case that frequency auxiliary filming apparatus is adjusted or large change has occurred in illumination condition or scene content
It is demarcated.Specific step is as follows for the calibration of plane mirror normal vector:
Firstly, shooting an image using the smart phone for having installed three-dimensional video-frequency auxiliary filming apparatus additional, it is denoted asIt is above-mentioned
The corresponding photographed scene of image is needed comprising more rich texture, colouring information, cannot be metope, desktop etc. is that color is single
Scene.
It is extracted secondly, algorithm is detected and described using SIFT image local featureSparse features point and each characteristic point
Description vectors.For each characteristic point, using distance than test feature matching process obtain itsAll characteristic points (remove
Current signature point itself) in matching characteristic point, distance is 0.6 than the value of threshold tau.Remember that final Feature Points Matching quantity is
N, Feature Points Matching collection are combined intoWherein,For figure
As characteristic point coordinate (unit is pixel), image coordinate system origin is located at picture centre, and x-axis is directed toward right, y-axis edge along image long side
Under image short side is directed toward.If N < 100, the smart phone shooting for having installed three-dimensional video-frequency auxiliary filming apparatus additional is re-used
One image, and re-execute this step.
Again, N is repeateditThe operation of secondary flat mirror candidate's normal estimation, jth time normal estimation operation includes following two
A step (Nit=min (N (N-1)/2,1000), j={ 1,2 ... Nit): first, two pairs of Feature Points Matchings are randomly selected, are remembered
ForSecond, calculate the normal vector determining by the matching of features described above pointCalculation formula is such as
Under:
Wherein, f indicates that the focal length (unit is pixel) of camera, symbol "×" indicate the multiplication cross of vector.
Finally, Calculation Plane mirror final method vector n, calculation formula are as follows:
Wherein, symbol " []k" indicate vector k-th of element.[if n]1< 0, then enable n ←-n.Fig. 4 is shown
One embodiment of plane mirror normal estimation.Figure 4, it is seen that in candidate normal estimation, there are many noises and mistakes
Accidentally, but concentration is compared in intermediate region.
Step S105 calculates perspective geometry transformation matrix and grey level compensation mapping function using smart phone.
Firstly, calculating image transformation matrix H, calculation formula is as follows:
Wherein, w and h respectively indicate video frame width and height (unit is pixel), R indicate around rotary shaft (0,
[n]3,-[n]2)TIt rotates angle arccos ([n]1) spin matrix.
Secondly, grey level compensation mapping function is estimated, and include following two steps: first, obtain the matching pair of validity feature point
SetFirstly, enablingThe matching of validity feature point initializes set.Then, according to following formula meter
Calculate each Feature Points Matching pairThe coincident indicator δ of (i={ 1,2 ..., N }):
IfThen willFromMiddle deletion.Wherein, the value of parameter θ is 3 °.Note
Final validity feature point matching is to setSecond, building actual situation grey level compensation mapping
Function: firstly, creation NinA binary group(j=1,2 ..., Nin).Wherein, g () indicates image characteristic point
The gray value of image of position.Using cubic polynomial curve fitting algorithm, according to binary groupFitting ash
Spend compensation map functionThe value interval of s is 0~255,Value interval be 0~255.Fig. 5 shows ash
Spend one embodiment of penalty function fitting result.As shown in figure 5,Nonlinear characteristic is presented, and different scenes are correspondingIt is different.
Step S107 shoots video data using smart phone.
Video is shot using the smart phone for having installed three-dimensional video-frequency auxiliary filming apparatus additional, after the completion of parameter calibration, is being protected
Hold camera focal length and three-dimensional video-frequency auxiliary each component relative position of filming apparatus it is constant in the case where, imaged using smart phone
Head shoots video data, and the long side of mobile phone and the angle of horizontal plane are less than 30 ° in shooting process.
The video data of shooting is converted three-dimensional video-frequency by step S109.
Each frame for shooting video carries out Geometry rectification, transformation, cutting operation, obtains the left and right view of stereo video frame
Figure, and the relevant three-dimensional video-frequency of synthesis device, specific method is:
It (is denoted as firstly, for the arbitrary frame in video), video frame is generated in accordance with the following stepsCorresponding left and right
View
First, creating a width is wp, highly be hpGeometry rectification video framewp、hpRespectively by following formula
It calculates:
Wherein, u 'll=H (- w/2, h/2,1)T、u′ul=H (- w/2 ,-h/2,1)T、u′lr=H (w/2, h/2,1)T、u′ur
=H (w/2 ,-h/2,1)T.ImageIn the color value of any location of pixels (m, n) be equal toMiddle location of pixelsColor value.Wherein,
For the mobile phone that the present embodiment uses for iphone 4s mobile phone, the length-width ratio of video frame is 4:3.Fig. 6 left figure is to install this additional
The video frame example of the smart phone shooting of invention auxiliary filming apparatus, can be seen that from Fig. 6 left figure due to installation error feature
The matched line of point is simultaneously not parallel, therefore cannot realize Geometry rectification by simple image rotation.Fig. 6 right figure is to utilize this hair
The video frame Geometry rectification that bright method is realized is as a result, obtain the video frame almost without horizontal parallax.
Second, create video frameCorresponding left and right viewWith WithCorrespond respectively to imageLeft and right two etc.
Point, then to imageCarry out left and right mirror surface transformation.
Third, forEach pixel, it is assumed that its RGB color value is respectively R, G, B, then the pixel after color compensating
RGB color value is respectivelyWherein t=0.299R+0.587G+0.114B.Fig. 7
To cut the left view obtained and right view example from Geometry rectification video, wherein left figure is left view, and middle figure is without face
The right view of colorimetric compensation, right figure are the right view after color compensating.It can be seen from figure 7 that color correction method of the present invention can
Effectively to compensate light reflection loss.
Then, it for stereoscopic video observation mode (such as: VR glasses, naked eye, red blue glasses), is differentiated according to video
The parameters such as rate, aspect ratio are to left and right viewWithIt is cut, is then corresponding stereopsis by each frame or so view merging
All stereo video frames are finally synthesized three-dimensional video-frequency by frequency frame.Fig. 8 illustrates the length-width ratio generated of scheme according to the present invention
For the stereo video frame example of 16:9.Wherein, Fig. 8 left figure is Cross-eye (cross eye) three-dimensional video-frequency for being suitble to naked eye,
Fig. 8 right figure is the left and right three-dimensional video-frequency being suitble to using the viewing of VR glasses.
From the above it can be seen that the three-dimensional video-frequency generation method provided in an embodiment of the present invention towards smart phone is only
Need a camera that can shoot three-dimensional video-frequency, thus can avoid two cameras it is asynchronous caused by left and right picture is uncoordinated asks
Topic, and three-dimensional video-frequency of the present invention auxiliary filming apparatus is with of simple structure and low cost, easy to carry and easy to use etc. excellent
Point.Therefore, the method for the present invention and device can help ordinary consumer to obtain the stronger stereopsis of feeling of immersion using smart phone
Frequently, it enriches smart phone visual form, promote shooting and ornamental enjoyment.
In yet another embodiment of the present invention, the three-dimensional video-frequency auxiliary filming apparatus structure chart towards smart phone is as schemed
Shown in 2, comprising: stainless steel plane mirror 1, plane mirror fixed frame 2, bearing attachment part 3, bracket holder 4, mobile phone connecting component
5, fastener 6.Wherein, stainless steel plane mirror 1 is used to obtain the virtual image of scene；Plane mirror fixed frame 2 is flat for fixing stainless steel
Face mirror 1；Bearing attachment part 3 is for connecting plane mirror fixed frame 2 and bracket holder 4；Bracket holder 4 is for fixing bracket
Connecting component 3；Mobile phone connecting component 5 is used to three-dimensional video-frequency auxiliary filming apparatus being fixed on smart phone 0；Fastener 6 is used
In fixation, relative position and the angle of adjusting fixed frame 4 and connecting component 5.
From the above it can be seen that the three-dimensional video-frequency generation method provided in an embodiment of the present invention towards smart phone is only
Need a camera that can shoot three-dimensional video-frequency, thus can avoid two cameras it is asynchronous caused by left and right picture is uncoordinated asks
Topic, and three-dimensional video-frequency of the present invention auxiliary filming apparatus is with of simple structure and low cost, easy to carry and easy to use etc. excellent
Point.Therefore, the method for the present invention and device can help ordinary consumer to obtain the stronger stereopsis of feeling of immersion using smart phone
Frequently, it enriches smart phone visual form, promote shooting and ornamental enjoyment.
Claims (8)
1. a kind of three-dimensional video-frequency generation method towards smart phone characterized by comprising
Building three-dimensional video-frequency auxiliary filming apparatus is simultaneously installed on smart phone；
Demarcate the installation parameter of three-dimensional video-frequency auxiliary filming apparatus；
Perspective geometry transformation matrix and grey level compensation mapping function are calculated using smart phone；
Video data is shot using smart phone；
Three-dimensional video-frequency is converted by the video data of shooting；
It is described to convert three-dimensional video-frequency for the video data of shooting and include:
Video data is decomposed by frame, and geometric correction is carried out using image transformation matrix to every frame image and cuts blank area
Domain；
It is same size left view and right view by every frame image segmentation, and left and right mirror surface transformation is carried out to right view；
Color compensating is carried out to each pixel in right view according to grey level compensation mapping；
According to each frame left view of stereoscopic video observation mode adaptive processes and right view, and by each frame left view and right view
Figure merges into stereo video frame, then all stereo video frames are merged into three-dimensional video-frequency；
The three-dimensional video-frequency auxiliary filming apparatus includes the stainless steel plane mirror (1) for obtaining the scene virtual image, stainless steel plane
Plane and smart phone (0) back side intersect at stainless steel plane mirror (1) inward flange where mirror (1): by connection, fixation and tune
Section, it is ensured that can observe the essentially identical scene real image of range size and the virtual image simultaneously from smart phone；
It is described calibration three-dimensional video-frequency auxiliary filming apparatus installation parameter include:
Image is shot using smart phone, wherein the corresponding photographed scene of the image includes more rich texture, colouring information,
It and cannot be the single scene of color；
Multiple sparse features points are extracted from the image and carry out Feature Points Matching, generate Feature Points Matching set；
Multiplicating randomly selects two pairs of characteristic points and matches, and generates the determining plane mirror normal vector of features described above point matching and estimates
Meter；
Determine plane mirror normal vector as installation parameter according to the intermediate value of each component of multiple normal estimations.
2. the method according to claim 1, wherein plane and smart phone (0) where stainless steel plane mirror (1)
The back side is perpendicular, and stainless steel plane mirror (1) inward flange and smart phone (0) side are perpendicular: meanwhile, stainless steel plane mirror (1)
Fixed by plane mirror fixed frame (2), plane mirror fixed frame (2) is connected to bracket holder (4) by bearing attachment part (3),
Bracket holder (4) is connected by mobile phone connecting component (5) with smart phone (0), and plane mirror fixed frame (2) is connect with bracket
Between component (3), between bracket holder (4) and mobile phone connecting component (5) with fastener (6) it is fixed or adjust relative position with
Angle.
3. extracting multiple sparse features points from the image the method according to claim 1, wherein described and going forward side by side
Row Feature Points Matching, generating Feature Points Matching set includes:
Algorithm is detected and described according to image local feature, and multiple sparse features points and each sparse features are extracted from the image
The description vectors of point；
Each sparse features point is matched with other characteristic points in the image than test feature matching process according to distance,
Generate matching characteristic point pair；
It include the Feature Points Matching set of at least 100 characteristic points pair to foundation according to multiple sparse features points.
4. the method according to claim 1, wherein the multiplicating randomly selects two pairs of characteristic points progress
Match, generating the determining plane mirror normal vector of features described above point matching includes:
Determine that randomly selecting two pairs of characteristic points carries out matched number according to the quantity of characteristic point pair；
It randomly selects two pairs of characteristic points every time to match, and according to the image coordinate of two pairs of characteristic points and smart phone camera
Focal length generate the determining normal vector of features described above point matching.
5. the method according to claim 1, wherein described calculate perspective geometry transformation matrix using smart phone
Include: with grey level compensation mapping function
Image transformation matrix is generated according to the normal vector of the length of video frame, width and plane mirror；
The coincident indicator of all characteristic points pair in Feature Points Matching set is obtained according to the focal length of smart phone camera；
It collects coincident indicator and is greater than all characteristic points pair for preassigning wealthy value, establish the matching of validity feature point to set；
The matching of validity feature point is obtained to intend to the image grayscale of characteristic point position each in set, and according to cubic polynomial curve
The image grayscale of hop algorithm and each characteristic point position is fitted grey level compensation mapping function.
6. the method according to claim 1, wherein it is described using smart phone shooting video data when, mobile phone
Transverse sides angle with horizontal plane is no more than 30 degree.
7. the method according to claim 1, wherein the stereoscopic video observation mode include it is following at least it
One: VR glasses, red blue glasses, naked eye: described according to each frame left view of stereoscopic video observation mode adaptive processes and right view
Figure, video resolution and aspect ratio to be specified according to stereoscopic video observation mode fit each frame left view with right view
Answering property is handled.
8. a kind of three-dimensional video-frequency applied to the method according to claim 1 to 7 assists filming apparatus, feature exists
In, including the stainless steel plane mirror (1) for obtaining the scene virtual image, plane and smart phone (0) where stainless steel plane mirror (1)
The back side is perpendicular, and plane and smart phone (0) back side intersect at stainless steel plane mirror (1) inner edge where stainless steel plane mirror (1)
Edge, stainless steel plane mirror (1) inward flange and smart phone (0) side are perpendicular: meanwhile, stainless steel plane mirror (1) is by plane mirror
Fixed frame (2) is fixed, and plane mirror fixed frame (2) is connected to bracket holder (4) by bearing attachment part (3), and bracket is fixed
Frame (4) is connected by mobile phone connecting component (5) with smart phone (0), plane mirror fixed frame (2) and bearing attachment part (3)
Between, it is fixed with fastener (6) between bracket holder (4) and mobile phone connecting component (5) or adjust relative position and angle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710333255.3A CN106954060B (en) | 2017-05-12 | 2017-05-12 | A kind of three-dimensional video-frequency generation method and auxiliary filming apparatus towards smart phone |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710333255.3A CN106954060B (en) | 2017-05-12 | 2017-05-12 | A kind of three-dimensional video-frequency generation method and auxiliary filming apparatus towards smart phone |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106954060A CN106954060A (en) | 2017-07-14 |
| CN106954060B true CN106954060B (en) | 2019-03-19 |
Family
ID=59478499
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710333255.3A Active CN106954060B (en) | 2017-05-12 | 2017-05-12 | A kind of three-dimensional video-frequency generation method and auxiliary filming apparatus towards smart phone |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106954060B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114786001B (en) * | 2022-05-20 | 2023-12-05 | 广东未来科技有限公司 | 3D picture shooting method and 3D shooting system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2436409A (en) * | 2006-03-20 | 2007-09-26 | Sharp Kk | Camera with reflector for forming images on different sensor portions |
| KR101391951B1 (en) * | 2013-02-04 | 2014-05-07 | 주식회사 고글텍 | Single lens camera and method for taking side by side stereo pictures |
| CN105892216A (en) * | 2015-12-14 | 2016-08-24 | 乐视网信息技术（北京）股份有限公司 | 3D video recording device, 3D video processing method and mobile equipment |
-
2017
- 2017-05-12 CN CN201710333255.3A patent/CN106954060B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106954060A (en) | 2017-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109615703B (en) | Augmented reality image display method, device and equipment | |
| CN103207664B (en) | A kind of image processing method and equipment | |
| DE202017105894U1 (en) | Headset removal in virtual, augmented and mixed reality using a look database | |
| CN108663799A (en) | A kind of display control program and its display control method of VR images | |
| US20100002073A1 (en) | Blur enhancement of stereoscopic images | |
| CN103034330B (en) | A kind of eye interaction method for video conference and system | |
| Chen et al. | New stereoscopic video shooting rule based on stereoscopic distortion parameters and comfortable viewing zone | |
| WO2018188277A1 (en) | Sight correction method and device, intelligent conference terminal and storage medium | |
| CN108154514A (en) | Image processing method, device and equipment | |
| Jung et al. | Visual comfort improvement in stereoscopic 3D displays using perceptually plausible assessment metric of visual comfort | |
| CN109919911A (en) | Moving three dimension method for reconstructing based on multi-angle of view photometric stereo | |
| CN110099268B (en) | Blind area perspective display method with natural color matching and natural display area fusion | |
| CN108377355A (en) | A kind of video data handling procedure, device and equipment | |
| JPH07296185A (en) | Three-dimensional image display device | |
| CN108259881A (en) | 3D synthetic methods and its system based on parallax estimation | |
| CN105761243A (en) | Three-dimensional full face photographing system based on structured light projection and photographing method thereof | |
| CN106023073A (en) | Image splicing system | |
| CN109428987A (en) | A kind of 360 degree of stereo photographic devices of wear-type panorama and image pickup processing method | |
| CN112995638A (en) | Naked eye 3D acquisition and display system and method capable of automatically adjusting parallax | |
| CN106954060B (en) | A kind of three-dimensional video-frequency generation method and auxiliary filming apparatus towards smart phone | |
| CN109255838A (en) | Augmented reality is avoided to show the method and apparatus of equipment viewing ghost image | |
| GB2585197A (en) | Method and system for obtaining depth data | |
| CN113225484B (en) | Method and device for rapidly acquiring high-definition picture shielding non-target foreground | |
| CN113139913B (en) | New view correction generation method for portrait | |
| CN106028018B (en) | Real scene shooting double vision point 3D method for optimizing video and system towards naked eye 3D display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |