CN108833876B - A kind of stereoscopic image content recombination method - Google Patents
A kind of stereoscopic image content recombination method Download PDFInfo
- Publication number
- CN108833876B CN108833876B CN201810555934.XA CN201810555934A CN108833876B CN 108833876 B CN108833876 B CN 108833876B CN 201810555934 A CN201810555934 A CN 201810555934A CN 108833876 B CN108833876 B CN 108833876B
- Authority
- CN
- China
- Prior art keywords
- grid
- delaunay
- coordinate position
- indicate
- vertex
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a kind of stereoscopic image content recombination methods, it scales energy by the left view point image and the corresponding picture quality energy of right visual point image, object of extracting stereo-picture, position adjusts energy and parallax adapts to energy, and by optimization so that gross energy is minimum, obtain best similitude transformation matrix, stereo-picture after aloowing recombining contents in this way retains accurate object shapes, sense of depth with higher, and position and the size of important content can be adaptively controlled according to the user's choice, meet significant semantic feature；The coordinate position of Delaunay grid in its clustering object and other clustering objects by user's selection in control stereo-picture, and the deformation of Delaunay grid is controlled in turn, so as to guarantee the visual experience quality of the stereo-picture after recombining contents.
Description
Technical field
The present invention relates to a kind of processing methods of picture signal, more particularly, to a kind of stereoscopic image content recombination method.
Background technique
With the fast development of 3D technology, stereo-picture and three-dimensional video-frequency have been to be concerned by more and more people and like.It is special
It is not with the development of mobile phone, plate and PC, the display of mobile terminal is increasingly by the welcome of users.However,
When showing stereo-picture and three-dimensional video-frequency on the screen of mobile terminal, three-dimensional sense can weaken therewith even to disappear, and content producer attempts
It is primarily focused on viewer on the object by adjusting contents and distribution and depth, to promote the three-dimensional sense of the object.
When therefore, for showing stereo-picture and three-dimensional video-frequency on the screen of mobile terminal, the concern of the object is can be enhanced in recombining contents
Degree and sense of depth.
Traditional picture material recombination is broadly divided into two classes: preceding one kind method is to the factor for influencing image aesthetic feeling, such as
Color, light, lines, composition etc. are adjusted, to enhance image aesthetics；Latter class method be directly by object extraction and
It pastes, it will be in the recombination to same image of different objects.Stereoscopic image content is recombinated, the solid after how reducing recombining contents
The image deformation of image, how according to the user's choice adaptively how the size of control object is protected with the significant content of protrusion
The depth consistency of stereo-picture after demonstrate,proving recombining contents is all to need to study during carrying out recombining contents to stereo-picture
It solves the problems, such as.
Summary of the invention
Technical problem to be solved by the invention is to provide one kind to meet significant semantic feature, and can effectively adjust vertical
The stereoscopic image content recombination method of body image content layout.
The technical scheme of the invention to solve the technical problem is: a kind of stereoscopic image content recombination method,
Be characterized in that the following steps are included:
Step 1: by width to be processed is W and height is H the left view point image of stereo-picture, right visual point image and
Left view difference image correspondence is denoted as { L (x, y) }, { R (x, y) } and { dL(x,y)}；Wherein, 1≤x≤W, 1≤y≤H, L (x, y) table
Show that coordinate position in { L (x, y) } is the pixel value of the pixel of (x, y), R (x, y) indicate that coordinate position is in { R (x, y) } (x,
Y) pixel value of pixel, dL(x, y) indicates { dL(x, y) } in coordinate position be (x, y) pixel pixel value；
Step 2: go out the notable figure of { L (x, y) } using the significant model extraction of vision based on graph theory, be denoted as { SML(x,
y)}；Then according to { SML(x, y) } and { dL(x, y) }, the visual saliency map of { L (x, y) } is obtained, { S is denoted asL(x,y)}；Then
According to { SL(x, y) } and { dL(x, y) }, the visual saliency map of { R (x, y) } is obtained, { S is denoted asR(x,y)}；Wherein, SML(x,y)
Indicate { SML(x, y) } in coordinate position be (x, y) pixel pixel value, SL(x, y) indicates { SL(x, y) } in coordinate position
For the pixel value of the pixel of (x, y), SR(x, y) indicates { SR(x, y) } in coordinate position be (x, y) pixel pixel value；
Step 3: { L (x, y) } is divided into multiple irregular Delaunay grids not overlapped, by { L (x, y) }
In k-th of Delaunay grid be denoted as UL,k, UL,kIt is described with set that its 3 grid vertexes are constituted,Then according to all Delaunay grids and { d in { L (x, y) }L(x, y) }, obtain R (x,
Y) all irregular Delaunay grids not overlapped in }, k-th of Delaunay grid in { R (x, y) } is denoted as
UR,k, UR,kIt is described with set that its 3 grid vertexes are constituted,Wherein, k is positive integer, 1≤k
The total number for the Delaunay grid for including in≤M, M expression { L (x, y) }, M > 1,It is corresponding to indicate UL,k?
1 grid vertex, the 2nd grid vertex, the 3rd grid vertex,WithHorizontal coordinate positionAnd vertical coordinate
PositionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionTo retouch
It states, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, It is corresponding to indicate UR,kThe 1st grid vertex, the 2nd grid vertex, the 3rd grid
Vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, It indicates
{dL(x, y) } in coordinate position bePixel pixel value,WithHorizontal coordinate positionWith hang down
Straight coordinate positionIt describes, Indicate { dL
(x, y) } in coordinate position bePixel pixel value,WithHorizontal coordinate positionWith it is vertical
Coordinate positionIt describes, Indicate { dL
(x, y) } in coordinate position bePixel pixel value；
Step 4: cluster segmentation is carried out to all pixels point in { L (x, y) } using K mean cluster method, obtains { L
(x, y) } in all clustering objects, and then the object exposure mask of each clustering object in { L (x, y) } is obtained, by { L (x, y) }
In the object exposure mask of n-th of clustering object be denoted as On；Setting user has selected n-th of clustering object in { L (x, y) }, then
According to all Delaunay grids in { L (x, y) }, by OnIn kth " a Delaunay grid is denoted as With
The set that its 3 grid vertex is constituted describes,Wherein, n is positive integer, 1≤n≤N, N
Indicate the total number of the clustering object in { L (x, y) }, N > 1, k " are positive integer, and 1≤k "≤M ", M " indicate OnIn include
The total number of Delaunay grid, M " > 1,It is corresponding to indicateThe 1st grid vertex, the 2nd net
Lattice vertex, the 3rd grid vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes,
Step 5: the references object exposure mask that user provides is denoted as P；Then by P be divided into it is multiple do not overlap do not advise
Kth in P ' a Delaunay grid is denoted as U by Delaunay grid thenP,k', UP,k'The collection constituted with its 3 grid vertexes
It closes to describe,Wherein, k' is positive integer, includes in 1≤k'≤M', M' expression P
The total number of Delaunay grid, M'> 1,It is corresponding to indicate UP,k'The 1st grid vertex, the 2nd grid
Vertex, the 3rd grid vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes,
Step 6: on the basis of n-th of clustering object that user has selected in { L (x, y) }, O is calculatednWith the object of P
Horizontal offset, object vertical offset and object zoom factor, correspondence are denoted asAnd ρ,Wherein,Indicate level side
To,Indicate vertical direction,Indicate OnIn all Delaunay grids all grid vertexes in t-th
Grid vertex,It indicatesHorizontal coordinate position,It indicatesVertical coordinate position,Indicate all in P
The t' grid vertex in all grid vertexes of Delaunay grid,It indicatesHorizontal coordinate position,It indicatesVertical coordinate position,Indicate OnIn all Delaunay grids constitute set, also illustrate that OnIn it is all
The set that all grid vertexes of Delaunay grid are constituted, VPIndicate the set that all Delaunay grids in P are constituted,
Indicate the set that all grid vertexes of all Delaunay grids in P are constituted,1≤t'≤NP,It indicates
OnIn all Delaunay grids grid vertex total number, NPIndicate the grid vertex of all Delaunay grids in P
Total number, Symbol " | | " it is to take absolutely
To value symbol；
Step 7: each Delaunay grid in { L (x, y) } is corresponding with target Delaunay grid, by UL,kIt is corresponding
Target Delaunay grid is denoted as It is described with set that its 3 grid vertexes are constituted,Then according to the corresponding target Delaunay net of all Delaunay grids in { L (x, y) }
Lattice carry out similarity transformation to each Delaunay grid in { L (x, y) }, so that the Delaunay grid of script and script
The mapping fault for the target Delaunay grid that Delaunay grid obtains after similarity transformation is minimum, obtains in { L (x, y) }
The corresponding target Delaunay grid of each Delaunay grid similitude transformation matrix, willSimilitude transformation matrix note
For Wherein,It is corresponding to indicateThe 1st grid top
Point, the 2nd grid vertex, the 3rd grid vertex,It indicatesI-th of grid vertex, i=1,2,3, WithIt is corresponding to indicateHorizontal coordinate position and vertical sit
Cursor position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicate's
Horizontal coordinate position and vertical coordinate position, (AL,k)TFor AL,kTransposition, ((AL,k)TAL,k)-1For (AL,k)TAL,kIt is inverse；
Equally, each Delaunay grid in { R (x, y) } is corresponding with target Delaunay grid, by UR,kCorresponding mesh
Mark Delaunay grid is denoted as It is described with set that its 3 grid vertexes are constituted,Then according to the corresponding target Delaunay net of all Delaunay grids in { R (x, y) }
Lattice carry out similarity transformation to each Delaunay grid in { R (x, y) }, so that the Delaunay grid of script and script
The mapping fault for the target Delaunay grid that Delaunay grid obtains after similarity transformation is minimum, obtains in { R (x, y) }
The corresponding target Delaunay grid of each Delaunay grid similitude transformation matrix, willSimilitude transformation matrix note
For Wherein,It is corresponding to indicateThe 1st grid top
Point, the 2nd grid vertex, the 3rd grid vertex,It indicatesI-th of grid vertex, i=1,2,3, WithIt is corresponding to indicateHorizontal coordinate position and vertical sit
Cursor position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicate's
Horizontal coordinate position and vertical coordinate position, (AR,k)TFor AR,kTransposition, ((AR,k)TAR,k)-1For (AR,k)TAR,kIt is inverse；
Step 8: according to the similar change of the corresponding target Delaunay grid of each Delaunay grid in { L (x, y) }
The similitude transformation matrix of the corresponding target Delaunay grid of each Delaunay grid in matrix and { R (x, y) } is changed, and is tied
Close { SL(x, y) } and { SR(x, y) }, calculate the corresponding target of all Delaunay grids in { L (x, y) } and { R (x, y) }
The picture quality energy of Delaunay grid, is denoted as EIQ；
According to ρ, all Delaunay grids and { R in { L (x, y) } in n-th of clustering object of user's selection are calculated
(x, y) } in the object of the corresponding target Delaunay grid of corresponding all Delaunay grids scale energy, be denoted as ESO；
According toWithCalculate all Delaunay grids in { L (x, y) } in n-th of clustering object of user's selection
The position of corresponding target Delaunay grid adjusts energy, is denoted as EAO；
According to ρ, all Delaunay grids and { R in { L (x, y) } in n-th of clustering object of user's selection are calculated
(x, y) } in the parallax of the corresponding target Delaunay grid of corresponding all Delaunay grids adapt to energy, be denoted as EDS；
Step 9: according to EIQ、ESO、EAOAnd EDS, calculate all Delaunay grids pair in { L (x, y) } and { R (x, y) }
The gross energy for the target Delaunay grid answered, is denoted as Etotal, Etotal=EIQ+λSO×ESO+λAO×EAO+λDS×EDS；Then lead to
Cross Least-squares minimization solutionObtain the corresponding optimum target of each Delaunay grid in { L (x, y) }
The corresponding optimum target Delaunay grid of each Delaunay grid in Delaunay grid and { R (x, y) }, by UL,kIt is right
The optimum target Delaunay grid answered is denoted as By UR,kCorresponding optimum target
Delaunay grid is denoted as Then each Delaunay in { L (x, y) } is calculated
The best similitude transformation matrix of the corresponding optimum target Delaunay grid of grid, willBest similitude transformation matrix be denoted as And it is corresponding most to calculate each Delaunay grid in { R (x, y) }
The best similitude transformation matrix of good target Delaunay grid, willBest similitude transformation matrix be denoted as Wherein, λSOFor ESOWeighting parameters, λAOFor EAOWeighting parameters, λDSFor EDS
Weighting parameters, min () be minimized function,Indicate the corresponding target of all Delaunay grids in { L (x, y) }
The set that Delaunay grid is constituted,Indicate the corresponding target Delaunay of all Delaunay grids in { R (x, y) }
The set that grid is constituted,It is corresponding to indicateThe 1st grid vertex, the 2nd grid vertex, the 3rd
Grid vertex,It is corresponding to indicateThe 1st grid vertex, the 2nd grid vertex, the 3rd grid top
Point, WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicateLevel
Coordinate position and vertical coordinate position, WithIt is corresponding to indicateHorizontal coordinate position and hang down
Straight coordinate position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is right
It should indicateHorizontal coordinate position and vertical coordinate position；
Step 10: most according to the corresponding optimum target Delaunay grid of each Delaunay grid in { L (x, y) }
Good similitude transformation matrix calculates each pixel in each Delaunay grid in { L (x, y) } through best similarity transformation square
Horizontal coordinate position and vertical coordinate position after fractal transform, by UL,kMiddle horizontal coordinate position is x'L,kWith vertical coordinate position
y'L,kPixel through the transformed horizontal coordinate position of best similitude transformation matrix and vertical coordinate position correspondence be denoted as
With Then according to each pixel warp in each Delaunay grid in { L (x, y) }
Horizontal coordinate position and vertical coordinate position after best similarity transformation rectangular transform, the left view point diagram after obtaining recombining contents
Picture is denoted asWherein, 1≤x'L,k≤ W, 1≤y'L,k≤ H, It indicatesMiddle coordinate position is
The pixel value of the pixel of (x', y')；
Equally, according to the best of the corresponding optimum target Delaunay grid of each Delaunay grid in { R (x, y) }
Similitude transformation matrix calculates each pixel in each Delaunay grid in { R (x, y) } through best similarity transformation rectangle
Transformed horizontal coordinate position and vertical coordinate position, by UR,kMiddle horizontal coordinate position is x'R,kWith vertical coordinate position
y'R,kPixel through the transformed horizontal coordinate position of best similitude transformation matrix and vertical coordinate position correspondence be denoted as
With Then according to each pixel warp in each Delaunay grid in { R (x, y) }
Horizontal coordinate position and vertical coordinate position after best similarity transformation rectangular transform, the right viewpoint figure after obtaining recombining contents
Picture is denoted asWherein, 1≤x'R,k≤ W, 1≤y'R,k≤ H, Table
ShowMiddle coordinate position is the pixel value of the pixel of (x', y').
In the step two,SR(x, y)=SL(x+dL(x,
y),y)；Wherein,Indicate SMLThe weight of (x, y),Indicate dLThe weight of (x, y),SL(x+dL(x,y),
Y) { S is indicatedL(x, y) } in coordinate position be (x+dL(x, y), y) pixel pixel value.
E in the step eightIQCalculating process are as follows:
A, the shape for calculating the corresponding target Delaunay grid of all Delaunay grids in { L (x, y) } protects energy
Amount, is denoted as Wherein, SL(k) U is indicatedL,kIn all pixels point vision
The mean value of saliency value, namely indicate { SL(x, y) } in UL,kThe mean value of the pixel value of all pixels point in corresponding region,
Symbol " | | | | " it is to seek Euclidean distance symbol；
Equally, the shape protection of the corresponding target Delaunay grid of all Delaunay grids in { R (x, y) } is calculated
Energy is denoted as Wherein, SR(k) U is indicatedR,kIn all pixels point view
Feel the mean value of saliency value, namely indicates { SR(x, y) } in UR,kThe pixel value of all pixels point in corresponding region it is equal
Value；
B, the boundary curvature of the corresponding target Delaunay grid of all Delaunay grids in { L (x, y) } is calculated
Energy is denoted as Wherein, eL,kIndicate UL,kAll grids
The matrix of the edge composition on vertex,(eL,k)TFor eL,kTransposition, ((eL,k)TeL,k)-1
For (eL,k)TeL,kIt is inverse,It indicatesAll grid vertexes edge composition matrix,
Equally, the boundary bending of the corresponding target Delaunay grid of all Delaunay grids in { R (x, y) } is calculated
Energy is spent, is denoted as Wherein, eR,kIndicate UR,kIt is all
The matrix of the edge composition of grid vertex,(eR,k)TFor eR,kTransposition, ((eR,k)TeR,k)-1For (eR,k)TeR,kIt is inverse,It indicatesAll grid vertexes edge composition matrix,
C, basisWithCalculate EIQ,Its
In, λLBFor weighting parameters.
E in the step eightSOCalculating process are as follows:Wherein,It indicates in { L (x, y) }
The object of the corresponding target Delaunay grid of all Delaunay grids in n-th of clustering object of user's selection scales energy
Amount, Indicate area corresponding with n-th of clustering object of user's selection in { R (x, y) }
The object of the corresponding target Delaunay grid of all Delaunay grids in domain scales energy,Symbol " | | | | " it is to ask Euclidean distance symbol, eL,k″Indicate user's selection in { L (x, y) }
N-th of clustering object in kth " all grid vertexes of a Delaunay grid edge composition matrix, also indicateAll grid vertexes edge composition matrix,Indicate n-th of clustering object of user's selection in { L (x, y) }
In kth " matrix of the edge composition of all grid vertexes of the corresponding target Delaunay grid of a Delaunay grid, also
It indicatesThe matrix of the edge composition of all grid vertexes of corresponding target Delaunay grid, eR,k″Indicate R (x,
Y) " a Delaunay grid is corresponding with the kth in { L (x, y) } in n-th of clustering object of user's selection in }
All grid vertexes of Delaunay grid edge composition matrix, also indicate { R (x, y) } inIt is corresponding
The matrix of the edge composition of all grid vertexes of Delaunay grid,Indicate { R (x, y) } in { L (x, y) } in user
Kth in n-th of clustering object of selection " target corresponding to the corresponding Delaunay grid of a Delaunay grid
All grid vertexes of Delaunay grid edge composition matrix, also indicate { R (x, y) } inIt is corresponding
The matrix of the edge composition of all grid vertexes of target Delaunay grid corresponding to Delaunay grid.
E in the step eightAOCalculating process are as follows:Wherein,It indicatesAffiliated Delaunay
In the corresponding target Delaunay grid of grid withCorresponding grid vertexHorizontal coordinate position,It indicates
In the corresponding target Delaunay grid of affiliated Delaunay grid withCorresponding grid vertexVertical coordinate
Position.
E in the step eightDSCalculating process are as follows:
,
Wherein,It indicatesParallax value,It indicatesIn the corresponding target Delaunay grid of affiliated Delaunay grid
WithCorresponding grid vertexHorizontal coordinate position,It indicatesThe affiliated corresponding target of Delaunay grid
In Delaunay grid withCorresponding grid vertexVertical coordinate position,Indicate { R (x, y) } inInstitute
In target Delaunay grid corresponding to the corresponding Delaunay grid of the Delaunay grid of category withCorresponding net
The horizontal coordinate position on lattice vertex,Indicate { R (x, y) } inThe affiliated corresponding Delaunay of Delaunay grid
In target Delaunay grid corresponding to grid withThe vertical coordinate position of corresponding grid vertex,It indicates to remove On
The set that other outer all object exposure masks are constituted, It indicatesIn all Delaunay grids grid
The total number on vertex,It indicatesIn all Delaunay grids all grid vertexes in a grid vertex of t ",It indicatesParallax value,It indicatesIn the corresponding target Delaunay grid of affiliated Delaunay grid with
Corresponding grid vertexHorizontal coordinate position,It indicatesThe affiliated corresponding target of Delaunay grid
In Delaunay grid withCorresponding grid vertexVertical coordinate position,Indicate { R (x, y) } inInstitute
In target Delaunay grid corresponding to the corresponding Delaunay grid of the Delaunay grid of category withCorresponding net
The horizontal coordinate position on lattice vertex,Indicate { R (x, y) } inThe affiliated corresponding Delaunay of Delaunay grid
In target Delaunay grid corresponding to grid withThe vertical coordinate position of corresponding grid vertex,It indicates
In all Delaunay grids all grid vertexes constitute set.
Compared with the prior art, the advantages of the present invention are as follows:
1) left view point image and the corresponding picture quality energy of right visual point image that the method for the present invention passes through extraction stereo-picture
Amount, object scale energy, position adjustment energy and parallax and adapt to energy, and by optimization so that gross energy is minimum, acquisition is best
Similitude transformation matrix, the stereo-picture after aloowing recombining contents in this way retain accurate object shapes, with higher
Sense of depth, and position and the size of important content can be adaptively controlled according to the user's choice, meet significant semantic feature.
2) in clustering object and other clustering objects of the method for the present invention by user's selection in control stereo-picture
The coordinate position of Delaunay grid, and the deformation of Delaunay grid is controlled in turn, after guaranteeing recombining contents
The visual experience quality of stereo-picture.
Detailed description of the invention
Fig. 1 is that the overall of the method for the present invention realizes block diagram；
Fig. 2 a is " red green " figure of the original three-dimensional image of " Image1 "；
Fig. 2 b is " red green " figure of " Image1 " after recombining contents；
Fig. 3 a is " red green " figure of the original three-dimensional image of " Image2 "；
Fig. 3 b is " red green " figure of " Image2 " after recombining contents；
Fig. 4 a is " red green " figure of the original three-dimensional image of " Image3 "；
Fig. 4 b is " red green " figure of " Image3 " after recombining contents；
Fig. 5 a is " red green " figure of the original three-dimensional image of " Image4 "；
Fig. 5 b is " red green " figure of " Image4 " after recombining contents.
Specific embodiment
The present invention will be described in further detail below with reference to the embodiments of the drawings.
A kind of stereoscopic image content recombination method proposed by the present invention, it is overall to realize that block diagram is as shown in Figure 1 comprising with
Lower step:
Step 1: by width to be processed is W and height is H the left view point image of stereo-picture, right visual point image and
Left view difference image correspondence is denoted as { L (x, y) }, { R (x, y) } and { dL(x,y)}；Wherein, 1≤x≤W, 1≤y≤H, L (x, y) table
Show that coordinate position in { L (x, y) } is the pixel value of the pixel of (x, y), R (x, y) indicate that coordinate position is in { R (x, y) } (x,
Y) pixel value of pixel, dL(x, y) indicates { dL(x, y) } in coordinate position be (x, y) pixel pixel value.
Step 2: significant (Graph-Based Visual Saliency, GBVS) using the existing vision based on graph theory
Model extraction goes out the notable figure of { L (x, y) }, is denoted as { SML(x,y)}；Then according to { SML(x, y) } and { dL(x, y) }, obtain { L
(x, y) } visual saliency map, be denoted as { SL(x,y)}；Then according to { SL(x, y) } and { dL(x, y) }, obtain the view of { R (x, y) }
Feel notable figure, is denoted as { SR(x,y)}；Wherein, SML(x, y) indicates { SML(x, y) } in coordinate position be (x, y) pixel
Pixel value, SL(x, y) indicates { SL(x, y) } in coordinate position be (x, y) pixel pixel value, SR(x, y) indicates { SR(x,
Y) coordinate position is the pixel value of the pixel of (x, y) in }.
In this particular embodiment, in step 2,SR(x, y)=
SL(x+dL(x,y),y)；Wherein,Indicate SMLThe weight of (x, y),Indicate dLThe weight of (x, y),At this
It is taken in embodimentSL(x+dL(x, y), y) indicate { SL(x, y) } in coordinate position be (x+dL(x, y), y)
The pixel value of pixel.
Step 3: { L (x, y) } is divided into multiple irregular Delaunay grids not overlapped, by { L (x, y) }
In k-th of Delaunay grid be denoted as UL,k, UL,kIt is described with set that its 3 grid vertexes are constituted,Then according to all Delaunay grids and { d in { L (x, y) }L(x, y) }, obtain R (x,
Y) all irregular Delaunay grids not overlapped in }, k-th of Delaunay grid in { R (x, y) } is denoted as
UR,k, UR,kIt is described with set that its 3 grid vertexes are constituted,Wherein, k is positive integer, 1≤k
≤ M, M indicate { L (x, y) } in include Delaunay grid total number, the numerical value of M > 1, M depending on the size of image,It is corresponding to indicate UL,kThe 1st grid vertex, the 2nd grid vertex, the 3rd grid vertex,With's
Horizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionIt is sat with vertical
Cursor positionIt describes, It is corresponding to indicate UR,kThe 1st grid vertex, the 2nd net
Lattice vertex, the 3rd grid vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, Indicate { dL(x, y) } in coordinate position bePixel pixel value,WithLevel
Coordinate positionWith vertical coordinate positionIt describes, Indicate { dL
(x, y) } in coordinate position bePixel pixel value,WithHorizontal coordinate positionWith it is vertical
Coordinate positionIt describes, Indicate { dL
(x, y) } in coordinate position bePixel pixel value.
Step 4: cluster segmentation is carried out to all pixels point in { L (x, y) } using existing K mean cluster method, is obtained
To all clustering objects in { L (x, y) }, and then the object exposure mask of each clustering object in { L (x, y) } is obtained, will L (x,
Y) the object exposure mask of n-th of clustering object in } is denoted as On；Setting user has selected n-th of clustering object in { L (x, y) },
Then according to all Delaunay grids in { L (x, y) }, by OnIn kth " a Delaunay grid is denoted as
It is described with set that its 3 grid vertexes are constituted,Wherein, n is positive integer, 1≤n≤
N, N indicate the total number of the clustering object in { L (x, y) }, and depending on different picture materials, k's size of N > 1, N " is positive
Integer, 1≤k "≤M ", M " indicate OnIn include Delaunay grid total number, M " > 1, the numerical value of M " is according to OnSize
Depending on,It is corresponding to indicateThe 1st grid vertex, the 2nd grid vertex, the 3rd grid vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes,
WithHorizontal coordinate positionWith vertical coordinate positionIt describes,
Step 5: the references object exposure mask that user provides is denoted as P；Then by P be divided into it is multiple do not overlap do not advise
Kth in P ' a Delaunay grid is denoted as U by Delaunay grid thenP,k', UP,k'The collection constituted with its 3 grid vertexes
It closes to describe,Wherein, k' is positive integer, includes in 1≤k'≤M', M' expression P
The total number of Delaunay grid, the numerical value of M'> 1, M' depending on the size of P,It is corresponding to indicate UP,k''s
1st grid vertex, the 2nd grid vertex, the 3rd grid vertex,WithHorizontal coordinate positionIt is sat with vertical
Cursor positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionTo retouch
It states,
Step 6: on the basis of n-th of clustering object that user has selected in { L (x, y) }, O is calculatednWith the object of P
Horizontal offset, object vertical offset and object zoom factor, correspondence are denoted asAnd ρ,Wherein,Indicate horizontal direction,Indicate vertical direction,Indicate OnIn all Delaunay grids all grid vertexes in t-th of grid vertex,It indicatesHorizontal coordinate position,It indicatesVertical coordinate position,Indicate all Delaunay nets in P
The t' grid vertex in all grid vertexes of lattice,It indicatesHorizontal coordinate position,It indicatesVertical seat
Cursor position,Indicate OnIn all Delaunay grids constitute set, also illustrate that OnIn all Delaunay grids
The set that all grid vertexes are constituted, VPIt indicates the set that all Delaunay grids in P are constituted, also illustrates that all in P
The set that all grid vertexes of Delaunay grid are constituted,1≤t'≤NP,Indicate OnIn it is all
The total number of the grid vertex of Delaunay grid, NPIndicate the total number of the grid vertex of all Delaunay grids in P, Symbol " | | " it is the symbol that takes absolute value.
Step 7: each Delaunay grid in { L (x, y) } is corresponding with target Delaunay grid, by UL,kIt is corresponding
Target Delaunay grid is denoted as It is described with set that its 3 grid vertexes are constituted,Then according to the corresponding target Delaunay net of all Delaunay grids in { L (x, y) }
Lattice carry out similarity transformation to each Delaunay grid in { L (x, y) }, so that the Delaunay grid of script and script
The mapping fault for the target Delaunay grid that Delaunay grid obtains after similarity transformation is minimum, obtains in { L (x, y) }
The corresponding target Delaunay grid of each Delaunay grid similitude transformation matrix, willSimilitude transformation matrix note
For Wherein,It is corresponding to indicateThe 1st grid top
Point, the 2nd grid vertex, the 3rd grid vertex,It indicatesI-th of grid vertex, i=1,2,3, WithIt is corresponding to indicateHorizontal coordinate position and vertical sit
Cursor position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicate
Horizontal coordinate position and vertical coordinate position, (AL,k)TFor AL,kTransposition, ((AL,k)TAL,k)-1For (AL,k)TAL,kIt is inverse.
Equally, each Delaunay grid in { R (x, y) } is corresponding with target Delaunay grid, by UR,kCorresponding mesh
Mark Delaunay grid is denoted as It is described with set that its 3 grid vertexes are constituted,Then according to the corresponding target Delaunay net of all Delaunay grids in { R (x, y) }
Lattice carry out similarity transformation to each Delaunay grid in { R (x, y) }, so that the Delaunay grid of script and script
The mapping fault for the target Delaunay grid that Delaunay grid obtains after similarity transformation is minimum, obtains in { R (x, y) }
The corresponding target Delaunay grid of each Delaunay grid similitude transformation matrix, willSimilitude transformation matrix note
For Wherein,It is corresponding to indicateThe 1st grid top
Point, the 2nd grid vertex, the 3rd grid vertex,It indicatesI-th of grid vertex, i=1,2,3, WithIt is corresponding to indicateHorizontal coordinate position and vertical sit
Cursor position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicate
Horizontal coordinate position and vertical coordinate position, (AR,k)TFor AR,kTransposition, ((AR,k)TAR,k)-1For (AR,k)TAR,kIt is inverse.
Step 8: according to the similar change of the corresponding target Delaunay grid of each Delaunay grid in { L (x, y) }
The similitude transformation matrix of the corresponding target Delaunay grid of each Delaunay grid in matrix and { R (x, y) } is changed, and is tied
Close { SL(x, y) } and { SR(x, y) }, calculate the corresponding target of all Delaunay grids in { L (x, y) } and { R (x, y) }
The picture quality energy of Delaunay grid, is denoted as EIQ。
According to ρ, all Delaunay grids and { R in { L (x, y) } in n-th of clustering object of user's selection are calculated
(x, y) } in the object of the corresponding target Delaunay grid of corresponding all Delaunay grids scale energy, be denoted as ESO。
According toWithCalculate all Delaunay grids in { L (x, y) } in n-th of clustering object of user's selection
The position of corresponding target Delaunay grid adjusts energy, is denoted as EAO。
According to ρ, all Delaunay grids and { R in { L (x, y) } in n-th of clustering object of user's selection are calculated
(x, y) } in the parallax of the corresponding target Delaunay grid of corresponding all Delaunay grids adapt to energy, be denoted as EDS。
In this particular embodiment, the E in step 8IQCalculating process are as follows:
A, the shape for calculating the corresponding target Delaunay grid of all Delaunay grids in { L (x, y) } protects energy
Amount, is denoted as Wherein, SL(k) U is indicatedL,kIn all pixels point vision
The mean value of saliency value, namely indicate { SL(x, y) } in UL,kThe mean value of the pixel value of all pixels point in corresponding region,
Symbol " | | | | " it is to seek Euclidean distance symbol.
Equally, the shape protection of the corresponding target Delaunay grid of all Delaunay grids in { R (x, y) } is calculated
Energy is denoted as Wherein, SR(k) U is indicatedR,kIn all pixels point view
Feel the mean value of saliency value, namely indicates { SR(x, y) } in UR,kThe pixel value of all pixels point in corresponding region it is equal
Value.
B, the boundary curvature of the corresponding target Delaunay grid of all Delaunay grids in { L (x, y) } is calculated
Energy is denoted as Wherein, eL,kIndicate UL,kAll grids
The matrix of the edge composition on vertex,(eL,k)TFor eL,kTransposition, ((eL,k)TeL,k)-1
For (eL,k)TeL,kIt is inverse,It indicatesAll grid vertexes edge composition matrix,
Equally, the boundary bending of the corresponding target Delaunay grid of all Delaunay grids in { R (x, y) } is calculated
Energy is spent, is denoted as Wherein, eR,kIndicate UR,kIt is all
The matrix of the edge composition of grid vertex,(eR,k)TFor eR,kTransposition, ((eR,k)TeR,k)-1For (eR,k)TeR,kIt is inverse,It indicatesAll grid vertexes edge composition matrix,
C, basisWithCalculate EIQ,Its
In, λLBFor weighting parameters, λ is taken in the present embodimentLB=2.
In this particular embodiment, the E in step 8SOCalculating process are as follows:Wherein,It indicates
The corresponding target Delaunay grid of all Delaunay grids in n-th of clustering object that user selects in { L (x, y) }
Object scales energy, Indicate n-th of cluster pair in { R (x, y) } with user's selection
As the corresponding target Delaunay grid of all Delaunay grids in corresponding region object scale energy,Symbol " | | | | " it is to ask Euclidean distance symbol, eL,k″Indicate user's selection in { L (x, y) }
N-th of clustering object in kth " all grid vertexes of a Delaunay grid edge composition matrix, also indicateAll grid vertexes edge composition matrix,Indicate n-th of clustering object of user's selection in { L (x, y) }
In kth " matrix of the edge composition of all grid vertexes of the corresponding target Delaunay grid of a Delaunay grid, also
It indicatesThe matrix of the edge composition of all grid vertexes of corresponding target Delaunay grid, eR,k″Indicate R (x,
Y) " a Delaunay grid is corresponding with the kth in { L (x, y) } in n-th of clustering object of user's selection in }
All grid vertexes of Delaunay grid edge composition matrix, also indicate { R (x, y) } inIt is corresponding
The matrix of the edge composition of all grid vertexes of Delaunay grid,Indicate { R (x, y) } in { L (x, y) } in user
Kth in n-th of clustering object of selection " target corresponding to the corresponding Delaunay grid of a Delaunay grid
All grid vertexes of Delaunay grid edge composition matrix, also indicate { R (x, y) } inIt is corresponding
The matrix of the edge composition of all grid vertexes of target Delaunay grid corresponding to Delaunay grid.
In this particular embodiment, the E in step 8AOCalculating process are as follows:Wherein,It indicatesAffiliated Delaunay
In the corresponding target Delaunay grid of grid withCorresponding grid vertexHorizontal coordinate position,It indicates
In the corresponding target Delaunay grid of affiliated Delaunay grid withCorresponding grid vertexVertical coordinate
Position.
In this particular embodiment, the E in step 8DSCalculating process are as follows:,
Wherein,It indicatesParallax value,It indicatesIn the corresponding target Delaunay grid of affiliated Delaunay grid
WithCorresponding grid vertexHorizontal coordinate position,It indicatesThe affiliated corresponding target of Delaunay grid
In Delaunay grid withCorresponding grid vertexVertical coordinate position,Indicate { R (x, y) } inInstitute
In target Delaunay grid corresponding to the corresponding Delaunay grid of the Delaunay grid of category withCorresponding net
The horizontal coordinate position on lattice vertex,Indicate { R (x, y) } inThe affiliated corresponding Delaunay of Delaunay grid
In target Delaunay grid corresponding to grid withThe vertical coordinate position of corresponding grid vertex,It indicates to remove On
The set that other outer all object exposure masks are constituted, It indicatesIn all Delaunay grids grid
The total number on vertex,It indicatesIn all Delaunay grids all grid vertexes in a grid vertex of t ",It indicatesParallax value,It indicatesIn the corresponding target Delaunay grid of affiliated Delaunay grid with
Corresponding grid vertexHorizontal coordinate position,It indicatesThe affiliated corresponding target of Delaunay grid
In Delaunay grid withCorresponding grid vertexVertical coordinate position,Indicate { R (x, y) } inInstitute
In target Delaunay grid corresponding to the corresponding Delaunay grid of the Delaunay grid of category withCorresponding net
The horizontal coordinate position on lattice vertex,Indicate { R (x, y) } inThe affiliated corresponding Delaunay of Delaunay grid
In target Delaunay grid corresponding to grid withThe vertical coordinate position of corresponding grid vertex,It indicatesIn
All Delaunay grids all grid vertexes constitute set.
Step 9: according to EIQ、ESO、EAOAnd EDS, calculate all Delaunay grids pair in { L (x, y) } and { R (x, y) }
The gross energy for the target Delaunay grid answered, is denoted as Etotal, Etotal=EIQ+λSO×ESO+λAO×EAO+λDS×EDS；Then lead to
Cross Least-squares minimization solutionObtain the corresponding optimum target of each Delaunay grid in { L (x, y) }
The corresponding optimum target Delaunay grid of each Delaunay grid in Delaunay grid and { R (x, y) }, by UL,kIt is right
The optimum target Delaunay grid answered is denoted as By UR,kCorresponding optimum target
Delaunay grid is denoted as Then each Delaunay in { L (x, y) } is calculated
The best similitude transformation matrix of the corresponding optimum target Delaunay grid of grid, willBest similitude transformation matrix be denoted as And it is corresponding most to calculate each Delaunay grid in { R (x, y) }
The best similitude transformation matrix of good target Delaunay grid, willBest similitude transformation matrix be denoted as Wherein, λSOFor ESOWeighting parameters, λAOFor EAOWeighting parameters, λDSFor EDS
Weighting parameters, take λ in the present embodimentSO=4, λAO=20, λDS=2, min () are to be minimized function,Indicate L (x,
Y) set that the corresponding target Delaunay grid of all Delaunay grids in } is constituted,Indicate the institute in { R (x, y) }
The set for thering is the corresponding target Delaunay grid of Delaunay grid to constitute,It is corresponding to indicate?
1 grid vertex, the 2nd grid vertex, the 3rd grid vertex,It is corresponding to indicateThe 1st grid
Vertex, the 2nd grid vertex, the 3rd grid vertex, WithIt is corresponding to indicateLevel
Coordinate position and vertical coordinate position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position, With
It is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicateHorizontal coordinate position
Set with vertical coordinate position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position.
Step 10: most according to the corresponding optimum target Delaunay grid of each Delaunay grid in { L (x, y) }
Good similitude transformation matrix calculates each pixel in each Delaunay grid in { L (x, y) } through best similarity transformation square
Horizontal coordinate position and vertical coordinate position after fractal transform, by UL,kMiddle horizontal coordinate position is x'L,kWith vertical coordinate position
y'L,kPixel through the transformed horizontal coordinate position of best similitude transformation matrix and vertical coordinate position correspondence be denoted as
With Then according to each pixel warp in each Delaunay grid in { L (x, y) }
Horizontal coordinate position and vertical coordinate position after best similarity transformation rectangular transform, the left view point diagram after obtaining recombining contents
Picture is denoted asWherein, 1≤x'L,k≤ W, 1≤y'L,k≤ H, It indicatesMiddle coordinate position is
The pixel value of the pixel of (x', y').
Equally, according to the best of the corresponding optimum target Delaunay grid of each Delaunay grid in { R (x, y) }
Similitude transformation matrix calculates each pixel in each Delaunay grid in { R (x, y) } through best similarity transformation rectangle
Transformed horizontal coordinate position and vertical coordinate position, by UR,kMiddle horizontal coordinate position is x'R,kWith vertical coordinate position
y'R,kPixel through the transformed horizontal coordinate position of best similitude transformation matrix and vertical coordinate position correspondence be denoted as
With Then according to each pixel warp in each Delaunay grid in { R (x, y) }
Horizontal coordinate position and vertical coordinate position after best similarity transformation rectangular transform, the right viewpoint figure after obtaining recombining contents
Picture is denoted asWherein, 1≤x'R,k≤ W, 1≤y'R,k≤ H,
It indicatesMiddle coordinate position is the pixel value of the pixel of (x', y').
The feasibility and validity of method in order to further illustrate the present invention, tests the method for the present invention.
Below with regard to using the method for the present invention to tetra- width stereo-picture of Image1, Image2, Image3 and Image4 carry out in
Hold reconstruction experiment.Fig. 2 a gives " red green " figure of the original three-dimensional image of " Image1 ", and Fig. 2 b gives in " Image1 " warp
" red green " figure after bulk density group；Fig. 3 a gives " red green " figure of the original three-dimensional image of " Image2 ", and Fig. 3 b gives
" red green " figure of " Image2 " after recombining contents；Fig. 4 a gives " red green " figure of the original three-dimensional image of " Image3 ",
Fig. 4 b gives " red green " figure of " Image3 " after recombining contents；Fig. 5 a gives the original three-dimensional image of " Image4 "
" red green " figure, Fig. 5 b give " red green " figure of " Image4 " after recombining contents.As can be seen that adopting from Fig. 2 a to Fig. 5 b
Stereo-picture after the recombining contents obtained with the method for the present invention can preferably object of reservation shape, and can be according to user's
Selection carries out the reorganization operation of location of content and size.
Claims (2)
1. a kind of stereoscopic image content recombination method, it is characterised in that the following steps are included:
Step 1: by left view point image, right visual point image and the left view of the stereo-picture that width to be processed is W and height is H
Difference image correspondence is denoted as { L (x, y) }, { R (x, y) } and { dL(x,y)}；Wherein, 1≤x≤W, 1≤y≤H, L (x, y) indicate { L
(x, y) } in coordinate position be (x, y) pixel pixel value, R (x, y) indicates that coordinate position is (x, y) in { R (x, y) }
The pixel value of pixel, dL(x, y) indicates { dL(x, y) } in coordinate position be (x, y) pixel pixel value；
Step 2: go out the notable figure of { L (x, y) } using the significant model extraction of vision based on graph theory, be denoted as { SML(x,y)}；So
Afterwards according to { SML(x, y) } and { dL(x, y) }, the visual saliency map of { L (x, y) } is obtained, { S is denoted asL(x,y)}；Then according to { SL
(x, y) } and { dL(x, y) }, the visual saliency map of { R (x, y) } is obtained, { S is denoted asR(x,y)}；Wherein, SML(x, y) indicates { SML
(x, y) } in coordinate position be (x, y) pixel pixel value, SL(x, y) indicates { SL(x, y) } in coordinate position be (x, y)
Pixel pixel value, SR(x, y) indicates { SR(x, y) } in coordinate position be (x, y) pixel pixel value；
Step 3: being divided into multiple irregular Delaunay grids not overlapped for { L (x, y) }, will be in { L (x, y) }
K-th of Delaunay grid is denoted as UL,k, UL,kIt is described with set that its 3 grid vertexes are constituted,Then according to all Delaunay grids and { d in { L (x, y) }L(x, y) }, obtain R (x,
Y) all irregular Delaunay grids not overlapped in }, k-th of Delaunay grid in { R (x, y) } is denoted as
UR,k, UR,kIt is described with set that its 3 grid vertexes are constituted,Wherein, k is positive integer, 1≤
The total number for the Delaunay grid for including in k≤M, M expression { L (x, y) }, M > 1,It is corresponding to indicate UL,k's
1st grid vertex, the 2nd grid vertex, the 3rd grid vertex,WithHorizontal coordinate positionIt is sat with vertical
Cursor positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionCome
Description, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, It is corresponding to indicate UR,kThe 1st grid vertex, the 2nd grid vertex, the 3rd grid
Vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, Indicate { dL
(x, y) } in coordinate position bePixel pixel value,WithHorizontal coordinate positionWith it is vertical
Coordinate positionIt describes, Indicate { dL
(x, y) } in coordinate position bePixel pixel value,WithHorizontal coordinate positionWith it is vertical
Coordinate positionIt describes, Indicate { dL
(x, y) } in coordinate position bePixel pixel value；
Step 4: cluster segmentation is carried out to all pixels point in { L (x, y) } using K mean cluster method, obtains { L (x, y) }
In all clustering objects, and then the object exposure mask of each clustering object in { L (x, y) } is obtained, by n-th in { L (x, y) }
The object exposure mask of a clustering object is denoted as On；Setting user has selected n-th of clustering object in { L (x, y) }, then according to { L
(x, y) } in all Delaunay grids, by OnIn kth " a Delaunay grid is denoted as With its 3 nets
The set that lattice vertex is constituted describes,Wherein, n is positive integer, and 1≤n≤N, N indicate { L
(x, y) } in clustering object total number, N > 1, k " be positive integer, 1≤k "≤M ", M " indicate OnIn include Delaunay
The total number of grid, M " > 1,It is corresponding to indicateThe 1st grid vertex, the 2nd grid vertex,
3rd grid vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes,
Step 5: the references object exposure mask that user provides is denoted as P；Then by P be divided into it is multiple do not overlap it is irregular
Kth in P ' a Delaunay grid is denoted as U by Delaunay gridP,k', UP,k'Come with the set that its 3 grid vertexes are constituted
Description,Wherein, k' is positive integer, and 1≤k'≤M', M' indicate the Delaunay net for including in P
The total number of lattice, M'> 1,It is corresponding to indicate UP,k'The 1st grid vertex, the 2nd grid vertex, the 3rd
Grid vertex,WithHorizontal coordinate positionWith vertical coordinate positionIt describes, WithHorizontal coordinate positionWith vertical coordinate positionIt describes, With
Horizontal coordinate positionWith vertical coordinate positionIt describes,
Step 6: on the basis of n-th of clustering object that user has selected in { L (x, y) }, O is calculatednIt is inclined with the object horizontal of P
Shifting amount, object vertical offset and object zoom factor, correspondence are denoted asAnd ρ,Wherein,Indicate horizontal direction,Indicate vertical direction,Indicate OnIn all Delaunay grids all grid vertexes in t-th of grid vertex,It indicatesHorizontal coordinate position,It indicatesVertical coordinate position,Indicate all Delaunay nets in P
The t' grid vertex in all grid vertexes of lattice,It indicatesHorizontal coordinate position,It indicatesVertical seat
Cursor position,Indicate OnIn all Delaunay grids constitute set, also illustrate that OnIn all Delaunay grids
The set that all grid vertexes are constituted, VPIt indicates the set that all Delaunay grids in P are constituted, also illustrates that all in P
The set that all grid vertexes of Delaunay grid are constituted,1≤t'≤NP,Indicate OnIn it is all
The total number of the grid vertex of Delaunay grid, NPIndicate the total number of the grid vertex of all Delaunay grids in P, Symbol " | | " it is the symbol that takes absolute value
Number；
Step 7: each Delaunay grid in { L (x, y) } is corresponding with target Delaunay grid, by UL,kCorresponding target
Delaunay grid is denoted as It is described with set that its 3 grid vertexes are constituted,
Then according to the corresponding target Delaunay grid of all Delaunay grids in { L (x, y) }, to each of { L (x, y) }
Delaunay grid carries out similarity transformation, so that the Delaunay grid of script passes through similar change to the Delaunay grid of script
The mapping fault of the target Delaunay grid obtained after changing is minimum, and each Delaunay grid obtained in { L (x, y) } is corresponding
Target Delaunay grid similitude transformation matrix, willSimilitude transformation matrix be denoted as Wherein,It is corresponding to indicateThe 1st grid vertex, the 2nd
A grid vertex, the 3rd grid vertex,It indicatesI-th of grid vertex, i=1,2,3, WithIt is corresponding to indicateHorizontal coordinate position and vertical sit
Cursor position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicate
Horizontal coordinate position and vertical coordinate position, (AL,k)TFor AL,kTransposition, ((AL,k)TAL,k)-1For (AL,k)TAL,kIt is inverse；
Equally, each Delaunay grid in { R (x, y) } is corresponding with target Delaunay grid, by UR,kCorresponding target
Delaunay grid is denoted as It is described with set that its 3 grid vertexes are constituted,
Then according to the corresponding target Delaunay grid of all Delaunay grids in { R (x, y) }, to each of { R (x, y) }
Delaunay grid carries out similarity transformation, so that the Delaunay grid of script passes through similar change to the Delaunay grid of script
The mapping fault of the target Delaunay grid obtained after changing is minimum, and each Delaunay grid obtained in { R (x, y) } is corresponding
Target Delaunay grid similitude transformation matrix, willSimilitude transformation matrix be denoted as Wherein,It is corresponding to indicateThe 1st grid vertex, the 2nd
A grid vertex, the 3rd grid vertex,It indicatesI-th of grid vertex, i=1,2,3, WithIt is corresponding to indicateHorizontal coordinate position and vertical sit
Cursor position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicate
Horizontal coordinate position and vertical coordinate position, (AR,k)TFor AR,kTransposition, ((AR,k)TAR,k)-1For (AR,k)TAR,kIt is inverse；
Step 8: according to the similarity transformation square of the corresponding target Delaunay grid of each Delaunay grid in { L (x, y) }
The similitude transformation matrix of battle array and the corresponding target Delaunay grid of each Delaunay grid in { R (x, y) }, and combine
{SL(x, y) } and { SR(x, y) }, calculate the corresponding target of all Delaunay grids in { L (x, y) } and { R (x, y) }
The picture quality energy of Delaunay grid, is denoted as EIQ；
According to ρ, all Delaunay grids and { R (x, y) } in { L (x, y) } in n-th of clustering object of user's selection are calculated
In the object of the corresponding target Delaunay grid of corresponding all Delaunay grids scale energy, be denoted as ESO；
According toWithAll Delaunay grids calculated in { L (x, y) } in n-th of clustering object of user's selection are corresponding
Target Delaunay grid position adjust energy, be denoted as EAO；
According to ρ, all Delaunay grids and { R (x, y) } in { L (x, y) } in n-th of clustering object of user's selection are calculated
In the parallax of the corresponding target Delaunay grid of corresponding all Delaunay grids adapt to energy, be denoted as EDS；
E in the step eightIQCalculating process are as follows:
A, the shape for calculating the corresponding target Delaunay grid of all Delaunay grids in { L (x, y) } protects energy, note
For Wherein, SL(k) U is indicatedL,kIn all pixels point vision it is significant
The mean value of value, namely indicate { SL(x, y) } in UL,kThe mean value of the pixel value of all pixels point in corresponding region, symbol
" | | | | " it is to seek Euclidean distance symbol；
Equally, the shape for calculating the corresponding target Delaunay grid of all Delaunay grids in { R (x, y) } protects energy
Amount, is denoted as Wherein, SR(k) U is indicatedR,kIn all pixels point vision
The mean value of saliency value, namely indicate { SR(x, y) } in UR,kThe mean value of the pixel value of all pixels point in corresponding region；
B, the boundary curvature energy of the corresponding target Delaunay grid of all Delaunay grids in { L (x, y) } is calculated,
It is denoted as Wherein, eL,kIndicate UL,kAll grid vertexes
Edge composition matrix,(eL,k)TFor eL,kTransposition, ((eL,k)TeL,k)-1For
(eL,k)TeL,kIt is inverse,It indicatesAll grid vertexes edge composition matrix,
Equally, the boundary curvature energy of the corresponding target Delaunay grid of all Delaunay grids in { R (x, y) } is calculated
Amount, is denoted as Wherein, eR,kIndicate UR,kAll grids
The matrix of the edge composition on vertex,(eR,k)TFor eR,kTransposition, ((eR,k)TeR,k)-1
For (eR,k)TeR,kIt is inverse,It indicatesAll grid vertexes edge composition matrix,
C, basisWithCalculate EIQ,Wherein, λLB
For weighting parameters；
E in the step eightSOCalculating process are as follows:Wherein,Indicate user in { L (x, y) }
The object of the corresponding target Delaunay grid of all Delaunay grids in n-th of clustering object of selection scales energy, It indicates in { R (x, y) } in region corresponding with n-th of clustering object of user's selection
The corresponding target Delaunay grid of all Delaunay grids object scale energy,Symbol " | | | | " it is to ask Euclidean distance symbol, eL,k”Indicate user's selection in { L (x, y) }
N-th of clustering object in kth " a Delaunay grid all grid vertexes edge composition matrix, also indicateAll grid vertexes edge composition matrix,Indicate n-th of clustering object of user's selection in { L (x, y) }
In kth " the corresponding target Delaunay grid of a Delaunay grid all grid vertexes edge composition matrix, also
It indicatesThe matrix of the edge composition of all grid vertexes of corresponding target Delaunay grid, eR,k”It indicates { R (x, y) }
In with the kth in { L (x, y) } in n-th of clustering object of user's selection " the corresponding Delaunay net of a Delaunay grid
All grid vertexes of lattice edge composition matrix, also indicate { R (x, y) } inCorresponding Delaunay grid
All grid vertexes edge composition matrix,Indicate n-th with user's selection in { L (x, y) } in { R (x, y) }
Target Delaunay grid corresponding to the corresponding Delaunay grid of a Delaunay grid of kth in clustering object "
All grid vertexes edge composition matrix, also indicate { R (x, y) } inCorresponding Delaunay grid institute is right
The matrix of the edge composition of all grid vertexes for the target Delaunay grid answered；
E in the step eightAOCalculating process are as follows:Wherein,It indicatesAffiliated Delaunay
In the corresponding target Delaunay grid of grid withCorresponding grid vertexHorizontal coordinate position,It indicates
In the corresponding target Delaunay grid of affiliated Delaunay grid withCorresponding grid vertexVertical coordinate
Position；
E in the step eightDSCalculating process are as follows:,
Wherein,It indicatesParallax value,It indicatesIn the corresponding target Delaunay grid of affiliated Delaunay grid
WithCorresponding grid vertexHorizontal coordinate position,It indicatesThe affiliated corresponding target of Delaunay grid
In Delaunay grid withCorresponding grid vertexVertical coordinate position,Indicate { R (x, y) } inInstitute
In target Delaunay grid corresponding to the corresponding Delaunay grid of the Delaunay grid of category withCorresponding net
The horizontal coordinate position on lattice vertex,Indicate { R (x, y) } inThe affiliated corresponding Delaunay of Delaunay grid
In target Delaunay grid corresponding to grid withThe vertical coordinate position of corresponding grid vertex,It indicates to remove On
The set that other outer all object exposure masks are constituted, It indicatesIn all Delaunay grids grid
The total number on vertex,It indicatesIn all Delaunay grids all grid vertexes in t " a grid vertex,It indicatesParallax value,It indicatesIn the corresponding target Delaunay grid of affiliated Delaunay grid with
Corresponding grid vertexHorizontal coordinate position,It indicatesThe affiliated corresponding target of Delaunay grid
In Delaunay grid withCorresponding grid vertexVertical coordinate position,Indicate { R (x, y) } inInstitute
In target Delaunay grid corresponding to the corresponding Delaunay grid of the Delaunay grid of category withCorresponding net
The horizontal coordinate position on lattice vertex,Indicate { R (x, y) } inThe affiliated corresponding Delaunay of Delaunay grid
In target Delaunay grid corresponding to grid withThe vertical coordinate position of corresponding grid vertex,It indicatesIn
All Delaunay grids all grid vertexes constitute set；
Step 9: according to EIQ、ESO、EAOAnd EDS, calculate { L (x, y) } and { R (x, y) } in all Delaunay grids it is corresponding
The gross energy of target Delaunay grid, is denoted as Etotal, Etotal=EIQ+λSO×ESO+λAO×EAO+λDS×EDS；Then by most
Small two multiply Optimization SolutionObtain the corresponding optimum target of each Delaunay grid in { L (x, y) }
The corresponding optimum target Delaunay grid of each Delaunay grid in Delaunay grid and { R (x, y) }, by UL,kIt is right
The optimum target Delaunay grid answered is denoted as By UR,kCorresponding optimum target
Delaunay grid is denoted as Then each Delaunay in { L (x, y) } is calculated
The best similitude transformation matrix of the corresponding optimum target Delaunay grid of grid, willBest similitude transformation matrix be denoted as And it is corresponding most to calculate each Delaunay grid in { R (x, y) }
The best similitude transformation matrix of good target Delaunay grid, willBest similitude transformation matrix be denoted as Wherein, λSOFor ESOWeighting parameters, λAOFor EAOWeighting parameters, λDSFor EDS
Weighting parameters, min () be minimized function,Indicate the corresponding target of all Delaunay grids in { L (x, y) }
The set that Delaunay grid is constituted,Indicate the corresponding target Delaunay of all Delaunay grids in { R (x, y) }
The set that grid is constituted,It is corresponding to indicateThe 1st grid vertex, the 2nd grid vertex, the 3rd
Grid vertex,It is corresponding to indicateThe 1st grid vertex, the 2nd grid vertex, the 3rd grid top
Point, WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is corresponding to indicateLevel
Coordinate position and vertical coordinate position, WithIt is corresponding to indicateHorizontal coordinate position and hang down
Straight coordinate position,WithIt is corresponding to indicateHorizontal coordinate position and vertical coordinate position,WithIt is right
It should indicateHorizontal coordinate position and vertical coordinate position；
Step 10: according to the best phase of the corresponding optimum target Delaunay grid of each Delaunay grid in { L (x, y) }
Like transformation matrix, each pixel calculated in each Delaunay grid in { L (x, y) } becomes through best similarity transformation rectangle
Horizontal coordinate position and vertical coordinate position after changing, by UL,kMiddle horizontal coordinate position is x'L,kWith vertical coordinate position y'L,k
Pixel through the transformed horizontal coordinate position of best similitude transformation matrix and vertical coordinate position correspondence be denoted asWith Then according to each pixel in each Delaunay grid in { L (x, y) } through most
Horizontal coordinate position and vertical coordinate position after good similarity transformation rectangular transform, the left view point image after obtaining recombining contents,
It is denoted asWherein, 1≤x'L,k≤ W, 1≤y'L,k≤ H,1≤x'≤W, 1
≤ y'≤H,It indicatesMiddle coordinate position is the pixel value of the pixel of (x', y')；
Equally, according to the best similar of the corresponding optimum target Delaunay grid of each Delaunay grid in { R (x, y) }
Transformation matrix calculates each pixel in each Delaunay grid in { R (x, y) } through best similarity transformation rectangular transform
Horizontal coordinate position and vertical coordinate position afterwards, by UR,kMiddle horizontal coordinate position is x'R,kWith vertical coordinate position y'R,k's
Pixel is denoted as through the transformed horizontal coordinate position of best similitude transformation matrix and vertical coordinate position correspondenceWith Then according to each pixel in each Delaunay grid in { R (x, y) } through best similar
Horizontal coordinate position and vertical coordinate position after converting rectangular transform, the right visual point image after obtaining recombining contents, are denoted asWherein, 1≤x'R,k≤ W, 1≤y'R,k≤ H, It indicatesMiddle coordinate position is the pixel value of the pixel of (x', y').
2. a kind of stereoscopic image content recombination method according to claim 1, it is characterised in that in the step two,SR(x, y)=SL(x+dL(x,y),y)；Wherein,Indicate SML(x,y)
Weight,Indicate dLThe weight of (x, y),SL(x+dL(x, y), y) indicate { SL(x, y) } in coordinate position be
(x+dL(x, y), y) pixel pixel value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810555934.XA CN108833876B (en) | 2018-06-01 | 2018-06-01 | A kind of stereoscopic image content recombination method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810555934.XA CN108833876B (en) | 2018-06-01 | 2018-06-01 | A kind of stereoscopic image content recombination method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108833876A CN108833876A (en) | 2018-11-16 |
| CN108833876B true CN108833876B (en) | 2019-10-25 |
Family
ID=64145721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810555934.XA Active CN108833876B (en) | 2018-06-01 | 2018-06-01 | A kind of stereoscopic image content recombination method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108833876B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859300B (en) * | 2019-01-24 | 2022-12-23 | 宁波大学 | Three-dimensional image content editing method based on aesthetics |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101840574B (en) * | 2010-04-16 | 2012-05-23 | 西安电子科技大学 | Depth estimation method based on edge pixel features |
| KR101315398B1 (en) * | 2010-12-28 | 2013-10-07 | 주식회사 팬택 | Apparatus and method for display 3D AR information |
| CN102695070B (en) * | 2012-06-12 | 2014-06-04 | 浙江大学 | Depth consistency fusion processing method for stereo image |
| TWI529662B (en) * | 2014-12-15 | 2016-04-11 | 財團法人工業技術研究院 | Depth information-based modeling method, graphic processing apparatus and storage medium |
| CN104616286B (en) * | 2014-12-17 | 2017-10-31 | 浙江大学 | Quick semi-automatic multi views depth restorative procedure |
-
2018
- 2018-06-01 CN CN201810555934.XA patent/CN108833876B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108833876A (en) | 2018-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110324664A (en) | A kind of video neural network based mends the training method of frame method and its model | |
| CN105574922B (en) | A kind of threedimensional model high quality texture mapping method of robust | |
| CN107105214B (en) | A kind of 3 d video images method for relocating | |
| CN103927727B (en) | Method for converting scalar image into vector image | |
| CN108022223B (en) | Tone mapping method based on logarithm mapping function blocking processing fusion | |
| CN106570900B (en) | A kind of stereo-picture method for relocating | |
| CN111047709B (en) | Binocular vision naked eye 3D image generation method | |
| CN103761734A (en) | Binocular stereoscopic video scene fusion method for keeping time domain consistency | |
| CN110634147A (en) | Image matting method based on bilateral boot up-sampling | |
| CN109410133A (en) | A kind of face texture repairing method based on 3DMM | |
| CN102930518A (en) | Improved sparse representation based image super-resolution method | |
| CN103761736B (en) | A kind of image partition method based on Bayes's harmony degree | |
| CN105376553B (en) | A kind of 3 D video method for relocating | |
| CN104751508B (en) | The full-automatic of new view is quickly generated and complementing method in the making of 3D three-dimensional films | |
| CN108833876B (en) | A kind of stereoscopic image content recombination method | |
| CN107945151A (en) | A kind of reorientation image quality evaluating method based on similarity transformation | |
| CN105427242B (en) | Interactive gridding restrained deformation image adaptive Zoom method based on perception of content | |
| CN108307170B (en) | A kind of stereo-picture method for relocating | |
| CN108810512B (en) | A kind of object-based stereo-picture depth method of adjustment | |
| CN111105350B (en) | Real-time video splicing method based on self homography transformation under large parallax scene | |
| CN109583450A (en) | Salient region detecting method based on feedforward neural network fusion vision attention priori | |
| CN111010605B (en) | Method for displaying video picture-in-picture window | |
| CN112449170B (en) | Stereo video repositioning method | |
| Qi et al. | Shift-map based stereo image retargeting with disparity adjustment | |
| CN107918926B (en) | A kind of digital image magnification method based on quadrangle and triangle surface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230919 Address after: Room 2202, 22 / F, Wantong building, No. 3002, Sungang East Road, Sungang street, Luohu District, Shenzhen City, Guangdong Province Patentee after: Shenzhen dragon totem technology achievement transformation Co.,Ltd. Address before: 315211, Fenghua Road, Jiangbei District, Zhejiang, Ningbo 818 Patentee before: Ningbo University |
|
| TR01 | Transfer of patent right |