CN107147894B - A kind of virtual visual point image generating method in Auto-stereo display - Google Patents
A kind of virtual visual point image generating method in Auto-stereo display Download PDFInfo
- Publication number
- CN107147894B CN107147894B CN201710226879.5A CN201710226879A CN107147894B CN 107147894 B CN107147894 B CN 107147894B CN 201710226879 A CN201710226879 A CN 201710226879A CN 107147894 B CN107147894 B CN 107147894B
- Authority
- CN
- China
- Prior art keywords
- image
- visual point
- virtual visual
- point image
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention proposes the virtual visual point image generating method in a kind of Auto-stereo display.This method includes that virtual visual point image generates and free stereo image two processes of synthesis.According to multi-view free stereoscopic displayer display parameters, determine the positional relationship between multiple views anaglyph, camera parameter is adjusted, the virtual visual point image of specific position is obtained, is shown using reference view image and virtual visual point image synthesis free stereo image and on auto-stereoscopic display.The present invention consider virtual view synthesis between prospect and background block with the discontinuous problem of boundary depth value, improve the quality of the virtual visual point image of generation.
Description
Technical field
The present invention relates to free 3 D display technology, in particular to the virtual visual point image in a kind of Auto-stereo display is raw
At method.
Background technique
Free 3 D display technology is related to the acquisition of multiple views anaglyph and free stereo image synthetic technology.Multiple views view
Difference image can be obtained by synchronous camera or 3D modeling software, and free stereo image is pressed by the multiple views anaglyph obtained
Certain method is synthesized.It is relatively more high using the price of multiple synchronous cameras in the acquisition methods of multiple views anaglyph
It is expensive, it is very high using the time cost of modeling software.Virtual visual point synthesizing method can use two reference view texture images and its
Corresponding depth image generates virtual visual point image by way of three-dimensional mapping, by the adjustment of camera parameter, can obtain
The virtual visual point image of any position between two reference views.Then it is closed using two reference view texture images and virtual visual point image
At free stereo image.However traditional virtual visual point synthesizing method there are depth image edge depth values discontinuous, prospect back
The problems such as blocking between scape has a great impact to the quality of virtual visual point image.
Summary of the invention
The present invention is directed to realize the Auto-stereo display of high quality.In order to reach the purpose, the present invention proposes a kind of freedom
Virtual visual point image generating method in stereoscopic display.This method includes that virtual visual point image generates and free stereo image synthesis
Two processes, detailed process are as shown in Fig. 1.In virtual visual point image generation, first forward to reference view depth image
Mapping generates virtual depth image, is then filled out using the method that background is filled to the cavity in the virtual depth image of generation
It fills, is finally mapped according to the virtual depth image of filling and generate virtual visual point image.During this, to depth image after mapping into
The method of row background filling has been obviously improved the quality of virtual visual point image.In free stereo image synthesis process, root first
According to the display parameters of Auto-stereo display, the positional relationship between viewpoint number and multiple views anaglyph is determined, so that it is determined that
Then the position for the virtual visual point image for needing to generate adjusts camera parameter and generates virtual visual point image, multiple views anaglyph
It is made of the virtual visual point image of reference view image and specific position, finally synthesizes free stereo using multiple views anaglyph
Image.During this, multiple views anaglyph simply and rapidly can be generated using virtual view composition algorithm.
The virtual visual point image generating process, the geological information of scene is expressed using depth information and camera parameter,
It, as shown in Fig. 2, first will by three-dimension varying equation using the above geological information for the quasi- virtual visual point image generated
In the pixel projection of reference picture to the image coordinate system of target view, to generate the image of target view.Reference picture
Each pixel is from two-dimensional surface back projection to three-dimensional space:
(1)
(2)
(x, y, z )TIndicate the point of three-dimensional space, then the point of three-dimensional space is by re-projection to target view plane,
(u,v,1)TIt is the homogeneous coordinates of pixel in reference picture,K r, R rWitht rRespectively indicate internal matrix, the rotation of reference picture
Matrix, translation matrix,K v,R vWitht vInternal matrix, spin matrix, the translation matrix of virtual visual point image are respectively indicated,dIt indicates
(u, v) at depth value, coordinate (l, m, n) indicate virtual view in rounded coordinate.
However due to the problems such as being blocked between there are edge depth values in depth image discontinuous, prospect background, mapping life
At virtual depth image in exist cavity.The cavity in the virtual depth image of generation is carried out using the method for background filling
It fills, the depth value that prospect is belonged in depth image is greater than the depth value of background,d (i, j) expression pixel (i, j) at
Depth value indicates that this pixel is sky pixel when it is 0, utilizes empty pixel near zone ΩγIn non-zero minimum depth valued
(m, n) rightd(i, j) it is filled:
(3)
Whereind(m, n) indicate region Ω in pixel (m, n) at depth value,γIndicate pixel and empty picture in the Ω of region
The distance between element,γ=1,2 ..., 9, value is carried out by the subjective evaluation quality of the virtual visual point image generated
Screening.Corresponding virtual visual point image, final virtual visual point image are then generated according to the mapping of filled depth imageI v
(u, v) by the virtual visual point image after the mapping of left and right reference viewI l(u l , v l)、I r(u r, v r) synthesis:
(4)
(5)
WhereinαIndicate weighted factor,t l、t rRespectively indicate the translation matrix of left and right reference view external parameter.
The free stereo image synthesis process, by taking 3 tunnel anaglyphs as an example, as the small rectangle of attached drawing 3, proper alignment is
1 sub-pixel on display screen, 3 sub-pixels constitute 1 pixel, and 1,2,3 in Fig. 3 respectively represent 3 tunnel anaglyphs.If light
Grid cycle isp, grating slope angle isθ, 2D display screen sub-pixel width isW p, used viewpoint number isK, this 4 parameters
It need to appropriately match, meet following relationship:
(6)
In formula,ceil() expression rounds up.Determine viewpoint numberKAnd tilt angleθAfterwards, so that it may willKWidth anaglyph is closed
At a width free stereo image.KThe conjunction primitive reason of road anaglyph is identical as principle as shown in Fig. 3, aobvious according to free stereo
The display parameters for showing device determine the positional relationship between multiple views anaglyph, so that it is determined that the virtual view figure for needing to generate
Then the position of picture adjusts camera parameter and generates virtual visual point image.Camera parameterA n、R n、t nRespectively cameraC n Inside square
Battle array, spin matrix, translation matrix, whereinnIndicate camera label.Two reference view cameras areC 0、C 1, corresponding spin matrix difference
ForR 0、R 1；Corresponding to translation matrix is respectivelyt 0、t 1；Internal matrix isA 0、A 1.Two reference view middle positionsC', translation matrixt(c') bet 0Witht 1Average value, internal matrixA(c') withA 0WithA 1Keep identical, spin matrix isR(c') by following
Formula acquires:
(7)
(8)
WhereinRFor the spin matrix factor of introducing, any two viewpoint middle position can be calculated by formula (7), (8)
Camera rotation parameter, it is possible thereby to which iteration goes out the camera rotation parameter of any positionR n, andA nIt remains unchanged,t nFor two with reference to view
The mean value of point, to generate multiple views anaglyph using virtual view composition algorithm.
Four, Detailed description of the invention
Attached drawing 1 is a kind of flow chart of the virtual visual point image generating method in Auto-stereo display
Geometrical relationship schematic diagram of the attached drawing 2 between reference view and target view
Attached drawing 3 is free stereo image composition principle schematic diagram
Attached drawing 4 is that the mapping of unfilled virtual depth image generates virtual visual point image
Attached drawing 5 is that the mapping of filled virtual depth image generates virtual visual point image
Shown by reference numeral in above-mentioned attached drawing are as follows:
1 reference view plane coordinate system, 2 three-dimensional space, 3 target view plane coordinate systems, 4 Lenticular screens, 5 composite diagrams
As sub-pixel.
It should be understood that above-mentioned attached drawing is only schematical, it is not drawn to draw.
Five, specific embodiment
The following detailed description of a typical case of the virtual visual point image generating method in a kind of Auto-stereo display of the present invention
The present invention is further described specifically in embodiment.It is necessarily pointed out that following embodiment is served only for the present invention
It is described further, should not be understood as limiting the scope of the invention, field person skilled in the art is according to above-mentioned
Summary of the invention makes some nonessential modifications and adaptations to the present invention, still falls within protection scope of the present invention.
The present invention is directed to realize the Auto-stereo display of high quality.In order to reach the purpose, the present invention proposes a kind of freedom
Virtual visual point image generating method in stereoscopic display.This method includes that virtual visual point image generates and free stereo image synthesis
Two processes, detailed process are as shown in Fig. 1.In virtual visual point image generation, first forward to reference view depth image
Mapping generates virtual depth image, is then filled out using the method that background is filled to the cavity in the virtual depth image of generation
It fills, is finally mapped according to the virtual depth image of filling and generate virtual visual point image.During this, to depth image after mapping into
The method of row background filling has been obviously improved the quality of virtual visual point image.In free stereo image synthesis process, root first
According to the display parameters of Auto-stereo display, the positional relationship between viewpoint number and multiple views anaglyph is determined, so that it is determined that
Then the position for the virtual visual point image for needing to generate adjusts camera parameter and generates virtual visual point image, multiple views anaglyph
It is made of the virtual visual point image of reference view image and specific position, finally synthesizes free stereo using multiple views anaglyph
Image.During this, multiple views anaglyph simply and rapidly can be generated using virtual view composition algorithm.
The virtual visual point image generating process, the geological information of scene is expressed using depth information and camera parameter,
It, as shown in Fig. 2, first will by three-dimension varying equation using the above geological information for the quasi- virtual visual point image generated
In the pixel projection of reference picture to the image coordinate system of target view, to generate the image of target view.Reference picture
Each pixel is from two-dimensional surface back projection to three-dimensional space:
(1)
(2)
(x, y, z )TIndicate the point of three-dimensional space, then the point of three-dimensional space is by re-projection to target view plane,
(u,v,1)TIt is the homogeneous coordinates of pixel in reference picture,K r, R rWitht rRespectively indicate internal matrix, the rotation of reference picture
Matrix, translation matrix,K v,R vWitht vInternal matrix, spin matrix, the translation matrix of virtual visual point image are respectively indicated,dIt indicates
(u, v) at depth value, coordinate (l, m, n) indicate virtual view in rounded coordinate.
However due to the problems such as being blocked between there are edge depth values in depth image discontinuous, prospect background, mapping life
At virtual depth image in exist cavity.The cavity in the virtual depth image of generation is carried out using the method for background filling
It fills, the depth value that prospect is belonged in depth image is greater than the depth value of background,d (i, j) expression pixel (i, j) at
Depth value indicates that this pixel is sky pixel when it is 0, utilizes empty pixel near zone ΩγIn non-zero minimum depth valued
(m, n) rightd(i, j) it is filled:
(3)
Whereind(m, n) indicate region Ω in pixel (m, n) at depth value,γIndicate pixel and empty picture in the Ω of region
The distance between element,γ=1,2 ..., 9, value is carried out by the subjective evaluation quality of the virtual visual point image generated
Screening.Corresponding virtual visual point image, final virtual visual point image are then generated according to the mapping of filled depth imageI v
(u, v) by the virtual visual point image after the mapping of left and right reference viewI l(u l , v l)、I r(u r, v r) synthesis:
(4)
(5)
WhereinαIndicate weighted factor,t l、t rRespectively indicate the translation matrix of left and right reference view external parameter.
The free stereo image synthesis process, by taking 3 tunnel anaglyphs as an example, as the small rectangle of attached drawing 3, proper alignment is
1 sub-pixel on display screen, 3 sub-pixels constitute 1 pixel, and 1,2,3 in Fig. 3 respectively represent 3 tunnel anaglyphs.If light
Grid cycle isp, grating slope angle isθ, 2D display screen sub-pixel width isW p, used viewpoint number isK, this 4 parameters
It need to appropriately match, meet following relationship:
(6)
In formula,ceil() expression rounds up.Determine viewpoint numberKAnd tilt angleθAfterwards, so that it may willKWidth anaglyph is closed
At a width free stereo image.KThe conjunction primitive reason of road anaglyph is identical as principle as shown in Fig. 3, for 28 cun of 9 road parallax
Stereoscopic display, 9 subpixel periods are covered under screen periods, are provided in two reference views with Microsoft Research, Asia
For viewpoint 3 and 5 in " breakdancers ", camera parameter is adjusted, successively grey iterative generation 7 virtually regards among two viewpoints
Point image, two width reference view images and seven width virtual visual point images constitute nine tunnel anaglyphs.Camera parameterA n、R n、t nRespectively
For cameraC nInternal matrix, spin matrix, translation matrix, whereinnIndicate camera label.Two reference view cameras areC 0、C 1,
The corresponding spin matrix of corresponding spin matrixR 0、R 1；Corresponding translation matrixt 0、t 1；Internal matrixA 0、A 1.Two reference views
Middle positionC', translation matrixt(c') bet 0Witht 1Average value, internal matrixA(c') withA 0WithA 1Keep identical,
Spin matrix isR(c') acquired by following formula:
(7)
(8)
WhereinRFor the spin matrix factor of introducing, any two viewpoint middle position can be calculated by formula (7), (8)
Camera rotation parameter, it is possible thereby to which iteration goes out the camera rotation parameter of any positionR n, andA nIt remains unchanged,t nFor two with reference to view
The mean value of point, to generate multiple views anaglyph using virtual view composition algorithm.
The present invention is generated by virtual visual point image and free stereo image synthesizes two processes, vertical according to multi-view auto
Body display display parameters determine the positional relationship between multiple views anaglyph, adjust camera parameter, obtain specific position
Virtual visual point image, using reference view image and virtual visual point image synthesis free stereo image and in auto-stereoscopic display
It is shown above.The present invention considers in virtual view synthesis blocking and boundary depth value is discontinuous asks between prospect background
Topic, improves the quality of the virtual visual point image of generation.
Claims (1)
1. the virtual visual point image generating method in a kind of Auto-stereo display, which is characterized in that this method includes virtual view
Image generates and free stereo image synthesizes two processes；In virtual visual point image generation, reference view depth image is forward
It is mapped to virtual depth image, the cavity in the virtual depth image of generation is filled using the method that background is filled, it is deep
The depth value for belonging to prospect in degree image is greater than the depth value for belonging to background, and d (i, j) indicates the depth value at pixel (i, j), when
It indicates that this pixel is sky pixel for 0, utilizes empty pixel near zone ΩγIn non-zero minimum depth value d (m, n) to d (i,
J) it is filled, wherein γ indicates the distance between pixel and empty pixel in empty pixel near zone Ω, γ=1,2 ..., 9, d
(m, n) indicates the depth value in the Ω of region at pixel (m, n), by the virtual visual point image of generation it is subjective and objective on evaluation quality
The value of γ is screened；In free stereo image synthesis process, first according to the display parameters of Auto-stereo display, determine
Then the position for the virtual visual point image for needing to generate adjusts camera parameter and generates virtual visual point image, multiple views anaglyph
It is made of the virtual visual point image of reference view image and specific position, finally synthesizes free stereo using multiple views anaglyph
Image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710226879.5A CN107147894B (en) | 2017-04-10 | 2017-04-10 | A kind of virtual visual point image generating method in Auto-stereo display |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710226879.5A CN107147894B (en) | 2017-04-10 | 2017-04-10 | A kind of virtual visual point image generating method in Auto-stereo display |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107147894A CN107147894A (en) | 2017-09-08 |
| CN107147894B true CN107147894B (en) | 2019-07-30 |
Family
ID=59775292
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710226879.5A Active CN107147894B (en) | 2017-04-10 | 2017-04-10 | A kind of virtual visual point image generating method in Auto-stereo display |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107147894B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109961406B (en) * | 2017-12-25 | 2021-06-25 | 深圳市优必选科技有限公司 | Image processing method and device and terminal equipment |
| US20190281273A1 (en) * | 2018-03-08 | 2019-09-12 | Mediatek Inc. | Adaptive loop filtering method for reconstructed projection-based frame that employs projection layout of 360-degree virtual reality projection |
| CN108810511B (en) * | 2018-06-21 | 2019-08-30 | 华中科技大学 | A kind of multiple views compression depth video enhancement method based on viewpoint consistency |
| CN108769662B (en) | 2018-07-03 | 2020-01-07 | 京东方科技集团股份有限公司 | Multi-view naked eye 3D image hole filling method and device and electronic equipment |
| CN110246146B (en) * | 2019-04-29 | 2021-07-30 | 北京邮电大学 | Full-parallax light field content generation method and device based on multiple-time depth image rendering |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102325259A (en) * | 2011-09-09 | 2012-01-18 | 青岛海信数字多媒体技术国家重点实验室有限公司 | Method and device for synthesizing virtual viewpoints in multi-viewpoint video |
| CN102568026A (en) * | 2011-12-12 | 2012-07-11 | 浙江大学 | Three-dimensional enhancing realizing method for multi-viewpoint free stereo display |
| CN102714748A (en) * | 2010-03-12 | 2012-10-03 | 石保罗 | Stereoscopic image generation method and a device therefor |
| CN102831602A (en) * | 2012-07-26 | 2012-12-19 | 清华大学 | Image rendering method and image rendering device based on depth image forward mapping |
| CN104851129A (en) * | 2015-05-21 | 2015-08-19 | 成都绿野起点科技有限公司 | Multi-view-based 3D reconstruction method |
| CN105163104A (en) * | 2015-08-31 | 2015-12-16 | 四川虹微技术有限公司 | Intermediate view synthesis method without generating cavity |
-
2017
- 2017-04-10 CN CN201710226879.5A patent/CN107147894B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102714748A (en) * | 2010-03-12 | 2012-10-03 | 石保罗 | Stereoscopic image generation method and a device therefor |
| CN102325259A (en) * | 2011-09-09 | 2012-01-18 | 青岛海信数字多媒体技术国家重点实验室有限公司 | Method and device for synthesizing virtual viewpoints in multi-viewpoint video |
| CN102568026A (en) * | 2011-12-12 | 2012-07-11 | 浙江大学 | Three-dimensional enhancing realizing method for multi-viewpoint free stereo display |
| CN102831602A (en) * | 2012-07-26 | 2012-12-19 | 清华大学 | Image rendering method and image rendering device based on depth image forward mapping |
| CN104851129A (en) * | 2015-05-21 | 2015-08-19 | 成都绿野起点科技有限公司 | Multi-view-based 3D reconstruction method |
| CN105163104A (en) * | 2015-08-31 | 2015-12-16 | 四川虹微技术有限公司 | Intermediate view synthesis method without generating cavity |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107147894A (en) | 2017-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107147894B (en) | A kind of virtual visual point image generating method in Auto-stereo display | |
| Bertel et al. | Megaparallax: Casual 360 panoramas with motion parallax | |
| US7643025B2 (en) | Method and apparatus for applying stereoscopic imagery to three-dimensionally defined substrates | |
| CN102325259A (en) | Method and device for synthesizing virtual viewpoints in multi-viewpoint video | |
| CN108513123B (en) | Image array generation method for integrated imaging light field display | |
| US20130187910A1 (en) | Conversion of a digital stereo image into multiple views with parallax for 3d viewing without glasses | |
| JP4828506B2 (en) | Virtual viewpoint image generation device, program, and recording medium | |
| CN103974055A (en) | 3D photo creation system and method | |
| WO2013158784A1 (en) | Systems and methods for improving overall quality of three-dimensional content by altering parallax budget or compensating for moving objects | |
| JP2008090617A (en) | Device, method and program for creating three-dimensional image | |
| US20060171028A1 (en) | Device and method for display capable of stereoscopic vision | |
| TW201618042A (en) | Method and apparatus for generating a three dimensional image | |
| CN102447934A (en) | Synthetic method of stereoscopic elements in combined stereoscopic image system collected by sparse lens | |
| CN109791704B (en) | Texture rendering method, system and device based on multi-layer UV mapping for free-running FVV application | |
| US9196080B2 (en) | Medial axis decomposition of 2D objects to synthesize binocular depth | |
| WO2012140397A2 (en) | Three-dimensional display system | |
| CA2540538C (en) | Stereoscopic imaging | |
| CN106878692A (en) | A kind of method for displaying three-dimensional object thereby that is blocked based on Fourier spectrum | |
| KR20170025214A (en) | Method for Multi-view Depth Map Generation | |
| JP2014042238A (en) | Apparatus and method for depth-based image scaling of 3d visual content | |
| CN104284177A (en) | Convergence stereo image parallax control method | |
| CN110149508A (en) | A kind of array of figure generation and complementing method based on one-dimensional integrated imaging system | |
| CN102780900B (en) | Image display method of multi-person multi-view stereoscopic display | |
| Paradiso et al. | A novel interpolation method for 3D view synthesis | |
| CN105578170B (en) | A kind of micro- pattern matrix directionality mapping method of integration imaging based on depth data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |