CN107545537A - A kind of method from dense point cloud generation 3D panoramic pictures - Google Patents
A kind of method from dense point cloud generation 3D panoramic pictures Download PDFInfo
- Publication number
- CN107545537A CN107545537A CN201710716319.8A CN201710716319A CN107545537A CN 107545537 A CN107545537 A CN 107545537A CN 201710716319 A CN201710716319 A CN 201710716319A CN 107545537 A CN107545537 A CN 107545537A
- Authority
- CN
- China
- Prior art keywords
- video camera
- under
- pixel
- panoramic pictures
- panoramic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a kind of method from dense point cloud generation 3D panoramic pictures, utilize 360 degree of panoramic pictures and corresponding 360 degree of depth images, by disposing a virtual video camera to each orientation, render the correct picture that the virtual video camera should be seen that, after a Zhang Quanjing image is got with its corresponding depth image, a pair of 3D panoramic pictures can be generated, correct parallax relation can be produced for arbitrary orientation, depth of field sense will not change with the difference of viewing angle and original camera angles, the vertical parallax for causing human eye uncomfortable will not be produced, so as to greatly improve feeling of immersion and comfort level when utilizing VR helmets viewing panorama 3D.In addition, also the parallax range B of virtual camera according to photographed scene size, can be adjusted and reach different degrees of 3D impressions.
Description
Technical field
The invention belongs to 3D Integral Imagings field, and in particular to a kind of from the side of dense point cloud generation 3D panoramic pictures
Method.
Background technology
At present, virtual reality (Virtual Reality) market is burning hot.Wherein, panoramic video is due to its low manufacture cost,
Cycle is short, turns into one of product that VR markets are most landed soon.The making of panoramic video is typically (to be more than or equal to by multiple video cameras
2) while scene is shot, then later stage splicing is carried out, the panorama of a secondary 360x180 degree is obtained, user utilizes VR
When the helmet is watched, head can be arbitrarily rotated, to obtain the vision content of whole scene.However, in order to improve VR scenes
Feeling of immersion, it is a kind of more for attraction content be 360 degree of 3D panoramic video, 360 3D panoramic videos allow user
In the case of any rotation head, depth of field sense is additionally provided so that user obtains 360 degree of three-dimensional video tastes.
It can shoot in 360 3D scheme on the market at present, 2 cameras typically are placed to same direction to simulate people
Body right and left eyes produce parallax information, so that brain can obtain depth of field sense.However, two problems can be brought by so doing：
1st, the part for there was only straight-on camera can obtain depth of field sense, and the depth of field sense in remaining direction will fade away；
2nd, when being introduced into vertical parallax on non-camera face direction, right and left eyes image can produce in non-camera face orientation
Raw vertical parallax, cause human eye uncomfortable.
Therefore, the correct 3D relations on 360 degree of directions to be realized, it is necessary to each orientation, all ensure that stereoscopic camera aligns
To the orientation, i.e.,：Around stereoscopic camera to base center rotating camera, each row pixel is all carried out using the image of different azimuth
Splicing.In theory, for secondary a width of 4096 pixels isometric projection (Equirectangular) panoramic picture, it is necessary to clap
Take the photograph the stereo-picture in 4096 orientation, i.e., everyAll shoot a pair of stereoscopic photographs.This will cause data volume to increase
Greatly, the heavy redundancy of later stage splicing operation, the drawbacks such as video can not be shot.
The content of the invention
It is an object of the invention to provide one kind by obtaining dense scene three-dimensional structure, have to produce on arbitrary orientation
There is the method for the 3D panoramic pictures of correct parallax relation, wear-type VR equipment viewing 3D panorama contents are being utilized to improve user
When feeling of immersion and comfort level.
To achieve the above object, the present invention provides following technical scheme：It is a kind of to generate 3D panoramic pictures from dense point cloud
Method, comprise the following steps：
S1, the three-dimensional structure of scene obtain：It includes active mode acquisition and passive mode obtains, and active mode is obtained and adopted
Scene is scanned with laser radar, structure light depth camera or ToF depth cameras, obtains the depth image corresponding to it；
Passive mode obtains to be shot by video camera array, and the depth of each pixel value in scene is directly calculated using multi-view geometry principle
Degree, obtains the depth image corresponding to it；Depth image is the dense three-dimensional structure for representing scene；
S2, generate 3D rendering：With reference to panoramic picture and three-dimensional scene structure, using 360 degree of panoramic pictures and correspondingly
360 degree of depth images, by each orientation dispose a virtual video camera, render what the virtual video camera should be seen that
Correct picture, 360 degree 3D renderings of the generation with correct parallax information；
S3, generate 3D panoramic pictures：When having the pixel projection in multiple original panoramic pictures in S2 to virtual view
In same pixel when, show the movement due to virtual video camera, object nearby has sheltered from background, now takes distance former
The smaller pixel of point distance is final pixel value；After getting Zhang Quanjing image depth image corresponding with its, you can raw
3D panoramic pictures in a pair.
Preferably, original panoramic video camera is world coordinate system origin defined in the S2, panoramic pictures a width of w, a height ofThen its spherical radius represented isFor the pixel p (u, v) in panoramic picture, pass through query depth figure
Value as in, it is t to obtain its corresponding depth value；
Under isometric projection projection model, three dimensions point coordinate under world coordinate system is corresponding to pixel p (u, v)WhereinFor two incidences under spherical coordinate system
Angle, in units of radian；
If the parallax range between right and left eyes virtual video camera is B, corresponding to the camera represented by the pixel
Orientation θ, now coordinates of the spatial point X under virtual left video camera beVirtual right is taken the photograph
Coordinate under camera
For the spatial point X under left video cameraL=(xL,yL,zL), it is apart from the distance of origin
Pixel coordinate under its corresponding left eye panoramic picture isWhereinθL=atan2
(xL,zL) be spherical coordinate system under two incidence angles, in units of radian；
For the spatial point X under right video cameraR=(xR,yR,zR), it is apart from the distance of origin
Pixel coordinate under its corresponding left eye panoramic picture isWhereinθR=
atan2(xR,zR) be spherical coordinate system under two incidence angles, in units of radian.
The technique effect and advantage of the present invention：
1) each orientation corresponding to its visible top surface, it is by correct virtualphase seat in the plane when user's rotation head
The virtual visual point image rendered according to scene structure is put, therefore, there is correct parallax information, user can obtain truer
The depth of field impression.
2) by adjusting parallax range B, in that context it may be convenient to the depth of field sense of the 3D panoramic pictures of generation is adjusted, to adapt to difference
The interpupillary distance of crowd, obtain optimal viewing experience.
3) there is no vertical parallax, user's ophthalmic uncomfortable will not be caused, the 360 degree of 3D distant view photographs generated by the program
There is no vertical parallax.
Embodiment
The technical scheme in the embodiment of the present invention will be clearly and completely described below, it is clear that described implementation
Example only part of the embodiment of the present invention, rather than whole embodiments.It is common based on the embodiment in the present invention, this area
The every other embodiment that technical staff is obtained under the premise of creative work is not made, belong to the model that the present invention protects
Enclose.
Embodiment
A kind of method from dense point cloud generation 3D panoramic pictures, comprise the following steps：
S1, the three-dimensional structure of scene obtain：It includes active mode acquisition and passive mode obtains, and active mode is obtained and adopted
Scene is scanned with laser radar, structure light depth camera or ToF depth cameras, obtains the depth image corresponding to it；
Passive mode obtains to be shot by video camera array, and the depth of each pixel value in scene is directly calculated using multi-view geometry principle
Degree, obtains the depth image corresponding to it；Depth image is the dense three-dimensional structure for representing scene；
S2, generate 3D rendering：With reference to panoramic picture and three-dimensional scene structure, using 360 degree of panoramic pictures and correspondingly
360 degree of depth images, by each orientation dispose a virtual video camera, render what the virtual video camera should be seen that
Correct picture, 360 degree 3D renderings of the generation with correct parallax information；
It is world coordinate system origin to define original panoramic video camera, panoramic pictures a width of w, a height ofThen it is represented
Spherical radius isFor the pixel p (u, v) in panoramic picture, by the value in query depth image, it is right to obtain its
The depth value answered is t；
Under isometric projection projection model, three dimensions point coordinate under world coordinate system is corresponding to pixel p (u, v)WhereinFor two incidence angles under spherical coordinate system,
In units of radian；
If the parallax range between right and left eyes virtual video camera is B, corresponding to the camera represented by the pixel
Orientation θ, now coordinates of the spatial point X under virtual left video camera beVirtual right is taken the photograph
Coordinate under camera
For the spatial point X under left video cameraL=(xL,yL,zL), it is apart from the distance of origin
Pixel coordinate under its corresponding left eye panoramic picture isWhereinθL=atan2
(xL,zL) be spherical coordinate system under two incidence angles, in units of radian；
For the spatial point X under right video cameraR=(xR,yR,zR), it is apart from the distance of origin
Pixel coordinate under its corresponding left eye panoramic picture isWhereinθR=
atan2(xR,zR) be spherical coordinate system under two incidence angles, in units of radian；
S3, generate 3D panoramic pictures：When having the pixel projection in multiple original panoramic pictures in S2 to virtual view
In same pixel when, show the movement due to virtual video camera, object nearby has sheltered from background, now takes distance former
The smaller pixel of point distance is final pixel value；After getting Zhang Quanjing image depth image corresponding with its, you can raw
3D panoramic pictures in a pair.
Each orientation corresponding to its visible top surface, it is by correct virtual camera positions when user's rotation head
The virtual visual point image rendered according to scene structure, therefore, there is correct parallax information, user can obtain more real
The depth of field is experienced.By adjusting parallax range B, in that context it may be convenient to the depth of field sense of the 3D panoramic pictures of generation is adjusted, to adapt to difference
The interpupillary distance of crowd, obtain optimal viewing experience.There is no vertical parallax, user's ophthalmic uncomfortable will not be caused, given birth to by the program
Into 360 degree of 3D distant view photographs there is no vertical parallax.
Finally it should be noted that：The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the invention,
Although the present invention is described in detail with reference to the foregoing embodiments, for those skilled in the art, it still may be used
To be modified to the technical scheme described in foregoing embodiments, or equivalent substitution is carried out to which part technical characteristic,
Within the spirit and principles of the invention, any modification, equivalent substitution and improvements made etc., it should be included in the present invention's
Within protection domain.
Claims (2)
- A kind of 1. method from dense point cloud generation 3D panoramic pictures, it is characterised in that comprise the following steps：S1, the three-dimensional structure of scene obtain：It includes active mode acquisition and passive mode obtains, and active mode is obtained using sharp Optical radar, structure light depth camera or ToF depth cameras are scanned to scene, obtain the depth image corresponding to it；Passively Mode obtains to be shot by video camera array, and the depth of each pixel value in scene is directly calculated using multi-view geometry principle, Obtain the depth image corresponding to it；Depth image is the dense three-dimensional structure for representing scene；S2, generate 3D rendering：With reference to panoramic picture and three-dimensional scene structure, 360 degree of panoramic pictures and corresponding 360 are utilized Spend depth image, by each orientation dispose a virtual video camera, render the virtual video camera should be seen that it is correct Picture, 360 degree 3D renderings of the generation with correct parallax information；S3, generate 3D panoramic pictures：When having the pixel projection in multiple original panoramic pictures in S2 into virtual view During same pixel, show the movement due to virtual video camera, object nearby has sheltered from background, now take apart from origin away from It is final pixel value from smaller pixel；After getting Zhang Quanjing image depth image corresponding with its, you can generation one To 3D panoramic pictures.
- A kind of 2. method from dense point cloud generation 3D panoramic pictures according to claim 1, it is characterised in that：The S2 Defined in original panoramic video camera be world coordinate system origin, panoramic pictures a width of w is a height ofThen its sphere represented half Footpath isFor the pixel p (u, v) in panoramic picture, by the value in query depth image, obtain deep corresponding to it Angle value is t；Under isometric projection projection model, three dimensions point coordinate under world coordinate system is corresponding to pixel p (u, v)Wherein For ball Two incidence angles under coordinate system, in units of radian；If the parallax range between right and left eyes virtual video camera is B, corresponding to the camera orientation θ represented by the pixel, this space-time Between coordinates of the point X under virtual left video camera beCoordinate under virtual right cameraFor the spatial point X under left video cameraL=(xL,yL,zL), it is apart from the distance of origin Pixel coordinate under its corresponding left eye panoramic picture isWhereinθL=atan2 (xL,zL) be spherical coordinate system under two incidence angles, in units of radian；For the spatial point X under right video cameraR=(xR,yR,zR), it is apart from the distance of origin Pixel coordinate under its corresponding left eye panoramic picture isWhereinθR= atan2(xR,zR) be spherical coordinate system under two incidence angles, in units of radian.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710716319.8A CN107545537A (en) | 2017-08-21 | 2017-08-21 | A kind of method from dense point cloud generation 3D panoramic pictures |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710716319.8A CN107545537A (en) | 2017-08-21 | 2017-08-21 | A kind of method from dense point cloud generation 3D panoramic pictures |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107545537A true CN107545537A (en) | 2018-01-05 |
Family
ID=60958425
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710716319.8A Pending CN107545537A (en) | 2017-08-21 | 2017-08-21 | A kind of method from dense point cloud generation 3D panoramic pictures |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107545537A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108629829A (en) * | 2018-03-23 | 2018-10-09 | 中德（珠海）人工智能研究院有限公司 | The three-dimensional modeling method and system that one bulb curtain camera is combined with depth camera |
| WO2020199153A1 (en) * | 2019-04-03 | 2020-10-08 | 南京泊路吉科技有限公司 | Orthophoto map generation method based on panoramic map |
| CN112116530A (en) * | 2019-06-19 | 2020-12-22 | 杭州海康威视数字技术股份有限公司 | Fisheye image distortion correction method and device and virtual display system |
| CN114513646A (en) * | 2022-01-04 | 2022-05-17 | 聚好看科技股份有限公司 | Method and device for generating panoramic video in three-dimensional virtual scene |
| CN116704129A (en) * | 2023-06-14 | 2023-09-05 | 维坤智能科技（上海）有限公司 | Panoramic view-based three-dimensional image generation method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103971378A (en) * | 2014-05-29 | 2014-08-06 | 福州大学 | Three-dimensional reconstruction method of panoramic image in mixed vision system |
| CN104519340A (en) * | 2014-12-30 | 2015-04-15 | 余俊池 | Panoramic video stitching method based on multi-depth image transformation matrix |
| CN105225230A (en) * | 2015-09-11 | 2016-01-06 | 浙江宇视科技有限公司 | A kind of method and device identifying foreground target object |
| CN105635551A (en) * | 2014-10-29 | 2016-06-01 | 浙江大华技术股份有限公司 | Method of dome camera for generating panoramic image, and dome camera |
| CN106657809A (en) * | 2016-12-13 | 2017-05-10 | 深圳先进技术研究院 | Panoramic 3D video stitching system and method |
-
2017
- 2017-08-21 CN CN201710716319.8A patent/CN107545537A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103971378A (en) * | 2014-05-29 | 2014-08-06 | 福州大学 | Three-dimensional reconstruction method of panoramic image in mixed vision system |
| CN105635551A (en) * | 2014-10-29 | 2016-06-01 | 浙江大华技术股份有限公司 | Method of dome camera for generating panoramic image, and dome camera |
| CN104519340A (en) * | 2014-12-30 | 2015-04-15 | 余俊池 | Panoramic video stitching method based on multi-depth image transformation matrix |
| CN105225230A (en) * | 2015-09-11 | 2016-01-06 | 浙江宇视科技有限公司 | A kind of method and device identifying foreground target object |
| CN106657809A (en) * | 2016-12-13 | 2017-05-10 | 深圳先进技术研究院 | Panoramic 3D video stitching system and method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108629829A (en) * | 2018-03-23 | 2018-10-09 | 中德（珠海）人工智能研究院有限公司 | The three-dimensional modeling method and system that one bulb curtain camera is combined with depth camera |
| CN108629829B (en) * | 2018-03-23 | 2019-02-15 | 中德（珠海）人工智能研究院有限公司 | Three-dimensional modeling method and system of the one bulb curtain camera in conjunction with depth camera |
| WO2020199153A1 (en) * | 2019-04-03 | 2020-10-08 | 南京泊路吉科技有限公司 | Orthophoto map generation method based on panoramic map |
| CN112116530A (en) * | 2019-06-19 | 2020-12-22 | 杭州海康威视数字技术股份有限公司 | Fisheye image distortion correction method and device and virtual display system |
| CN112116530B (en) * | 2019-06-19 | 2023-08-18 | 杭州海康威视数字技术股份有限公司 | Fisheye image distortion correction method, device and virtual display system |
| CN114513646A (en) * | 2022-01-04 | 2022-05-17 | 聚好看科技股份有限公司 | Method and device for generating panoramic video in three-dimensional virtual scene |
| CN114513646B (en) * | 2022-01-04 | 2023-09-15 | 聚好看科技股份有限公司 | Method and device for generating panoramic video in three-dimensional virtual scene |
| CN116704129A (en) * | 2023-06-14 | 2023-09-05 | 维坤智能科技（上海）有限公司 | Panoramic view-based three-dimensional image generation method, device, equipment and storage medium |
| CN116704129B (en) * | 2023-06-14 | 2024-01-30 | 维坤智能科技（上海）有限公司 | Panoramic view-based three-dimensional image generation method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106131530B (en) | A kind of bore hole 3D virtual reality display system and its methods of exhibiting | |
| US9407904B2 (en) | Method for creating 3D virtual reality from 2D images | |
| EP3130143B1 (en) | Stereo viewing | |
| CN107545537A (en) | A kind of method from dense point cloud generation 3D panoramic pictures | |
| US20090066786A1 (en) | Depth Illusion Digital Imaging | |
| US20110216160A1 (en) | System and method for creating pseudo holographic displays on viewer position aware devices | |
| CN101729920B (en) | Method for displaying stereoscopic video with free visual angles | |
| JP2006042298A (en) | Stereoscopic vision image creation method and apparatus | |
| CN103034330B (en) | A kind of eye interaction method for video conference and system | |
| CN205901977U (en) | Interactive display system of bore hole 3D augmented reality | |
| CN211128024U (en) | 3D display device | |
| CN104599317A (en) | Mobile terminal and method for achieving 3D (three-dimensional) scanning modeling function | |
| CN103281507B (en) | Video-phone system based on true Three-dimensional Display and method | |
| CN106507096B (en) | A kind of tracing type ground light field 3D display method and system with super large visual angle | |
| WO2021110038A1 (en) | 3d display apparatus and 3d image display method | |
| CN102929091A (en) | Method for manufacturing digital spherical curtain three-dimensional film | |
| WO2017144783A1 (en) | A multi-camera device and a calibration method | |
| WO2018000892A1 (en) | Imaging method, apparatus and system for panoramic stereo image | |
| US10110876B1 (en) | System and method for displaying images in 3-D stereo | |
| JP2009500878A (en) | Depth illusion digital imaging | |
| CN102402115A (en) | Method for obtaining 3D (third-dimensional) images of games and device | |
| CN110913199A (en) | VR image transmission method | |
| US20220351653A1 (en) | System and method for augmenting lightfield images | |
| WO2023005606A1 (en) | Display device and display method therefor | |
| KR20170059879A (en) | three-dimensional image photographing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180105 |