CN109147027B - Monocular image three-dimensional rebuilding method, system and device based on reference planes - Google Patents
Monocular image three-dimensional rebuilding method, system and device based on reference planes Download PDFInfo
- Publication number
- CN109147027B CN109147027B CN201811009447.XA CN201811009447A CN109147027B CN 109147027 B CN109147027 B CN 109147027B CN 201811009447 A CN201811009447 A CN 201811009447A CN 109147027 B CN109147027 B CN 109147027B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- reference planes
- image
- dimensional image
- reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The present invention relates to fields of measurement more particularly to a kind of monocular image three-dimensional rebuilding method, system and devices based on reference planes.Monocular image three-dimensional rebuilding method is the following steps are included: shooting obtains a two dimensional image；Divide the two dimensional image, obtains multiple reference planes in corresponding three-dimensional space；According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the three-dimensional reconstruction of all objects is completed.Monocular image three-dimensional rebuilding method, system and device provided by the invention based on reference planes, using monocular to the three-dimensional reconstruction ability of plane, reference planes are created first, it is then based on reference planes and each object point in image is mapped to each simulation plane, the depth of each object point is obtained, completes the three-dimensional reconstruction of whole image.Its structure is simple, realizes and is easy, there is good scene ease for use.
Description
Technical field
The present invention relates to fields of measurement more particularly to a kind of monocular image three-dimensional rebuilding method based on reference planes, it is
System and device.
Background technique
3-dimensional reconstruction is applied to every field.Monocular vision structure is simple, using convenience, not by known standard
Under the premise of object, three-dimensional reconstruction can only be carried out to the object in the single plane specified in image.Binocular stereo vision simulates people
Eye function completes three-dimensional reconstruction by parallax, compared with monocular vision, can carry out three-dimensional reconstruction to all objects in image,
But binocular stereo vision has, and structure is complicated, accurate calibration process is difficult, the big disadvantage of corresponding point matching error, in body surface
In the sparse scene of characteristic point, it is difficult to obtain accurate shape completion three-dimensional reconstruction.Structure light video camera head needs to cooperate laser
The projector, diffractive-optical element and the first-class core devices of infrared photography, the infrared speckle of diffusion is captured by infrared camera,
The depth information of each point is calculated, but diffractive-optical element higher cost, non-general user can bear.It is high-cost three-dimensional sharp
Photoscanner and binocular solid camera have higher requirement, it is therefore desirable to which a better alternative solution is asked to solve this
Topic.
Summary of the invention
The technical problems to be solved by the present invention are: providing a kind of monocular image three-dimensional reconstruction side based on reference planes
Method, system and device.
In order to solve the above-mentioned technical problem, the first technical solution that the present invention uses are as follows:
A kind of monocular image three-dimensional rebuilding method based on reference planes, comprising the following steps:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property
The three-dimensional reconstruction of body.
The second technical solution that the present invention uses are as follows:
A kind of monocular image three-dimensional reconstruction system based on reference planes, including one or more processors and memory,
The memory is stored with program, which performs the steps of when being executed by processor
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property
The three-dimensional reconstruction of body.
The third technical solution that the present invention uses are as follows:
A kind of monocular image three-dimensional reconstruction apparatus based on reference planes, including camera interconnected and three-dimensional reconstruction
Unit, the camera are configured as shooting and obtain a two dimensional image, and the content of the two dimensional image is to need three-dimensional reconstruction
Scene；
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple references in corresponding three-dimensional space
Plane；According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；It extracts in the two dimensional image and owns
Subpoint of the object in its corresponding reference planes, completes the three-dimensional reconstruction of all objects.
The beneficial effects of the present invention are: the monocular image three-dimensional rebuilding method provided by the invention based on reference planes,
System and device are then based on reference planes figure using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes
Each object point as in is mapped to each simulation plane, to obtain the depth of each object point, completes the three-dimensional of whole image
It rebuilds.Its structure is simple, realizes and is easy, there is good scene ease for use.
Detailed description of the invention
Fig. 1 is the step flow chart of the monocular image three-dimensional rebuilding method of the invention based on reference planes；
Fig. 2 is the structural schematic diagram of the monocular image three-dimensional reconstruction system of the invention based on reference planes；
Fig. 3 is that the subpoint planar reconstruction of the monocular image three-dimensional reconstruction system of the invention based on reference planes is illustrated
Figure；
Label declaration:
1, processor；2, memory.
Specific embodiment
To explain the technical content, the achieved purpose and the effect of the present invention in detail, below in conjunction with embodiment and cooperate attached
Figure is explained.
Please refer to Fig. 1, a kind of monocular image three-dimensional rebuilding method based on reference planes, comprising the following steps:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property
The three-dimensional reconstruction of body.
As can be seen from the above description, the beneficial effects of the present invention are: the monocular figure provided by the invention based on reference planes
As three-dimensional rebuilding method is then based on reference planes using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes
Each object point in image is mapped to each simulation plane, to obtain the depth of each object point, completes whole image
Three-dimensional reconstruction.
Further, step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image bat is carried out to scene areas by camera
It takes the photograph, obtains a two dimensional image；
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space；
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes
It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image；
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained
The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector
In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image
The three-dimensional reconstruction of object.
Further, step S2 further include:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, the two dimensional image
In be not belonging to the pixel in reference planes region labeled as non-reference plane type.
Referring to Fig.2, the present invention also provides a kind of monocular image three-dimensional reconstruction system based on reference planes, including one
Or multiple processors 1 and memory 2, the memory 2 are stored with program, and following step is realized when which is executed by processor 1
It is rapid:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property
The three-dimensional reconstruction of body.
As can be seen from the above description, the beneficial effects of the present invention are: the monocular figure provided by the invention based on reference planes
As three-dimensional reconstruction system is then based on reference planes using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes
Each object point in image is mapped to each simulation plane, to obtain the depth of each object point, completes whole image
Three-dimensional reconstruction.
Further, further realized when which is executed by the processor the following steps are included:
Step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image bat is carried out to scene areas by camera
It takes the photograph, obtains a two dimensional image；
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space；
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes
It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image；
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained
The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector
In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image
The three-dimensional reconstruction of object.
Further, further realized when which is executed by the processor the following steps are included:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, the two dimensional image
In be not belonging to the pixel in reference planes region labeled as non-reference plane type.
The present invention also provides a kind of monocular image three-dimensional reconstruction apparatus based on reference planes, including interconnected take the photograph
As head and three-dimensional reconstruction unit, the camera is configured as shooting and obtains a two dimensional image, and the content of the two dimensional image is
Need the scene of three-dimensional reconstruction；
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple references in corresponding three-dimensional space
Plane；According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；It extracts in the two dimensional image and owns
Subpoint of the object in its corresponding reference planes, completes the three-dimensional reconstruction of all objects.
As can be seen from the above description, the beneficial effects of the present invention are: the monocular figure provided by the invention based on reference planes
As three-dimensional reconstruction apparatus is then based on reference planes using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes
Each object point in image is mapped to each simulation plane, to obtain the depth of each object point, completes whole image
Three-dimensional reconstruction.Its structure is simple, realizes and is easy, there is good scene ease for use.
Further, after the camera is specifically configured to rotating camera to the scene areas for needing three-dimensional reconstruction,
Image taking is carried out to scene areas by camera, obtains a two dimensional image；
The three-dimensional reconstruction unit is specifically configured to analyze two dimensional image using disaggregated model, is partitioned into correspondence
Multiple reference planes in three-dimensional space；
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes
It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image；
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained
The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector
In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image
The three-dimensional reconstruction of object.
Further, the three-dimensional reconstruction unit is specifically configured to that reference planes region will be belonged in the two dimensional image
Pixel be labeled as reference planes type, the pixel that reference planes region is not belonging in the two dimensional image is flat labeled as non-reference
Noodles type.
Please refer to Fig. 1, the embodiment of the present invention one are as follows:
Monocular image three-dimensional rebuilding method provided by the invention based on reference planes, comprising steps of
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property
The three-dimensional reconstruction of body.
It should be noted that 3-dimensional reconstruction of the present invention, refers to and acquires object unified three-dimensional by image
Coordinate under reference frame, camera are a kind of mappings between D world space and two dimensional image, and mapping model can
It indicates are as follows:
The mapping model indicates the homogeneous coordinates (X of certain point on three-dimensional reference framew, Yw, Zw, 1) with the point be mapped to two
Relationship between the homogeneous coordinates (u, v, 1) fastened of dimension image coordinate, can be by camera inner parameter K and camera outside
Parameter (rotation R and translation t) obtains.Wherein, camera inner parameterIt is camera shooting
The intrinsic matrix of head, (u0, v0) it is projected position of the camera optical centre on CCD imaging plane, f is the focal length of camera,
dxAnd dyIt is the physical size of each pixel of CCD in the horizontal direction and the vertical direction respectively.
Above-mentioned step S1 specifically:
In the present embodiment, rotating camera to the scene areas for needing three-dimensional reconstruction, camera carries out figure to the region
As crawl, two dimensional image coordinate system is established；The two dimensional image coordinate system be using the upper left corner of two dimensional image as origin, to the right for
U axis, the coordinate system created by v axis downwards；And the rotation angle value of camera optical axis, the optical axis rotation are obtained by holder
Angle value includes the vertical rotational angle α of holdercWith horizontally rotate angle betac；
Above-mentioned step S2 specifically:
In the present embodiment, a large amount of similar application scenarios pictures are acquired in advance, and super picture is carried out to image using SLIC algorithm
Element processing, obtains the distribution situation of color of image and texture；Super-pixel with the same characteristics is grouped, identical characteristic is
Refer to the area pixel in image with same class geometric meaning, for example for building site scene, generally divides the image into two class geometry
Type, i.e. reference planes (construction surface) and the non-reference plane (object stretched out from reference planes, such as reinforcing bar, scaffold, concrete column
Deng)；After carrying out super-pixel grouping to the scene picture collection of acquisition, it is labeled that (reference planes or non-reference are put down to grouping
Face), then pass through the geometric classfication model of deep learning creation scene；
After S1 step grabs image, disaggregated model analyzes image, is partitioned into reference planes geometric areas,
Belong to the pixel in reference planes region in image labeled as reference planes type, other residual pixels are labeled as non-reference plane class
Type；
Above-mentioned step S3 specifically:
In the present embodiment, for convenience of describing, being in initial position null direction with holder, (horizontal angle and vertical angle are all 0
Degree) when optical axis be ZcAxis establishes camera coordinate system XcYcZc；On the reference plane, it using optical axis as origin, is sat with camera
Mark system XcYcZcChange in coordinate axis direction be reference direction, establish three-dimensional reference frame XwYwZw, wherein YwPerpendicular to reference planes；
Control holder respectively navigates to camera optical axis any three location points of reference planes, reference planes location point
Positioning picture center n × n-pixel region can be compared and reference planes type that step S2 is obtained by image matching algorithm
Set of pixels is determined, and then according to cloud platform rotation angle value and the distance of laser ranging, obtains three location points of reference planes
In coordinate system XcYcZcUnder coordinate value；
In the present embodiment, pass through first location point P of holder position laser beam to reference planes1, by point P1To laser
The distance of measuring applianceAnd the vertical rotational angle α of holder1, horizontally rotate angle beta1, point P is calculated1In coordinate system
XcYcZcUnder coordinate value
Similarly, laser beam can be obtained to second point P of reference planes2Coordinate valueAnd third point P3's
Coordinate valueDetails are not described herein again；
By the normal vector of reference planesVector can be obtainedIt is referring to
Projection vector on plane normal
Can further obtain, optical axis and reference planes apart from it is nearest when, vertical drift angle of the optical axis with respect to null directionLevel angle of the optical axis with respect to null direction
By grabbing the vertical rotational angle α of holder when image in step S1cWith horizontally rotate angle betac, optical axis can be obtained at that time
Unit vector
It can further obtain, projection vectorAnd unit vectorBetween angle
It can further obtain, in the image of step S1 crawl, the translation vector of optical axis to reference planesAnd rotating vector
(rotation RcWith translation tc) substitute into reference planes three-dimensional reference frame and two dimensional image coordinate system between mapping modelIn, the Three-dimensional Gravity to reference planes can be realized
It builds, obtains the coordinate in image under three-dimensional reference frame corresponding to each pixel of reference planes；
Above-mentioned step S4 specifically:
In the present embodiment, the extracting method of project objects point is as follows:
To image application SharpMask image segmentation algorithm, the edge segmentation texture of objects in images is obtained, to belonging to S2
The texture pixel point of obtained non-reference plane type set of pixels range, calculates it in the subpoint of reference planes；
The subpoint refers to that the every bit on object projects to the point in reference planes, the line of object point and subpoint
It is perpendicular to reference planes, i.e., parallel with the normal vector of reference planes；
The junction of object and reference planes, reflection are exactly the edge segmentation texture pixel point and ginseng of object on the image
The adjacent area for examining image plane vegetarian refreshments, object edge segmentation texture pixel point eight connectivity neighborhood in, search whether include
The reference planes classes of pixels collection that step S2 is obtained, comprising reference planes pixel be included in the object edge and divide texture picture
The projection point set of vegetarian refreshments；Each reference planes pixel that texture pixel point is concentrated with subpoint respectively is combined, is obtained
Plane and straight line collection of the texture pixel point in two dimensional image coordinate system；
In three-dimensional reference frame XwYwZwUnder, take the outer point n of a point m (0,0,0) and reference planes in reference planes (0,
0,1), it is known that vectorThat is the normal vector of reference planes substituted into point m, n and the three-dimensional ginseng of the plane vertical with reference planes
Examine the mapping model between coordinate system and two dimensional image coordinate system In, the three-dimensional reconstruction to the vertical plane can be realized, obtain normal vectorStraight line under two dimensional image coordinate system；
According to the object point constraint condition principle parallel with the normal vector of reference planes with the line of subpoint, described flat
Face straight line is focused to find out and normal vectorThe highest straight line of the straight line parallel degree of correlation, the parallel degree of correlation can two straight lines angle
Make criterion, the subpoint reference planes pixel corresponding to the highest straight line of the parallel degree of correlation, as object point；And
By the three-dimensional reconstruction result of step S2 reference planes, the coordinate P under three-dimensional reference frame corresponding to subpoint can be obtainedp=(Xp,
Yp, Zp)；
By unit vector in step S3With translation vector tc, optical axis vector when step S1 capturing pictures can be obtained
By the projection vector on reference planes normalIt can obtain, vectorProjection vector on the reference plane
With projection vectorAs normal vector, in conjunction with subpoint Pp, subpoint and vertical with reference planes flat can be obtained
Face, this vertical plane are the plane where object point；
It can further obtain, vectorWith the intersection point (X of vertical planei, Yi, Zi), wherein
It can further obtain, the shift value of optical axis to vertical plane
Substitute into the mapping model between the three-dimensional reference frame and two dimensional image coordinate system of vertical planeIn, the reference of three-dimensional corresponding to object point can be obtained
Coordinate under coordinate system；
More than repeating, coordinate of all objects point under three-dimensional reference frame in image can be obtained, complete in image
The three-dimensional reconstruction of all objects；
As shown in figure 3, on reference planes ρ (building site flooring), having an oblique cutting in reference planes in two dimensional image
On reinforcing bar, subpoint of pixel A, the B of reinforcing bar on reference planes ρ be a, b, subpoint a, b and reference planes ρ is obtained
Vertical simulation plane π1、π2, can further obtain plane π1、π2Mapping model between two dimensional image, to obtain image slices
The three-dimensional coordinate of vegetarian refreshments A, B；The rest may be inferred, can obtain the three-dimensional coordinate of reinforcing bar all pixels point, complete the Three-dimensional Gravity of entire reinforcing bar
It builds.
Referring to Fig. 2, the embodiment of the present invention two are as follows:
A kind of monocular image three-dimensional reconstruction system based on reference planes provided by the invention, including one or more processing
Device and memory, the memory are stored with program, which performs the steps of when being executed by processor
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property
The three-dimensional reconstruction of body.
It should be noted that 3-dimensional reconstruction of the present invention, refers to and acquires object unified three-dimensional by image
Coordinate under reference frame, camera are a kind of mappings between D world space and two dimensional image, and mapping model can
It indicates are as follows:
The mapping model indicates the homogeneous coordinates (X of certain point on three-dimensional reference framew, Yw, Zw, 1) with the point be mapped to two
Relationship between the homogeneous coordinates (u, v, 1) fastened of dimension image coordinate, can be by camera inner parameter K and camera outside
Parameter (rotation R and translation t) obtains.Wherein, camera inner parameterIt is camera shooting
The intrinsic matrix of head, (u0, v0) it is projected position of the camera optical centre on CCD imaging plane, f is the focal length of camera,
dxAnd dyIt is the physical size of each pixel of CCD in the horizontal direction and the vertical direction respectively.
Above-mentioned step S1 specifically:
In the present embodiment, rotating camera to the scene areas for needing three-dimensional reconstruction, camera carries out figure to the region
As crawl, two dimensional image coordinate system is established；The two dimensional image coordinate system be using the upper left corner of two dimensional image as origin, to the right for
U axis, the coordinate system created by v axis downwards；And the rotation angle value of camera optical axis, the optical axis rotation are obtained by holder
Angle value includes the vertical rotational angle α of holdercWith horizontally rotate angle betac；
Above-mentioned step S2 specifically:
In the present embodiment, a large amount of similar application scenarios pictures are acquired in advance, and super picture is carried out to image using SLIC algorithm
Element processing, obtains the distribution situation of color of image and texture；Super-pixel with the same characteristics is grouped, identical characteristic is
Refer to the area pixel in image with same class geometric meaning, for example for building site scene, generally divides the image into two class geometry
Type, i.e. reference planes (construction surface) and the non-reference plane (object stretched out from reference planes, such as reinforcing bar, scaffold, concrete column
Deng)；After carrying out super-pixel grouping to the scene picture collection of acquisition, it is labeled that (reference planes or non-reference are put down to grouping
Face), then pass through the geometric classfication model of deep learning creation scene；
After S1 step grabs image, disaggregated model analyzes image, is partitioned into reference planes geometric areas,
Belong to the pixel in reference planes region in image labeled as reference planes type, other residual pixels are labeled as non-reference plane class
Type；
Above-mentioned step S3 specifically:
In the present embodiment, for convenience of describing, being in initial position null direction with holder, (horizontal angle and vertical angle are all 0
Degree) when optical axis be ZcAxis establishes camera coordinate system XcYcZc；On the reference plane, it using optical axis as origin, is sat with camera
Mark system XcYcZcChange in coordinate axis direction be reference direction, establish three-dimensional reference frame XwYwZw, wherein YwPerpendicular to reference planes；
Control holder respectively navigates to camera optical axis any three location points of reference planes, reference planes location point
Positioning picture center n × n-pixel region can be compared and reference planes type that step S2 is obtained by image matching algorithm
Set of pixels is determined, and then according to cloud platform rotation angle value and the distance of laser ranging, obtains three location points of reference planes
In coordinate system XcYcZcUnder coordinate value；
In the present embodiment, pass through first location point P of holder position laser beam to reference planes1, by point P1To laser
The distance of measuring applianceAnd the vertical rotational angle α of holder1, horizontally rotate angle beta1, point P is calculated1In coordinate system
XcYcZcUnder coordinate value
Similarly, laser beam can be obtained to second point P of reference planes2Coordinate valueAnd third point P3's
Coordinate valueDetails are not described herein again；
By the normal vector of reference planesVector can be obtainedIt is referring to
Projection vector on plane normal
Can further obtain, optical axis and reference planes apart from it is nearest when, vertical drift angle of the optical axis with respect to null directionLevel angle of the optical axis with respect to null direction
By grabbing the vertical rotational angle α of holder when image in step S1cWith horizontally rotate angle betac, optical axis can be obtained at that time
Unit vector
It can further obtain, projection vectorAnd unit vectorBetween angle
It can further obtain, in the image of step S1 crawl, the translation vector of optical axis to reference planesAnd rotating vector
(rotation RcWith translation tc) substitute into reference planes three-dimensional reference frame and two dimensional image coordinate system between mapping modelIn, the Three-dimensional Gravity to reference planes can be realized
It builds, obtains the coordinate in image under three-dimensional reference frame corresponding to each pixel of reference planes；
Above-mentioned step S4 specifically:
In the present embodiment, the extracting method of project objects point is as follows:
To image application SharpMask image segmentation algorithm, the edge segmentation texture of objects in images is obtained, to belonging to S2
The texture pixel point of obtained non-reference plane type set of pixels range, calculates it in the subpoint of reference planes；
The subpoint refers to that the every bit on object projects to the point in reference planes, the line of object point and subpoint
It is perpendicular to reference planes, i.e., parallel with the normal vector of reference planes；
The junction of object and reference planes, reflection are exactly the edge segmentation texture pixel point and ginseng of object on the image
The adjacent area for examining image plane vegetarian refreshments, object edge segmentation texture pixel point eight connectivity neighborhood in, search whether include
The reference planes classes of pixels collection that step S2 is obtained, comprising reference planes pixel be included in the object edge and divide texture picture
The projection point set of vegetarian refreshments；Each reference planes pixel that texture pixel point is concentrated with subpoint respectively is combined, is obtained
Plane and straight line collection of the texture pixel point in two dimensional image coordinate system；
In three-dimensional reference frame XwYwZwUnder, take the outer point n of a point m (0,0,0) and reference planes in reference planes (0,
0,1), it is known that vectorThat is the normal vector of reference planes substituted into point m, n and the three-dimensional ginseng of the plane vertical with reference planes
Examine the mapping model between coordinate system and two dimensional image coordinate system In, the three-dimensional reconstruction to the vertical plane can be realized, obtain normal vectorStraight line under two dimensional image coordinate system；
According to the object point constraint condition principle parallel with the normal vector of reference planes with the line of subpoint, described flat
Face straight line is focused to find out and normal vectorThe highest straight line of the straight line parallel degree of correlation, the parallel degree of correlation can two straight lines angle
Make criterion, the subpoint reference planes pixel corresponding to the highest straight line of the parallel degree of correlation, as object point；And
By the three-dimensional reconstruction result of step S2 reference planes, the coordinate P under three-dimensional reference frame corresponding to subpoint can be obtainedp=(Xp,
Yp, Zp)；
By unit vector in step S3With translation vector tc, optical axis vector when step S1 capturing pictures can be obtained
By the projection vector on reference planes normalIt can obtain, vectorProjection vector on the reference plane
With projection vectorAs normal vector, in conjunction with subpoint Pp, subpoint and vertical with reference planes flat can be obtained
Face, this vertical plane are the plane where object point；
It can further obtain, vectorWith the intersection point (X of vertical planei, Yi, Zi), wherein
It can further obtain, the shift value of optical axis to vertical plane
Substitute into the mapping model between the three-dimensional reference frame and two dimensional image coordinate system of vertical planeIn, the reference of three-dimensional corresponding to object point can be obtained
Coordinate under coordinate system；
More than repeating, coordinate of all objects point under three-dimensional reference frame in image can be obtained, complete in image
The three-dimensional reconstruction of all objects.
The embodiment of the present invention three are as follows:
The present invention also provides a kind of monocular image three-dimensional reconstruction apparatus based on reference planes, including interconnected take the photograph
As head and three-dimensional reconstruction unit, the camera is configured as shooting and obtains a two dimensional image, and the content of the two dimensional image is
Need the scene of three-dimensional reconstruction；
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple references in corresponding three-dimensional space
Plane；According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；It extracts in the two dimensional image and owns
Subpoint of the object in its corresponding reference planes, completes the three-dimensional reconstruction of all objects.
After the camera is specifically configured to rotating camera to the scene areas for needing three-dimensional reconstruction, pass through camera
Image taking is carried out to scene areas, obtains a two dimensional image；
The three-dimensional reconstruction unit is specifically configured to analyze two dimensional image using disaggregated model, is partitioned into correspondence
Multiple reference planes in three-dimensional space；
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes
It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image；
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained
The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector
In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image
The three-dimensional reconstruction of object.
The three-dimensional reconstruction unit is specifically configured to that the pixel mark in reference planes region will be belonged in the two dimensional image
It is denoted as reference planes type, the pixel in reference planes region is not belonging in the two dimensional image labeled as non-reference plane type.
In a particular embodiment, the above-mentioned monocular image three-dimensional reconstruction apparatus based on reference planes, including measurement end；Institute
Stating measurement end includes laser, camera, angle adjustor and processor；The laser is arranged on camera, the laser, camera
It is connect respectively with processor with angle adjustor, the laser and camera are connect with angle adjustor respectively, further include server-side and at least
More than one terminal；The measurement end is connect with terminal respectively by server-side.Server-side is logical with measurement end and terminal respectively
Cross network connection.Server-side provides the communication interface between measurement end and terminal, and server-side, which receives to come from/send, goes to measurement end
Or the electric signal of terminal.Terminal shows visual output to user, including two dimensional image, the text information of three-dimensional reconstruction result, figure
Shape information and any combination thereof.Terminal receives the control input of user, sends control signal to server-side, executes two dimensional image and grab
It takes, obtains object dimensional reconstructed results in image.
In conclusion a kind of monocular image three-dimensional rebuilding method, system and dress based on reference planes provided by the invention
It sets, using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes, is then based on reference planes every in image
A object point is mapped to each simulation plane, to obtain the depth of each object point, completes the three-dimensional reconstruction of whole image.It is tied
Structure is simple, realizes and is easy, there is good scene ease for use.
The above description is only an embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalents made by bright specification and accompanying drawing content are applied directly or indirectly in relevant technical field, similarly include
In scope of patent protection of the invention.
Claims (6)
1. a kind of monocular image three-dimensional rebuilding method based on reference planes, which comprises the following steps:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes all objects
Three-dimensional reconstruction；
Step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image taking is carried out to scene areas by camera, is obtained
To a two dimensional image；
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space；
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional with reference to seat of reference planes
In mapping model between mark system and two dimensional image coordinate system, the three-dimensional reconstruction of reference planes in two dimensional image is completed；
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point place is obtained
Plane, the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector are substituted into object
In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of point plane, all objects in two dimensional image are completed
Three-dimensional reconstruction.
2. the monocular image three-dimensional rebuilding method according to claim 1 based on reference planes, which is characterized in that step S2
Further include:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, in the two dimensional image not
Belong to the pixel in reference planes region labeled as non-reference plane type.
3. a kind of monocular image three-dimensional reconstruction system based on reference planes, which is characterized in that including one or more processors
And memory, the memory are stored with program, which performs the steps of when being executed by processor
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction；
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space；
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes all objects
Three-dimensional reconstruction；
Step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image taking is carried out to scene areas by camera, is obtained
To a two dimensional image；
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space；
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional with reference to seat of reference planes
In mapping model between mark system and two dimensional image coordinate system, the three-dimensional reconstruction of reference planes in two dimensional image is completed；
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point place is obtained
Plane, the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector are substituted into object
In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of point plane, all objects in two dimensional image are completed
Three-dimensional reconstruction.
4. the monocular image three-dimensional reconstruction system according to claim 3 based on reference planes, which is characterized in that the program
By the processor execute when further realize the following steps are included:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, in the two dimensional image not
Belong to the pixel in reference planes region labeled as non-reference plane type.
5. a kind of monocular image three-dimensional reconstruction apparatus based on reference planes, which is characterized in that including camera interconnected
And three-dimensional reconstruction unit, the camera are configured as shooting and obtain a two dimensional image, the content of the two dimensional image is to need
The scene of three-dimensional reconstruction；
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple with reference to flat in corresponding three-dimensional space
Face；According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction；Extract property in the two dimensional image
Subpoint of the body in its corresponding reference planes, completes the three-dimensional reconstruction of all objects；
After the camera is specifically configured to rotating camera to the scene areas for needing three-dimensional reconstruction, by camera to field
Scene area carries out image taking, obtains a two dimensional image；
The three-dimensional reconstruction unit is specifically configured to analyze two dimensional image using disaggregated model, is partitioned into corresponding three-dimensional
Multiple reference planes in space；
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional with reference to seat of reference planes
In mapping model between mark system and two dimensional image coordinate system, the three-dimensional reconstruction of reference planes in two dimensional image is completed；
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point place is obtained
Plane, the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector are substituted into object
In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of point plane, all objects in two dimensional image are completed
Three-dimensional reconstruction.
6. the monocular image three-dimensional reconstruction apparatus according to claim 5 based on reference planes, which is characterized in that described three
Dimension reconstruction unit is specifically configured to the pixel by reference planes region is belonged in the two dimensional image labeled as reference planes class
Type is not belonging to the pixel in reference planes region in the two dimensional image labeled as non-reference plane type.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811009447.XA CN109147027B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional rebuilding method, system and device based on reference planes |
| CN201910964298.0A CN110827392B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device |
| CN201910963573.7A CN110838164B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device based on object point depth |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811009447.XA CN109147027B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional rebuilding method, system and device based on reference planes |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910963573.7A Division CN110838164B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device based on object point depth |
| CN201910964298.0A Division CN110827392B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109147027A CN109147027A (en) | 2019-01-04 |
| CN109147027B true CN109147027B (en) | 2019-11-08 |
Family
ID=64825870
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910964298.0A Active CN110827392B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device |
| CN201910963573.7A Active CN110838164B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device based on object point depth |
| CN201811009447.XA Active CN109147027B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional rebuilding method, system and device based on reference planes |
Family Applications Before (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910964298.0A Active CN110827392B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device |
| CN201910963573.7A Active CN110838164B (en) | 2018-08-31 | 2018-08-31 | Monocular image three-dimensional reconstruction method, system and device based on object point depth |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN110827392B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741404B (en) * | 2019-01-10 | 2020-11-17 | 奥本未来(北京)科技有限责任公司 | Light field acquisition method based on mobile equipment |
| CN111220128B (en) * | 2019-01-31 | 2022-10-25 | 金钱猫科技股份有限公司 | Monocular focusing measuring method and terminal |
| CN112837404B (en) * | 2019-11-25 | 2024-01-19 | 北京初速度科技有限公司 | Method and device for constructing three-dimensional information of planar object |
| CN111415420B (en) * | 2020-03-25 | 2024-01-23 | 北京迈格威科技有限公司 | Spatial information determining method and device and electronic equipment |
| CN112198527B (en) * | 2020-09-30 | 2022-12-27 | 上海炬佑智能科技有限公司 | Reference plane adjustment and obstacle detection method, depth camera and navigation equipment |
| CN112198529B (en) * | 2020-09-30 | 2022-12-27 | 上海炬佑智能科技有限公司 | Reference plane adjustment and obstacle detection method, depth camera and navigation equipment |
| CN112884898B (en) * | 2021-03-17 | 2022-06-07 | 杭州思看科技有限公司 | Reference device for measuring texture mapping precision |
| CN114596406A (en) * | 2022-01-25 | 2022-06-07 | 海拓信息技术(佛山)有限公司 | Three-dimensional construction method and device based on monocular camera |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708566A (en) * | 2012-05-08 | 2012-10-03 | 天津工业大学 | Novel single-camera and single-projection light source synchronous calibrating method |
| CN103578133A (en) * | 2012-08-03 | 2014-02-12 | 浙江大华技术股份有限公司 | Method and device for reconstructing two-dimensional image information in three-dimensional mode |
| CN104809755A (en) * | 2015-04-09 | 2015-07-29 | 福州大学 | Single-image-based cultural relic three-dimensional reconstruction method |
| CN106204717A (en) * | 2015-05-28 | 2016-12-07 | 长沙维纳斯克信息技术有限公司 | A kind of stereo-picture quick three-dimensional reconstructing method and device |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6240198B1 (en) * | 1998-04-13 | 2001-05-29 | Compaq Computer Corporation | Method for figure tracking using 2-D registration |
| CN102697508B (en) * | 2012-04-23 | 2013-10-16 | 中国人民解放军国防科学技术大学 | Method for performing gait recognition by adopting three-dimensional reconstruction of monocular vision |
| CN103077524A (en) * | 2013-01-25 | 2013-05-01 | 福州大学 | Calibrating method of hybrid vision system |
| CN104424630A (en) * | 2013-08-20 | 2015-03-18 | 华为技术有限公司 | Three-dimension reconstruction method and device, and mobile terminal |
| CN105303554B (en) * | 2015-09-16 | 2017-11-28 | 东软集团股份有限公司 | The 3D method for reconstructing and device of a kind of image characteristic point |
| CN106960442A (en) * | 2017-03-01 | 2017-07-18 | 东华大学 | Based on the infrared night robot vision wide view-field three-D construction method of monocular |
| CN107063129B (en) * | 2017-05-25 | 2019-06-07 | 西安知象光电科技有限公司 | A kind of array parallel laser projection three-dimensional scan method |
| CN107945268B (en) * | 2017-12-15 | 2019-11-29 | 深圳大学 | A kind of high-precision three-dimensional method for reconstructing and system based on binary area-structure light |
| CN108062788A (en) * | 2017-12-18 | 2018-05-22 | 北京锐安科技有限公司 | A kind of three-dimensional rebuilding method, device, equipment and medium |
-
2018
- 2018-08-31 CN CN201910964298.0A patent/CN110827392B/en active Active
- 2018-08-31 CN CN201910963573.7A patent/CN110838164B/en active Active
- 2018-08-31 CN CN201811009447.XA patent/CN109147027B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708566A (en) * | 2012-05-08 | 2012-10-03 | 天津工业大学 | Novel single-camera and single-projection light source synchronous calibrating method |
| CN103578133A (en) * | 2012-08-03 | 2014-02-12 | 浙江大华技术股份有限公司 | Method and device for reconstructing two-dimensional image information in three-dimensional mode |
| CN104809755A (en) * | 2015-04-09 | 2015-07-29 | 福州大学 | Single-image-based cultural relic three-dimensional reconstruction method |
| CN106204717A (en) * | 2015-05-28 | 2016-12-07 | 长沙维纳斯克信息技术有限公司 | A kind of stereo-picture quick three-dimensional reconstructing method and device |
Non-Patent Citations (5)
| Title |
|---|
| "Accurate multiple view 3D reconstruction using patch-based stereo for large-scale scenes";Shen S;《IEEE Transactions on Image Processing》;20131231;全文 * |
| "医学图像三维重建研究综述";郑丽萍;《第十一届中国体视学与图像分析学术会议论文集》;20061031;全文 * |
| "单目图像序列光流三维重建技术研究综述";张聪炫;《电子学报》;20161231;第44卷(第12期);全文 * |
| "基于视觉的三维重建技术综述";佟帅;《计算机应用研究》;20110731;第28卷(第7期);全文 * |
| "基于计算机视觉的三维重建技术综述";徐超;《数字技术与应用》;20170115;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109147027A (en) | 2019-01-04 |
| CN110827392A (en) | 2020-02-21 |
| CN110838164A (en) | 2020-02-25 |
| CN110827392B (en) | 2023-03-24 |
| CN110838164B (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109147027B (en) | Monocular image three-dimensional rebuilding method, system and device based on reference planes | |
| CN110782524B (en) | Indoor three-dimensional reconstruction method based on panoramic image | |
| TW201915944A (en) | Image processing method, apparatus, and storage medium | |
| CN106462943A (en) | Aligning panoramic imagery and aerial imagery | |
| WO2020042970A1 (en) | Three-dimensional modeling method and device therefor | |
| WO2012153447A1 (en) | Image processing device, image processing method, program, and integrated circuit | |
| CN103971408A (en) | Three-dimensional facial model generating system and method | |
| KR20140057610A (en) | Using motion parallax to create 3d perception from 2d images | |
| CN104599317A (en) | Mobile terminal and method for achieving 3D (three-dimensional) scanning modeling function | |
| WO2018032841A1 (en) | Method, device and system for drawing three-dimensional image | |
| CN106264537B (en) | System and method for measuring human body posture height in image | |
| CN109510975A (en) | A kind of extracting method of video image, equipment and system | |
| CN113379901A (en) | Method and system for establishing house live-action three-dimension by utilizing public self-photographing panoramic data | |
| JP4354708B2 (en) | Multi-view camera system | |
| US10354399B2 (en) | Multi-view back-projection to a light-field | |
| CN111047678B (en) | Three-dimensional face acquisition device and method | |
| CN107864372A (en) | Solid picture-taking method, apparatus and terminal | |
| CN112102504A (en) | Three-dimensional scene and two-dimensional image mixing method based on mixed reality | |
| CN116860112A (en) | Combined scene experience generation method, system and medium based on XR technology | |
| Verhoeven et al. | From pixel to mesh: accurate and straightforward 3D documentation of cultural heritage from the Cres/Lošinj archipelago | |
| CN115205491A (en) | Method and device for handheld multi-view three-dimensional reconstruction | |
| CN110148206A (en) | The fusion method in more spaces | |
| CN115588069A (en) | Scene reconstruction method and device, equipment and storage medium | |
| CN114463520A (en) | Method and device for realizing Virtual Reality (VR) roaming | |
| Shimamura et al. | Construction and presentation of a virtual environment using panoramic stereo images of a real scene and computer graphics models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |