CN109147027B - Monocular image three-dimensional rebuilding method, system and device based on reference planes - Google Patents

Monocular image three-dimensional rebuilding method, system and device based on reference planes Download PDF

Info

Publication number
CN109147027B
CN109147027B CN201811009447.XA CN201811009447A CN109147027B CN 109147027 B CN109147027 B CN 109147027B CN 201811009447 A CN201811009447 A CN 201811009447A CN 109147027 B CN109147027 B CN 109147027B
Authority
CN
China
Prior art keywords
dimensional
reference planes
image
dimensional image
reconstruction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811009447.XA
Other languages
Chinese (zh)
Other versions
CN109147027A (en
Inventor
郑敏忠
黄宗荣
江世松
林大甲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jinqianmao Technology Co Ltd
Original Assignee
Jinqianmao Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jinqianmao Technology Co Ltd filed Critical Jinqianmao Technology Co Ltd
Priority to CN201811009447.XA priority Critical patent/CN109147027B/en
Priority to CN201910964298.0A priority patent/CN110827392B/en
Priority to CN201910963573.7A priority patent/CN110838164B/en
Publication of CN109147027A publication Critical patent/CN109147027A/en
Application granted granted Critical
Publication of CN109147027B publication Critical patent/CN109147027B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/10Geometric effects

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The present invention relates to fields of measurement more particularly to a kind of monocular image three-dimensional rebuilding method, system and devices based on reference planes.Monocular image three-dimensional rebuilding method is the following steps are included: shooting obtains a two dimensional image;Divide the two dimensional image, obtains multiple reference planes in corresponding three-dimensional space;According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the three-dimensional reconstruction of all objects is completed.Monocular image three-dimensional rebuilding method, system and device provided by the invention based on reference planes, using monocular to the three-dimensional reconstruction ability of plane, reference planes are created first, it is then based on reference planes and each object point in image is mapped to each simulation plane, the depth of each object point is obtained, completes the three-dimensional reconstruction of whole image.Its structure is simple, realizes and is easy, there is good scene ease for use.

Description

Monocular image three-dimensional rebuilding method, system and device based on reference planes
Technical field
The present invention relates to fields of measurement more particularly to a kind of monocular image three-dimensional rebuilding method based on reference planes, it is System and device.
Background technique
3-dimensional reconstruction is applied to every field.Monocular vision structure is simple, using convenience, not by known standard Under the premise of object, three-dimensional reconstruction can only be carried out to the object in the single plane specified in image.Binocular stereo vision simulates people Eye function completes three-dimensional reconstruction by parallax, compared with monocular vision, can carry out three-dimensional reconstruction to all objects in image, But binocular stereo vision has, and structure is complicated, accurate calibration process is difficult, the big disadvantage of corresponding point matching error, in body surface In the sparse scene of characteristic point, it is difficult to obtain accurate shape completion three-dimensional reconstruction.Structure light video camera head needs to cooperate laser The projector, diffractive-optical element and the first-class core devices of infrared photography, the infrared speckle of diffusion is captured by infrared camera, The depth information of each point is calculated, but diffractive-optical element higher cost, non-general user can bear.It is high-cost three-dimensional sharp Photoscanner and binocular solid camera have higher requirement, it is therefore desirable to which a better alternative solution is asked to solve this Topic.
Summary of the invention
The technical problems to be solved by the present invention are: providing a kind of monocular image three-dimensional reconstruction side based on reference planes Method, system and device.
In order to solve the above-mentioned technical problem, the first technical solution that the present invention uses are as follows:
A kind of monocular image three-dimensional rebuilding method based on reference planes, comprising the following steps:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property The three-dimensional reconstruction of body.
The second technical solution that the present invention uses are as follows:
A kind of monocular image three-dimensional reconstruction system based on reference planes, including one or more processors and memory, The memory is stored with program, which performs the steps of when being executed by processor
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property The three-dimensional reconstruction of body.
The third technical solution that the present invention uses are as follows:
A kind of monocular image three-dimensional reconstruction apparatus based on reference planes, including camera interconnected and three-dimensional reconstruction Unit, the camera are configured as shooting and obtain a two dimensional image, and the content of the two dimensional image is to need three-dimensional reconstruction Scene;
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple references in corresponding three-dimensional space Plane;According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;It extracts in the two dimensional image and owns Subpoint of the object in its corresponding reference planes, completes the three-dimensional reconstruction of all objects.
The beneficial effects of the present invention are: the monocular image three-dimensional rebuilding method provided by the invention based on reference planes, System and device are then based on reference planes figure using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes Each object point as in is mapped to each simulation plane, to obtain the depth of each object point, completes the three-dimensional of whole image It rebuilds.Its structure is simple, realizes and is easy, there is good scene ease for use.
Detailed description of the invention
Fig. 1 is the step flow chart of the monocular image three-dimensional rebuilding method of the invention based on reference planes;
Fig. 2 is the structural schematic diagram of the monocular image three-dimensional reconstruction system of the invention based on reference planes;
Fig. 3 is that the subpoint planar reconstruction of the monocular image three-dimensional reconstruction system of the invention based on reference planes is illustrated Figure;
Label declaration:
1, processor;2, memory.
Specific embodiment
To explain the technical content, the achieved purpose and the effect of the present invention in detail, below in conjunction with embodiment and cooperate attached Figure is explained.
Please refer to Fig. 1, a kind of monocular image three-dimensional rebuilding method based on reference planes, comprising the following steps:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property The three-dimensional reconstruction of body.
As can be seen from the above description, the beneficial effects of the present invention are: the monocular figure provided by the invention based on reference planes As three-dimensional rebuilding method is then based on reference planes using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes Each object point in image is mapped to each simulation plane, to obtain the depth of each object point, completes whole image Three-dimensional reconstruction.
Further, step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image bat is carried out to scene areas by camera It takes the photograph, obtains a two dimensional image;
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space;
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image;
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image The three-dimensional reconstruction of object.
Further, step S2 further include:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, the two dimensional image In be not belonging to the pixel in reference planes region labeled as non-reference plane type.
Referring to Fig.2, the present invention also provides a kind of monocular image three-dimensional reconstruction system based on reference planes, including one Or multiple processors 1 and memory 2, the memory 2 are stored with program, and following step is realized when which is executed by processor 1 It is rapid:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property The three-dimensional reconstruction of body.
As can be seen from the above description, the beneficial effects of the present invention are: the monocular figure provided by the invention based on reference planes As three-dimensional reconstruction system is then based on reference planes using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes Each object point in image is mapped to each simulation plane, to obtain the depth of each object point, completes whole image Three-dimensional reconstruction.
Further, further realized when which is executed by the processor the following steps are included:
Step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image bat is carried out to scene areas by camera It takes the photograph, obtains a two dimensional image;
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space;
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image;
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image The three-dimensional reconstruction of object.
Further, further realized when which is executed by the processor the following steps are included:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, the two dimensional image In be not belonging to the pixel in reference planes region labeled as non-reference plane type.
The present invention also provides a kind of monocular image three-dimensional reconstruction apparatus based on reference planes, including interconnected take the photograph As head and three-dimensional reconstruction unit, the camera is configured as shooting and obtains a two dimensional image, and the content of the two dimensional image is Need the scene of three-dimensional reconstruction;
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple references in corresponding three-dimensional space Plane;According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;It extracts in the two dimensional image and owns Subpoint of the object in its corresponding reference planes, completes the three-dimensional reconstruction of all objects.
As can be seen from the above description, the beneficial effects of the present invention are: the monocular figure provided by the invention based on reference planes As three-dimensional reconstruction apparatus is then based on reference planes using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes Each object point in image is mapped to each simulation plane, to obtain the depth of each object point, completes whole image Three-dimensional reconstruction.Its structure is simple, realizes and is easy, there is good scene ease for use.
Further, after the camera is specifically configured to rotating camera to the scene areas for needing three-dimensional reconstruction, Image taking is carried out to scene areas by camera, obtains a two dimensional image;
The three-dimensional reconstruction unit is specifically configured to analyze two dimensional image using disaggregated model, is partitioned into correspondence Multiple reference planes in three-dimensional space;
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image;
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image The three-dimensional reconstruction of object.
Further, the three-dimensional reconstruction unit is specifically configured to that reference planes region will be belonged in the two dimensional image Pixel be labeled as reference planes type, the pixel that reference planes region is not belonging in the two dimensional image is flat labeled as non-reference Noodles type.
Please refer to Fig. 1, the embodiment of the present invention one are as follows:
Monocular image three-dimensional rebuilding method provided by the invention based on reference planes, comprising steps of
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property The three-dimensional reconstruction of body.
It should be noted that 3-dimensional reconstruction of the present invention, refers to and acquires object unified three-dimensional by image Coordinate under reference frame, camera are a kind of mappings between D world space and two dimensional image, and mapping model can It indicates are as follows:
The mapping model indicates the homogeneous coordinates (X of certain point on three-dimensional reference framew, Yw, Zw, 1) with the point be mapped to two Relationship between the homogeneous coordinates (u, v, 1) fastened of dimension image coordinate, can be by camera inner parameter K and camera outside Parameter (rotation R and translation t) obtains.Wherein, camera inner parameterIt is camera shooting The intrinsic matrix of head, (u0, v0) it is projected position of the camera optical centre on CCD imaging plane, f is the focal length of camera, dxAnd dyIt is the physical size of each pixel of CCD in the horizontal direction and the vertical direction respectively.
Above-mentioned step S1 specifically:
In the present embodiment, rotating camera to the scene areas for needing three-dimensional reconstruction, camera carries out figure to the region As crawl, two dimensional image coordinate system is established;The two dimensional image coordinate system be using the upper left corner of two dimensional image as origin, to the right for U axis, the coordinate system created by v axis downwards;And the rotation angle value of camera optical axis, the optical axis rotation are obtained by holder Angle value includes the vertical rotational angle α of holdercWith horizontally rotate angle betac
Above-mentioned step S2 specifically:
In the present embodiment, a large amount of similar application scenarios pictures are acquired in advance, and super picture is carried out to image using SLIC algorithm Element processing, obtains the distribution situation of color of image and texture;Super-pixel with the same characteristics is grouped, identical characteristic is Refer to the area pixel in image with same class geometric meaning, for example for building site scene, generally divides the image into two class geometry Type, i.e. reference planes (construction surface) and the non-reference plane (object stretched out from reference planes, such as reinforcing bar, scaffold, concrete column Deng);After carrying out super-pixel grouping to the scene picture collection of acquisition, it is labeled that (reference planes or non-reference are put down to grouping Face), then pass through the geometric classfication model of deep learning creation scene;
After S1 step grabs image, disaggregated model analyzes image, is partitioned into reference planes geometric areas, Belong to the pixel in reference planes region in image labeled as reference planes type, other residual pixels are labeled as non-reference plane class Type;
Above-mentioned step S3 specifically:
In the present embodiment, for convenience of describing, being in initial position null direction with holder, (horizontal angle and vertical angle are all 0 Degree) when optical axis be ZcAxis establishes camera coordinate system XcYcZc;On the reference plane, it using optical axis as origin, is sat with camera Mark system XcYcZcChange in coordinate axis direction be reference direction, establish three-dimensional reference frame XwYwZw, wherein YwPerpendicular to reference planes;
Control holder respectively navigates to camera optical axis any three location points of reference planes, reference planes location point Positioning picture center n × n-pixel region can be compared and reference planes type that step S2 is obtained by image matching algorithm Set of pixels is determined, and then according to cloud platform rotation angle value and the distance of laser ranging, obtains three location points of reference planes In coordinate system XcYcZcUnder coordinate value;
In the present embodiment, pass through first location point P of holder position laser beam to reference planes1, by point P1To laser The distance of measuring applianceAnd the vertical rotational angle α of holder1, horizontally rotate angle beta1, point P is calculated1In coordinate system XcYcZcUnder coordinate value
Similarly, laser beam can be obtained to second point P of reference planes2Coordinate valueAnd third point P3's Coordinate valueDetails are not described herein again;
By the normal vector of reference planesVector can be obtainedIt is referring to Projection vector on plane normal
Can further obtain, optical axis and reference planes apart from it is nearest when, vertical drift angle of the optical axis with respect to null directionLevel angle of the optical axis with respect to null direction
By grabbing the vertical rotational angle α of holder when image in step S1cWith horizontally rotate angle betac, optical axis can be obtained at that time Unit vector
It can further obtain, projection vectorAnd unit vectorBetween angle
It can further obtain, in the image of step S1 crawl, the translation vector of optical axis to reference planesAnd rotating vector (rotation RcWith translation tc) substitute into reference planes three-dimensional reference frame and two dimensional image coordinate system between mapping modelIn, the Three-dimensional Gravity to reference planes can be realized It builds, obtains the coordinate in image under three-dimensional reference frame corresponding to each pixel of reference planes;
Above-mentioned step S4 specifically:
In the present embodiment, the extracting method of project objects point is as follows:
To image application SharpMask image segmentation algorithm, the edge segmentation texture of objects in images is obtained, to belonging to S2 The texture pixel point of obtained non-reference plane type set of pixels range, calculates it in the subpoint of reference planes;
The subpoint refers to that the every bit on object projects to the point in reference planes, the line of object point and subpoint It is perpendicular to reference planes, i.e., parallel with the normal vector of reference planes;
The junction of object and reference planes, reflection are exactly the edge segmentation texture pixel point and ginseng of object on the image The adjacent area for examining image plane vegetarian refreshments, object edge segmentation texture pixel point eight connectivity neighborhood in, search whether include The reference planes classes of pixels collection that step S2 is obtained, comprising reference planes pixel be included in the object edge and divide texture picture The projection point set of vegetarian refreshments;Each reference planes pixel that texture pixel point is concentrated with subpoint respectively is combined, is obtained Plane and straight line collection of the texture pixel point in two dimensional image coordinate system;
In three-dimensional reference frame XwYwZwUnder, take the outer point n of a point m (0,0,0) and reference planes in reference planes (0, 0,1), it is known that vectorThat is the normal vector of reference planes substituted into point m, n and the three-dimensional ginseng of the plane vertical with reference planes Examine the mapping model between coordinate system and two dimensional image coordinate system In, the three-dimensional reconstruction to the vertical plane can be realized, obtain normal vectorStraight line under two dimensional image coordinate system;
According to the object point constraint condition principle parallel with the normal vector of reference planes with the line of subpoint, described flat Face straight line is focused to find out and normal vectorThe highest straight line of the straight line parallel degree of correlation, the parallel degree of correlation can two straight lines angle Make criterion, the subpoint reference planes pixel corresponding to the highest straight line of the parallel degree of correlation, as object point;And By the three-dimensional reconstruction result of step S2 reference planes, the coordinate P under three-dimensional reference frame corresponding to subpoint can be obtainedp=(Xp, Yp, Zp);
By unit vector in step S3With translation vector tc, optical axis vector when step S1 capturing pictures can be obtained
By the projection vector on reference planes normalIt can obtain, vectorProjection vector on the reference plane
With projection vectorAs normal vector, in conjunction with subpoint Pp, subpoint and vertical with reference planes flat can be obtained Face, this vertical plane are the plane where object point;
It can further obtain, vectorWith the intersection point (X of vertical planei, Yi, Zi), wherein
It can further obtain, the shift value of optical axis to vertical plane
Substitute into the mapping model between the three-dimensional reference frame and two dimensional image coordinate system of vertical planeIn, the reference of three-dimensional corresponding to object point can be obtained Coordinate under coordinate system;
More than repeating, coordinate of all objects point under three-dimensional reference frame in image can be obtained, complete in image The three-dimensional reconstruction of all objects;
As shown in figure 3, on reference planes ρ (building site flooring), having an oblique cutting in reference planes in two dimensional image On reinforcing bar, subpoint of pixel A, the B of reinforcing bar on reference planes ρ be a, b, subpoint a, b and reference planes ρ is obtained Vertical simulation plane π1、π2, can further obtain plane π1、π2Mapping model between two dimensional image, to obtain image slices The three-dimensional coordinate of vegetarian refreshments A, B;The rest may be inferred, can obtain the three-dimensional coordinate of reinforcing bar all pixels point, complete the Three-dimensional Gravity of entire reinforcing bar It builds.
Referring to Fig. 2, the embodiment of the present invention two are as follows:
A kind of monocular image three-dimensional reconstruction system based on reference planes provided by the invention, including one or more processing Device and memory, the memory are stored with program, which performs the steps of when being executed by processor
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes property The three-dimensional reconstruction of body.
It should be noted that 3-dimensional reconstruction of the present invention, refers to and acquires object unified three-dimensional by image Coordinate under reference frame, camera are a kind of mappings between D world space and two dimensional image, and mapping model can It indicates are as follows:
The mapping model indicates the homogeneous coordinates (X of certain point on three-dimensional reference framew, Yw, Zw, 1) with the point be mapped to two Relationship between the homogeneous coordinates (u, v, 1) fastened of dimension image coordinate, can be by camera inner parameter K and camera outside Parameter (rotation R and translation t) obtains.Wherein, camera inner parameterIt is camera shooting The intrinsic matrix of head, (u0, v0) it is projected position of the camera optical centre on CCD imaging plane, f is the focal length of camera, dxAnd dyIt is the physical size of each pixel of CCD in the horizontal direction and the vertical direction respectively.
Above-mentioned step S1 specifically:
In the present embodiment, rotating camera to the scene areas for needing three-dimensional reconstruction, camera carries out figure to the region As crawl, two dimensional image coordinate system is established;The two dimensional image coordinate system be using the upper left corner of two dimensional image as origin, to the right for U axis, the coordinate system created by v axis downwards;And the rotation angle value of camera optical axis, the optical axis rotation are obtained by holder Angle value includes the vertical rotational angle α of holdercWith horizontally rotate angle betac
Above-mentioned step S2 specifically:
In the present embodiment, a large amount of similar application scenarios pictures are acquired in advance, and super picture is carried out to image using SLIC algorithm Element processing, obtains the distribution situation of color of image and texture;Super-pixel with the same characteristics is grouped, identical characteristic is Refer to the area pixel in image with same class geometric meaning, for example for building site scene, generally divides the image into two class geometry Type, i.e. reference planes (construction surface) and the non-reference plane (object stretched out from reference planes, such as reinforcing bar, scaffold, concrete column Deng);After carrying out super-pixel grouping to the scene picture collection of acquisition, it is labeled that (reference planes or non-reference are put down to grouping Face), then pass through the geometric classfication model of deep learning creation scene;
After S1 step grabs image, disaggregated model analyzes image, is partitioned into reference planes geometric areas, Belong to the pixel in reference planes region in image labeled as reference planes type, other residual pixels are labeled as non-reference plane class Type;
Above-mentioned step S3 specifically:
In the present embodiment, for convenience of describing, being in initial position null direction with holder, (horizontal angle and vertical angle are all 0 Degree) when optical axis be ZcAxis establishes camera coordinate system XcYcZc;On the reference plane, it using optical axis as origin, is sat with camera Mark system XcYcZcChange in coordinate axis direction be reference direction, establish three-dimensional reference frame XwYwZw, wherein YwPerpendicular to reference planes;
Control holder respectively navigates to camera optical axis any three location points of reference planes, reference planes location point Positioning picture center n × n-pixel region can be compared and reference planes type that step S2 is obtained by image matching algorithm Set of pixels is determined, and then according to cloud platform rotation angle value and the distance of laser ranging, obtains three location points of reference planes In coordinate system XcYcZcUnder coordinate value;
In the present embodiment, pass through first location point P of holder position laser beam to reference planes1, by point P1To laser The distance of measuring applianceAnd the vertical rotational angle α of holder1, horizontally rotate angle beta1, point P is calculated1In coordinate system XcYcZcUnder coordinate value
Similarly, laser beam can be obtained to second point P of reference planes2Coordinate valueAnd third point P3's Coordinate valueDetails are not described herein again;
By the normal vector of reference planesVector can be obtainedIt is referring to Projection vector on plane normal
Can further obtain, optical axis and reference planes apart from it is nearest when, vertical drift angle of the optical axis with respect to null directionLevel angle of the optical axis with respect to null direction
By grabbing the vertical rotational angle α of holder when image in step S1cWith horizontally rotate angle betac, optical axis can be obtained at that time Unit vector
It can further obtain, projection vectorAnd unit vectorBetween angle
It can further obtain, in the image of step S1 crawl, the translation vector of optical axis to reference planesAnd rotating vector (rotation RcWith translation tc) substitute into reference planes three-dimensional reference frame and two dimensional image coordinate system between mapping modelIn, the Three-dimensional Gravity to reference planes can be realized It builds, obtains the coordinate in image under three-dimensional reference frame corresponding to each pixel of reference planes;
Above-mentioned step S4 specifically:
In the present embodiment, the extracting method of project objects point is as follows:
To image application SharpMask image segmentation algorithm, the edge segmentation texture of objects in images is obtained, to belonging to S2 The texture pixel point of obtained non-reference plane type set of pixels range, calculates it in the subpoint of reference planes;
The subpoint refers to that the every bit on object projects to the point in reference planes, the line of object point and subpoint It is perpendicular to reference planes, i.e., parallel with the normal vector of reference planes;
The junction of object and reference planes, reflection are exactly the edge segmentation texture pixel point and ginseng of object on the image The adjacent area for examining image plane vegetarian refreshments, object edge segmentation texture pixel point eight connectivity neighborhood in, search whether include The reference planes classes of pixels collection that step S2 is obtained, comprising reference planes pixel be included in the object edge and divide texture picture The projection point set of vegetarian refreshments;Each reference planes pixel that texture pixel point is concentrated with subpoint respectively is combined, is obtained Plane and straight line collection of the texture pixel point in two dimensional image coordinate system;
In three-dimensional reference frame XwYwZwUnder, take the outer point n of a point m (0,0,0) and reference planes in reference planes (0, 0,1), it is known that vectorThat is the normal vector of reference planes substituted into point m, n and the three-dimensional ginseng of the plane vertical with reference planes Examine the mapping model between coordinate system and two dimensional image coordinate system In, the three-dimensional reconstruction to the vertical plane can be realized, obtain normal vectorStraight line under two dimensional image coordinate system;
According to the object point constraint condition principle parallel with the normal vector of reference planes with the line of subpoint, described flat Face straight line is focused to find out and normal vectorThe highest straight line of the straight line parallel degree of correlation, the parallel degree of correlation can two straight lines angle Make criterion, the subpoint reference planes pixel corresponding to the highest straight line of the parallel degree of correlation, as object point;And By the three-dimensional reconstruction result of step S2 reference planes, the coordinate P under three-dimensional reference frame corresponding to subpoint can be obtainedp=(Xp, Yp, Zp);
By unit vector in step S3With translation vector tc, optical axis vector when step S1 capturing pictures can be obtained
By the projection vector on reference planes normalIt can obtain, vectorProjection vector on the reference plane
With projection vectorAs normal vector, in conjunction with subpoint Pp, subpoint and vertical with reference planes flat can be obtained Face, this vertical plane are the plane where object point;
It can further obtain, vectorWith the intersection point (X of vertical planei, Yi, Zi), wherein
It can further obtain, the shift value of optical axis to vertical plane
Substitute into the mapping model between the three-dimensional reference frame and two dimensional image coordinate system of vertical planeIn, the reference of three-dimensional corresponding to object point can be obtained Coordinate under coordinate system;
More than repeating, coordinate of all objects point under three-dimensional reference frame in image can be obtained, complete in image The three-dimensional reconstruction of all objects.
The embodiment of the present invention three are as follows:
The present invention also provides a kind of monocular image three-dimensional reconstruction apparatus based on reference planes, including interconnected take the photograph As head and three-dimensional reconstruction unit, the camera is configured as shooting and obtains a two dimensional image, and the content of the two dimensional image is Need the scene of three-dimensional reconstruction;
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple references in corresponding three-dimensional space Plane;According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;It extracts in the two dimensional image and owns Subpoint of the object in its corresponding reference planes, completes the three-dimensional reconstruction of all objects.
After the camera is specifically configured to rotating camera to the scene areas for needing three-dimensional reconstruction, pass through camera Image taking is carried out to scene areas, obtains a two dimensional image;
The three-dimensional reconstruction unit is specifically configured to analyze two dimensional image using disaggregated model, is partitioned into correspondence Multiple reference planes in three-dimensional space;
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional ginseng of reference planes It examines in the mapping model between coordinate system and two dimensional image coordinate system, completes the three-dimensional reconstruction of reference planes in two dimensional image;
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point is obtained The plane at place substitutes into the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of object point plane, complete to own in two dimensional image The three-dimensional reconstruction of object.
The three-dimensional reconstruction unit is specifically configured to that the pixel mark in reference planes region will be belonged in the two dimensional image It is denoted as reference planes type, the pixel in reference planes region is not belonging in the two dimensional image labeled as non-reference plane type.
In a particular embodiment, the above-mentioned monocular image three-dimensional reconstruction apparatus based on reference planes, including measurement end;Institute Stating measurement end includes laser, camera, angle adjustor and processor;The laser is arranged on camera, the laser, camera It is connect respectively with processor with angle adjustor, the laser and camera are connect with angle adjustor respectively, further include server-side and at least More than one terminal;The measurement end is connect with terminal respectively by server-side.Server-side is logical with measurement end and terminal respectively Cross network connection.Server-side provides the communication interface between measurement end and terminal, and server-side, which receives to come from/send, goes to measurement end Or the electric signal of terminal.Terminal shows visual output to user, including two dimensional image, the text information of three-dimensional reconstruction result, figure Shape information and any combination thereof.Terminal receives the control input of user, sends control signal to server-side, executes two dimensional image and grab It takes, obtains object dimensional reconstructed results in image.
In conclusion a kind of monocular image three-dimensional rebuilding method, system and dress based on reference planes provided by the invention It sets, using monocular to the three-dimensional reconstruction ability of plane, first creation reference planes, is then based on reference planes every in image A object point is mapped to each simulation plane, to obtain the depth of each object point, completes the three-dimensional reconstruction of whole image.It is tied Structure is simple, realizes and is easy, there is good scene ease for use.
The above description is only an embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair Equivalents made by bright specification and accompanying drawing content are applied directly or indirectly in relevant technical field, similarly include In scope of patent protection of the invention.

Claims (6)

1. a kind of monocular image three-dimensional rebuilding method based on reference planes, which comprises the following steps:
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes all objects Three-dimensional reconstruction;
Step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image taking is carried out to scene areas by camera, is obtained To a two dimensional image;
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space;
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional with reference to seat of reference planes In mapping model between mark system and two dimensional image coordinate system, the three-dimensional reconstruction of reference planes in two dimensional image is completed;
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point place is obtained Plane, the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector are substituted into object In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of point plane, all objects in two dimensional image are completed Three-dimensional reconstruction.
2. the monocular image three-dimensional rebuilding method according to claim 1 based on reference planes, which is characterized in that step S2 Further include:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, in the two dimensional image not Belong to the pixel in reference planes region labeled as non-reference plane type.
3. a kind of monocular image three-dimensional reconstruction system based on reference planes, which is characterized in that including one or more processors And memory, the memory are stored with program, which performs the steps of when being executed by processor
S1, shooting obtain a two dimensional image, and the content of the two dimensional image is to need the scene of three-dimensional reconstruction;
S2, the segmentation two dimensional image, obtain multiple reference planes in corresponding three-dimensional space;
S3, according to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;
S4, subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, completes all objects Three-dimensional reconstruction;
Step S1 specifically:
After rotating camera to the scene areas for needing three-dimensional reconstruction, image taking is carried out to scene areas by camera, is obtained To a two dimensional image;
Step S2 specifically:
Two dimensional image is analyzed using disaggregated model, the multiple reference planes being partitioned into corresponding three-dimensional space;
Step S3 specifically:
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional with reference to seat of reference planes In mapping model between mark system and two dimensional image coordinate system, the three-dimensional reconstruction of reference planes in two dimensional image is completed;
Step S4 specifically:
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point place is obtained Plane, the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector are substituted into object In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of point plane, all objects in two dimensional image are completed Three-dimensional reconstruction.
4. the monocular image three-dimensional reconstruction system according to claim 3 based on reference planes, which is characterized in that the program By the processor execute when further realize the following steps are included:
The pixel in reference planes region will be belonged in the two dimensional image labeled as reference planes type, in the two dimensional image not Belong to the pixel in reference planes region labeled as non-reference plane type.
5. a kind of monocular image three-dimensional reconstruction apparatus based on reference planes, which is characterized in that including camera interconnected And three-dimensional reconstruction unit, the camera are configured as shooting and obtain a two dimensional image, the content of the two dimensional image is to need The scene of three-dimensional reconstruction;
The three-dimensional reconstruction unit is configured as dividing the two dimensional image, obtains multiple with reference to flat in corresponding three-dimensional space Face;According to multiple reference planes complete the two dimensional image corresponding to three-dimensional reconstruction;Extract property in the two dimensional image Subpoint of the body in its corresponding reference planes, completes the three-dimensional reconstruction of all objects;
After the camera is specifically configured to rotating camera to the scene areas for needing three-dimensional reconstruction, by camera to field Scene area carries out image taking, obtains a two dimensional image;
The three-dimensional reconstruction unit is specifically configured to analyze two dimensional image using disaggregated model, is partitioned into corresponding three-dimensional Multiple reference planes in space;
The translation vector of the optical axis of the camera to reference planes and rotating vector are substituted into the three-dimensional with reference to seat of reference planes In mapping model between mark system and two dimensional image coordinate system, the three-dimensional reconstruction of reference planes in two dimensional image is completed;
Subpoint of all objects in its corresponding reference planes in the two dimensional image is extracted, the object point place is obtained Plane, the translation vector of the optical axis of camera corresponding to the object point to reference planes and rotating vector are substituted into object In mapping model between the three-dimensional reference frame and two dimensional image coordinate system of point plane, all objects in two dimensional image are completed Three-dimensional reconstruction.
6. the monocular image three-dimensional reconstruction apparatus according to claim 5 based on reference planes, which is characterized in that described three Dimension reconstruction unit is specifically configured to the pixel by reference planes region is belonged in the two dimensional image labeled as reference planes class Type is not belonging to the pixel in reference planes region in the two dimensional image labeled as non-reference plane type.
CN201811009447.XA 2018-08-31 2018-08-31 Monocular image three-dimensional rebuilding method, system and device based on reference planes Active CN109147027B (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201811009447.XA CN109147027B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional rebuilding method, system and device based on reference planes
CN201910964298.0A CN110827392B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device
CN201910963573.7A CN110838164B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device based on object point depth

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811009447.XA CN109147027B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional rebuilding method, system and device based on reference planes

Related Child Applications (2)

Application Number Title Priority Date Filing Date
CN201910963573.7A Division CN110838164B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device based on object point depth
CN201910964298.0A Division CN110827392B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device

Publications (2)

Publication Number Publication Date
CN109147027A CN109147027A (en) 2019-01-04
CN109147027B true CN109147027B (en) 2019-11-08

Family

ID=64825870

Family Applications (3)

Application Number Title Priority Date Filing Date
CN201910964298.0A Active CN110827392B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device
CN201910963573.7A Active CN110838164B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device based on object point depth
CN201811009447.XA Active CN109147027B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional rebuilding method, system and device based on reference planes

Family Applications Before (2)

Application Number Title Priority Date Filing Date
CN201910964298.0A Active CN110827392B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device
CN201910963573.7A Active CN110838164B (en) 2018-08-31 2018-08-31 Monocular image three-dimensional reconstruction method, system and device based on object point depth

Country Status (1)

Country Link
CN (3) CN110827392B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109741404B (en) * 2019-01-10 2020-11-17 奥本未来(北京)科技有限责任公司 Light field acquisition method based on mobile equipment
CN111220128B (en) * 2019-01-31 2022-10-25 金钱猫科技股份有限公司 Monocular focusing measuring method and terminal
CN112837404B (en) * 2019-11-25 2024-01-19 北京初速度科技有限公司 Method and device for constructing three-dimensional information of planar object
CN111415420B (en) * 2020-03-25 2024-01-23 北京迈格威科技有限公司 Spatial information determining method and device and electronic equipment
CN112198527B (en) * 2020-09-30 2022-12-27 上海炬佑智能科技有限公司 Reference plane adjustment and obstacle detection method, depth camera and navigation equipment
CN112198529B (en) * 2020-09-30 2022-12-27 上海炬佑智能科技有限公司 Reference plane adjustment and obstacle detection method, depth camera and navigation equipment
CN112884898B (en) * 2021-03-17 2022-06-07 杭州思看科技有限公司 Reference device for measuring texture mapping precision
CN114596406A (en) * 2022-01-25 2022-06-07 海拓信息技术(佛山)有限公司 Three-dimensional construction method and device based on monocular camera

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102708566A (en) * 2012-05-08 2012-10-03 天津工业大学 Novel single-camera and single-projection light source synchronous calibrating method
CN103578133A (en) * 2012-08-03 2014-02-12 浙江大华技术股份有限公司 Method and device for reconstructing two-dimensional image information in three-dimensional mode
CN104809755A (en) * 2015-04-09 2015-07-29 福州大学 Single-image-based cultural relic three-dimensional reconstruction method
CN106204717A (en) * 2015-05-28 2016-12-07 长沙维纳斯克信息技术有限公司 A kind of stereo-picture quick three-dimensional reconstructing method and device

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6240198B1 (en) * 1998-04-13 2001-05-29 Compaq Computer Corporation Method for figure tracking using 2-D registration
CN102697508B (en) * 2012-04-23 2013-10-16 中国人民解放军国防科学技术大学 Method for performing gait recognition by adopting three-dimensional reconstruction of monocular vision
CN103077524A (en) * 2013-01-25 2013-05-01 福州大学 Calibrating method of hybrid vision system
CN104424630A (en) * 2013-08-20 2015-03-18 华为技术有限公司 Three-dimension reconstruction method and device, and mobile terminal
CN105303554B (en) * 2015-09-16 2017-11-28 东软集团股份有限公司 The 3D method for reconstructing and device of a kind of image characteristic point
CN106960442A (en) * 2017-03-01 2017-07-18 东华大学 Based on the infrared night robot vision wide view-field three-D construction method of monocular
CN107063129B (en) * 2017-05-25 2019-06-07 西安知象光电科技有限公司 A kind of array parallel laser projection three-dimensional scan method
CN107945268B (en) * 2017-12-15 2019-11-29 深圳大学 A kind of high-precision three-dimensional method for reconstructing and system based on binary area-structure light
CN108062788A (en) * 2017-12-18 2018-05-22 北京锐安科技有限公司 A kind of three-dimensional rebuilding method, device, equipment and medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102708566A (en) * 2012-05-08 2012-10-03 天津工业大学 Novel single-camera and single-projection light source synchronous calibrating method
CN103578133A (en) * 2012-08-03 2014-02-12 浙江大华技术股份有限公司 Method and device for reconstructing two-dimensional image information in three-dimensional mode
CN104809755A (en) * 2015-04-09 2015-07-29 福州大学 Single-image-based cultural relic three-dimensional reconstruction method
CN106204717A (en) * 2015-05-28 2016-12-07 长沙维纳斯克信息技术有限公司 A kind of stereo-picture quick three-dimensional reconstructing method and device

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
"Accurate multiple view 3D reconstruction using patch-based stereo for large-scale scenes";Shen S;《IEEE Transactions on Image Processing》;20131231;全文 *
"医学图像三维重建研究综述";郑丽萍;《第十一届中国体视学与图像分析学术会议论文集》;20061031;全文 *
"单目图像序列光流三维重建技术研究综述";张聪炫;《电子学报》;20161231;第44卷(第12期);全文 *
"基于视觉的三维重建技术综述";佟帅;《计算机应用研究》;20110731;第28卷(第7期);全文 *
"基于计算机视觉的三维重建技术综述";徐超;《数字技术与应用》;20170115;全文 *

Also Published As

Publication number Publication date
CN109147027A (en) 2019-01-04
CN110827392A (en) 2020-02-21
CN110838164A (en) 2020-02-25
CN110827392B (en) 2023-03-24
CN110838164B (en) 2023-03-24

Similar Documents

Publication Publication Date Title
CN109147027B (en) Monocular image three-dimensional rebuilding method, system and device based on reference planes
CN110782524B (en) Indoor three-dimensional reconstruction method based on panoramic image
TW201915944A (en) Image processing method, apparatus, and storage medium
CN106462943A (en) Aligning panoramic imagery and aerial imagery
WO2020042970A1 (en) Three-dimensional modeling method and device therefor
WO2012153447A1 (en) Image processing device, image processing method, program, and integrated circuit
CN103971408A (en) Three-dimensional facial model generating system and method
KR20140057610A (en) Using motion parallax to create 3d perception from 2d images
CN104599317A (en) Mobile terminal and method for achieving 3D (three-dimensional) scanning modeling function
WO2018032841A1 (en) Method, device and system for drawing three-dimensional image
CN106264537B (en) System and method for measuring human body posture height in image
CN109510975A (en) A kind of extracting method of video image, equipment and system
CN113379901A (en) Method and system for establishing house live-action three-dimension by utilizing public self-photographing panoramic data
JP4354708B2 (en) Multi-view camera system
US10354399B2 (en) Multi-view back-projection to a light-field
CN111047678B (en) Three-dimensional face acquisition device and method
CN107864372A (en) Solid picture-taking method, apparatus and terminal
CN112102504A (en) Three-dimensional scene and two-dimensional image mixing method based on mixed reality
CN116860112A (en) Combined scene experience generation method, system and medium based on XR technology
Verhoeven et al. From pixel to mesh: accurate and straightforward 3D documentation of cultural heritage from the Cres/Lošinj archipelago
CN115205491A (en) Method and device for handheld multi-view three-dimensional reconstruction
CN110148206A (en) The fusion method in more spaces
CN115588069A (en) Scene reconstruction method and device, equipment and storage medium
CN114463520A (en) Method and device for realizing Virtual Reality (VR) roaming
Shimamura et al. Construction and presentation of a virtual environment using panoramic stereo images of a real scene and computer graphics models

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant