CN114708316B - Structured light three-dimensional reconstruction method and device based on circular stripes and electronic equipment - Google Patents
Structured light three-dimensional reconstruction method and device based on circular stripes and electronic equipment Download PDFInfo
- Publication number
- CN114708316B CN114708316B CN202210361004.7A CN202210361004A CN114708316B CN 114708316 B CN114708316 B CN 114708316B CN 202210361004 A CN202210361004 A CN 202210361004A CN 114708316 B CN114708316 B CN 114708316B
- Authority
- CN
- China
- Prior art keywords
- circular
- reconstructed
- stripe pattern
- projection
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/254—Projection of a pattern, viewing through a pattern, e.g. moiré
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
Abstract
The invention provides a structured light three-dimensional reconstruction method based on circular stripes, which relates to the technical field of three-dimensional imaging and comprises the following steps: acquiring a preconfigured circular stripe pattern; projecting the circular stripe pattern onto an object to be reconstructed, and controlling the circular stripe pattern to move relative to the object to be reconstructed; collecting an image set of an object to be reconstructed in the moving process of the circular stripe pattern; wherein the image set comprises a plurality of object images carrying circular stripe patterns; and carrying out three-dimensional reconstruction on the object to be reconstructed based on the image set to obtain an object model corresponding to the object to be reconstructed. The invention can obviously improve the reconstruction quality, reduce the use limit of three-dimensional reconstruction equipment and effectively simplify the calculation amount required by reconstruction.
Description
Technical Field
The invention relates to the technical field of three-dimensional imaging, in particular to a structured light three-dimensional reconstruction method and device based on circular stripes and electronic equipment.
Background
The structured light three-dimensional reconstruction method is a method for calculating a three-dimensional image of an object to be measured by projecting a series of fringe images onto the surface of the object to be measured through a projector, acquiring the fringe images modulated by the object to be measured through a camera, and then acquiring a phase diagram formed by each pixel. At present, the related art proposes that unidirectional stripes can be selected for projection, the scheme needs to limit the position relation between a camera and a projector, the unidirectional stripes are poor in quality under the placement condition of some cameras and projectors, a lifting space still exists, if bidirectional stripes in horizontal and vertical directions are selected for projection, the number of acquired images is twice that of unidirectional stripe projection, the projection time is long, and the operation amount is large.
Disclosure of Invention
In view of the above, the present invention aims to provide a structured light three-dimensional reconstruction method, apparatus and electronic device based on circular fringes, which can significantly improve the reconstruction quality, reduce the use limitation of the three-dimensional reconstruction device, and effectively simplify the calculation amount required by reconstruction.
In a first aspect, an embodiment of the present invention provides a method for three-dimensional reconstruction of structured light based on circular fringes, where the method is applied to a server, and the server is connected to a camera and a projector, and the method includes: acquiring a preconfigured circular stripe pattern; projecting the circular stripe pattern onto an object to be reconstructed, and controlling the circular stripe pattern to move relative to the object to be reconstructed; collecting an image set of an object to be reconstructed in the moving process of the circular stripe pattern; wherein the image set comprises a plurality of object images carrying circular stripe patterns; and carrying out three-dimensional reconstruction on the object to be reconstructed based on the image set to obtain an object model corresponding to the object to be reconstructed.
In one embodiment, the method is applied to a control end of a three-dimensional reconstruction device, the three-dimensional reconstruction device further comprising a projection device communicatively coupled to the control end; the step of acquiring a preconfigured circular fringe pattern comprises: according to a pre-calibrated projection matrix, determining the epipolar point of a projection plane of projection equipment, and calculating the nearest distance information and the farthest distance information between the epipolar point and the field of view of the projector; a circular fringe pattern is generated based on the pair pole, the closest distance information, and the farthest distance information.
In one embodiment, the step of generating a circular fringe pattern from the pair pole, the closest distance information, and the farthest distance information comprises: carrying out normalization processing on the spatial frequency of the circular stripe pattern according to the poles, the nearest distance information and the farthest distance information; the circular fringe pattern is determined based on the pair pole, the closest distance information, the farthest distance information, the highest frequency value, and a predetermined phase shift value.
In one embodiment, the higher the highest frequency value, the higher the accuracy of the three-dimensional reconstruction, within a certain range.
In one embodiment, the three-dimensional reconstruction device further comprises an image acquisition device in communication with the control end; projecting the circular fringe pattern onto the object to be reconstructed and controlling the movement of the circular fringe pattern relative to the object to be reconstructed, comprising: projecting the circular stripe pattern onto an object to be reconstructed by using a projection device, and controlling the circular stripe pattern to move relative to the object to be reconstructed by using the projection device according to a phase shift value and a highest frequency value; during the movement of the circular fringe pattern, the step of acquiring an image set of the object to be reconstructed comprises the following steps: during the movement of the circular fringe pattern, an image collection of the object to be reconstructed is acquired with an image acquisition device.
In one embodiment, the step of performing three-dimensional reconstruction on the object to be reconstructed based on the image set to obtain an object model corresponding to the object to be reconstructed includes: determining a target reconstruction parameter corresponding to each pixel point in an object image based on a two-dimensional lookup table pre-configured by a projection matrix; the two-dimensional lookup table is used for representing the mapping relation between the phase and the three-dimensional coordinates; calculating the polar diameter information of the circular stripe pattern carried by the object image; the electrode diameter information is used for representing the distance between each pixel point in the object image and the opposite electrode point of the projection plane; and determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters and the polar diameter information corresponding to each pixel point.
In one embodiment, the step of calculating the polar diameter information of the circular stripe pattern carried by the object image includes: determining a polar path expression according to the geometric relationship between each pixel point and the opposite polar point in the projection plane; and calculating the absolute phase corresponding to each pixel point according to a preconfigured absolute phase formula, and calculating the electrode path information according to the electrode path expression and the absolute phase.
In one embodiment, the step of determining the object model corresponding to the object to be reconstructed according to the target reconstruction parameter and the polar diameter information corresponding to each pixel point further includes: according to the geometrical relationship between the plane formed by the optical center of the image acquisition equipment, the optical center of the projection equipment and the optical poles and the projection matrix, resolving and correlating the phase of the circular stripe pattern with the three-dimensional coordinates, and determining a point cloud resolving formula; and determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters, the polar diameter information and the point cloud analytic type corresponding to each pixel point.
In a second aspect, an embodiment of the present invention further provides a device for three-dimensional reconstruction of structured light based on circular fringes, where the device is applied to a server, and the server is connected to a camera and a projector, and the device includes: the pattern generation module is used for acquiring a preconfigured circular stripe pattern; the projection module projects the circular stripe pattern onto the object to be reconstructed and controls the circular stripe pattern to move relative to the object to be reconstructed; the image acquisition module is used for acquiring an image set of an object to be reconstructed in the moving process of the circular stripe pattern; wherein the image set comprises a plurality of object images carrying circular stripe patterns; and the calculation module is used for carrying out three-dimensional reconstruction on the object to be reconstructed based on the image set to obtain an object model corresponding to the object to be reconstructed.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor, a memory, a structured light three-dimensional reconstruction device based on circular stripes, and a bus, where the memory stores computer executable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a structured light three-dimensional reconstruction method, equipment and electronic equipment based on circular stripes, wherein the method is applied to a server, the server is connected with a camera and a projector, a preconfigured circular stripe pattern is projected onto an object to be reconstructed, the circular stripe pattern is controlled to move relative to the object to be reconstructed, an image set of the object to be reconstructed is acquired in the moving process of the circular stripe pattern, and three-dimensional reconstruction is carried out on the object to be reconstructed based on the image set, so that an object model corresponding to the object to be reconstructed is obtained. When the method is used for reconstructing an image of an object to be measured, the circular stripes are adopted for projection, so that a set of object images carrying circular stripe patterns is acquired, and further, an object model with higher quality can be obtained based on the image set.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of a three-dimensional reconstruction device according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a three-dimensional reconstruction method of structured light based on circular stripes according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a circular stripe pattern according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a circular stripe pattern carried by an object image according to an embodiment of the present invention;
FIG. 5 is a schematic illustration of an extended epipolar geometry provided by embodiments of the present invention;
FIG. 6 is a schematic flow chart of another three-dimensional reconstruction method of structured light based on circular stripes according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a three-dimensional reconstruction device for structured light based on circular stripes according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described in conjunction with the embodiments, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The scheme is used for limiting the position relation between the camera and the projector, the quality of the reconstructed image is poor, and if the two-way stripes in the horizontal direction and the vertical direction are selected for projection, the number of the obtained images is twice that of the one-way stripe projections, the projection time is long, and the operation amount is large.
Currently, structured light three-dimensional reconstruction is generally performed on an object to be measured by adopting stripe projection profilometry (Fringe Projection Profilometry, FPP) or bidirectional stripe projection profilometry (i.e. scanning along horizontal and vertical directions), a series of stripe images are projected onto the surface of the object to be measured by a projector, and a phase map formed by each pixel is obtained after the stripe images modulated by the object to be measured are collected by a camera, so that a reconstructed image of the object to be measured is obtained. In the prior art, when three-dimensional reconstruction is performed by using stripe projection profilometry, the direction of phase shift (the waveform sampling on a space domain is performed, the higher the phase shift is, the more accurate the acquired image is, and the definition of the acquired image can be understood) needs to be matched with the position layout of a camera and a projector, if the direction of the phase shift is not matched with the position layout of the camera and the projector, the image quality of the three-dimensional reconstruction image can be reduced, and when three-dimensional reconstruction is performed by using bidirectional stripe projection profilometry, the stable performance can be realized, but the number of the obtained patterns is twice that of unidirectional scanning, so that the operand is improved, and the cost is further improved. Based on the above, the embodiment of the invention provides a structured light three-dimensional reconstruction method, equipment and electronic equipment based on circular fringes, which are based on polar geometry of circular fringe projection expansion, so that even if a camera-projector pair is placed at will, the best precision and the fastest calculation speed can be obtained in the whole projector space, the number of patterns does not need to be doubled, the furthest point and the closest point are introduced for the proposed circular fringe pattern, so that fringe frequency can be conveniently controlled, when a three-dimensional point cloud is calculated, the real-time calculation of the three-dimensional point cloud is realized by resolving and correlating the circular phase with 3D world coordinates, and reducing the number of equations in a two-way fringe projection contour operation from 6 to 3, the reconstruction quality can be remarkably improved, the use limit of the three-dimensional reconstruction equipment is reduced, and the calculation amount required by reconstruction can be effectively simplified.
In order to facilitate understanding of the present embodiment, first, a detailed description is provided of a three-dimensional reconstruction method of structured light based on circular fringes, which is disclosed in the present embodiment of the present invention, the method is applied to a control end of a three-dimensional reconstruction device, and is used for three-dimensionally reconstructing an object to be reconstructed based on a configured circular fringe pattern and an image set of the acquired object to be reconstructed, so as to obtain an object model corresponding to the object to be reconstructed.
Based on the schematic structural diagram of the three-dimensional reconstruction device shown in fig. 1, the embodiment of the present invention describes a three-dimensional reconstruction method of structured light based on circular fringes in detail, referring to a schematic flow diagram of the three-dimensional reconstruction method of structured light based on circular fringes shown in fig. 2, and the method mainly includes the following steps S202 to S206:
step S202, a preconfigured circular stripe pattern is acquired. The circular stripe pattern consists of circular stripes with black and white intervals and equal intervals, and the number of stripes contained in the circular stripe pattern is determined by the frequency f.
In one embodiment, the circular stripe pattern captured by the image capture device comprises three adjacent circular stripes.
In step S204, the circular fringe pattern is projected onto the object to be reconstructed, and the movement of the circular fringe pattern relative to the object to be reconstructed is controlled, wherein the movement rate of the circular fringe pattern relative to the object to be reconstructed is determined by the phase shift N and the frequency f, the phase shift represents sampling the waveform on the spatial domain, and the higher the phase shift, the more accurate the acquired image, which can be understood as the definition of the acquired image.
In one embodiment, a projection device is used to project a circular fringe pattern onto an object to be reconstructed, and the projection device is used to control the circular fringe pattern to move relative to the object to be reconstructed according to a phase shift value and a highest frequency value, wherein the expression of the circular fringe pattern is:
wherein, the liquid crystal display device comprises a liquid crystal display device,
gray value of circular fringe pattern, r 1 p For the closest distance information between the pole and the projection matrix,
for the furthest distance information between the pole and the projection matrix, r p Is the polar diameter of the circular stripe pattern.
Step S206, collecting an image set of the object to be reconstructed in the moving process of the circular stripe pattern.
In one embodiment, an image collection of the object to be reconstructed is acquired with an image acquisition device during the movement of the circular fringe pattern. The image acquisition is connected through a hardware circuit, and the images of the circular stripes projected on the object to be reconstructed are synchronously acquired.
Step S208, performing three-dimensional reconstruction on the object to be reconstructed based on the image set to obtain an object model corresponding to the object to be reconstructed. The three-dimensional reconstruction is to calculate three-dimensional coordinate information from all pixel points to the circle center of the circular stripe pattern in the circular stripe pattern carried by the object image.
In one embodiment, the three-dimensional coordinate information from any pixel point to the center of the circular fringe pattern is calculated as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is the optical center of the photographic device, < >>
Is the optical pole (the intersection of the line passing through the camera center and the image plane), -the (x)>
The polar diameter of the circular stripe pattern carried by the object image.
The structured light three-dimensional reconstruction method based on the circular stripes provided by the embodiment of the invention can obviously improve the reconstruction quality, reduce the use limit of three-dimensional reconstruction equipment, and effectively simplify the calculation amount required by reconstruction and shorten the projection time compared with bidirectional scanning.
The embodiment of the invention also provides an implementation mode for generating the circular stripe pattern, and the implementation mode is specifically described in the following (1) to (2):
(1) As shown in FIG. 3, epipolar points of a projection plane of the projection device are determined from a pre-calibrated projection matrix, and epipolar points are calculated
And projection matrixThe closest distance information and the farthest distance information, wherein the point of the closest distance is the imaging point of the optical center of the image acquisition device on the plane of the projection device, and the geometric center of the generated circular stripe pattern.
In one embodiment, the expression for the closest and farthest distances between the epipolar point and the projection matrix is calculated as:
(2) A circular fringe pattern is generated from the pair pole, the closest distance information, and the farthest distance information, see steps 1.1 through 1.2 below.
And 1.1, carrying out normalization processing on the spatial frequency of the circular fringe pattern according to the pole, the nearest distance information and the farthest distance information, wherein the phase shift value and the highest frequency value are fixed values, and the higher the frequency value is in a certain range, the higher the three-dimensional reconstruction precision is.
In one embodiment, the determined highest frequency value is obtained by normalizing the phase of the expression of the circular fringe pattern to [0,2 pi ].
Step 1.2, determining a circular fringe pattern according to the pole, the nearest distance information, the farthest distance information, the highest frequency value and a preset phase shift value, wherein A is an expression of gray values of the circular fringe pattern p 、B p N and N are the index and total number of phase shifts, r p For the polar diameter of the circular stripe pattern (the polar diameter of the circular stripe pattern generated by the control end and projected to the object to be measured), the polar diameter is calculated as follows:
in one embodiment, A p 、B p May be set to 127.5, which is half of the gray value.
The embodiment of the invention also provides an implementation mode for obtaining the object model corresponding to the object to be reconstructed based on the image set, and the implementation mode is specifically described in the following (1) to (3):
(1) Determining a target reconstruction parameter corresponding to each pixel point in an object image based on a two-dimensional lookup table pre-configured by a projection matrix; the two-dimensional lookup table is used for representing the mapping relation between the phase and the three-dimensional coordinates, and the numerical values of the pixel points on the two-dimensional lookup table are different.
In one embodiment, four two-dimensional look-up tables are calculated and stored
And->
Wherein the resolution corresponds to the resolution of the image acquisition device.
(2) The calculation of the polar diameter information of the circular stripe pattern carried by the object image can be seen from the following steps 2.1 to 2.2.
Step 2.1, determining a polar path expression according to the geometric relationship between each pixel point and the opposite polar point in the projection plane, wherein, as shown in fig. 4, the calculation expression of the polar path information of the circular stripe pattern carried by the object image is:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is a phase pole.
In one embodiment, as shown in fig. 5, the points of the projector plane and the points of the camera plane correspond to each other, being equally scaled points, according to epipolar geometry.
Step 2.2, calculating an absolute phase corresponding to each pixel point according to a preconfigured absolute phase formula, and calculating pole diameter information (pole diameter information of a circular stripe pattern carried by an object image and acquired by image acquisition equipment) according to a pole diameter expression and the absolute phase, wherein the expression of the pole diameter information is as follows:
wherein phi is the wrapping phase and phi is the absolute phase.
In one embodiment, the phase will be wrapped
The time domain expansion is performed to cover exactly [0,2 pi ]]Unwrapped phase->
Where k is the fringe order obtained when unwinding.
(3) And determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters and the polar diameter information corresponding to each pixel point, wherein the polar diameter information is used for representing the distance between each pixel point in the object image and the opposite polar point of the projection plane. In one embodiment, the object model corresponding to the object to be reconstructed can be determined as follows from step 3.1 to step 3.2.
Step 3.1, referring to fig. 5, according to the geometric relationship between the plane formed by the optical center of the image acquisition device, the optical center of the projection device and the optical pole and the projection matrix, resolving and correlating the phase of the circular fringe pattern with the three-dimensional coordinates, and determining a point cloud resolving formula, wherein the point cloud resolving formula can be obtained according to the polar geometric relationship of the image shown in fig. 5.
In one embodiment, the polar geometry is used to determine the point cloud resolution, thereby simplifying the computation of the point cloud.
And 3.2, determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters, the polar diameter information and the point cloud analysis type corresponding to each pixel point, wherein the pixel points are the pixel points in the object image acquired by the image acquisition equipment.
In order to facilitate understanding the three-dimensional reconstruction method of structured light based on circular stripes provided in the foregoing embodiments, an application example of the three-dimensional reconstruction method of structured light based on circular stripes is provided in the embodiments of the present invention, and referring to a schematic flow chart of another three-dimensional reconstruction method of structured light based on circular stripes shown in fig. 6, the method mainly includes the following steps S602 to S614:
step S602, calibrating projection matrixes of the camera and the projector, and determining an opposite pole of a projection plane. The point of the imaging point of the optical center of the image acquisition device on the plane of the projection device is the center of the geometrically generated circular fringe pattern.
In one embodiment, the expression for the closest and farthest distances between the epipolar point and the projection matrix is calculated as:
step S604, generating a circular stripe pattern according to the projection matrix and the epipolar point calculation, wherein the expression of the circular stripe pattern is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
gray value of circular fringe pattern, r 1 p For the closest distance information between the pole and the projection matrix,
for the furthest distance information between the pole and the projection matrix, r p The polar diameter of the circular stripe pattern is A p 、B p N and N are the index and total number of phase shifts, r p For the polar diameter of the circular stripe pattern (the polar diameter of the circular stripe pattern generated by the control end and projected to the object to be measured), the polar diameter is calculated as follows:
in one embodiment, A p 、B p May be set to 127.5, which is half of the gray value.
In step S606, the circular fringe pattern is projected onto the object to be reconstructed, and an image set of the object to be reconstructed is acquired, where the acquired image is equal to the size of the calibrated projection matrix.
In one embodiment, an image collection of the object to be reconstructed is acquired with an image acquisition device during the movement of the circular fringe pattern.
Step S608, a two-dimensional lookup table is calculated based on the projection matrix, and the polar diameter information of the circular stripe pattern carried by the object image is calculated, wherein the two-dimensional lookup table is used for representing the mapping relation between the phase and the three-dimensional coordinates, and the numerical values of all pixel points on the two-dimensional lookup table are different.
In one embodiment, four two-dimensional look-up tables are calculated and stored
And->
Wherein the resolution corresponds to the resolution of the image acquisition device. />
In another embodiment, the expression of the polar diameter information of the circular stripe pattern carried by the object image acquired by the image acquisition device is:
step S610, substituting the two-dimensional lookup table and the polar diameter information into a point cloud analysis type to obtain a three-dimensional point cloud, and further obtaining an object model of the object to be reconstructed, wherein the pixel points are the pixel points in the object image acquired by the image acquisition equipment.
In one embodiment, the three-dimensional coordinate information from any pixel point to the center of the circular fringe pattern is calculated as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is the optical center of the photographic device, < >>
Is the optical pole (the intersection of the line passing through the camera center and the image plane), -the (x)>
The polar diameter of the circular stripe pattern carried by the object image.
In conclusion, the invention can ensure the definition of the reconstructed image, reduce the use limit and simplify the operation process, thereby reducing the operation amount.
For the three-dimensional reconstruction method of structured light based on circular fringes provided in the foregoing embodiment, an embodiment of the present invention provides a three-dimensional reconstruction device of structured light based on circular fringes, where the device is applied to a server, and the server is connected to a camera and a projector, and referring to a schematic structural diagram of the three-dimensional reconstruction device of structured light based on circular fringes shown in fig. 7, the device includes the following parts:
the pattern generation module 702 acquires a circular stripe pattern configured in advance.
The projection module 704 projects the circular fringe pattern onto the object to be reconstructed and controls the circular fringe pattern to move relative to the object to be reconstructed.
An image acquisition module 706, which acquires an image set of the object to be reconstructed during the movement of the circular fringe pattern; wherein the image set comprises a plurality of object images carrying a circular fringe pattern.
The calculation module 708 performs three-dimensional reconstruction on the object to be reconstructed based on the image set, so as to obtain an object model corresponding to the object to be reconstructed.
According to the data processing device provided by the embodiment of the application, the epipolar geometry based on the expansion of the circular fringe projection is realized, even if a camera-projector pair is placed at will, the optimal precision and the fastest calculation speed can be obtained in the whole projector space, the number of patterns is not required to be doubled, the furthest point and the closest point are introduced for the proposed circular fringe pattern so as to conveniently control fringe frequency, when the three-dimensional point cloud is calculated, the number of equations in the bidirectional fringe projection contour operation is reduced from 6 to 3 by resolving and correlating the circular phase with the 3D world coordinates, the real-time calculation of the three-dimensional point cloud is realized, the calculation process is simplified, the number of fringe patterns is halved, the time of the fringe pattern projection and the point cloud calculation is reduced, the reconstruction precision of the three-dimensional point cloud is also improved, the reconstruction quality is remarkably improved, the use limit of three-dimensional reconstruction equipment is reduced, and the calculation amount required by reconstruction can be effectively simplified.
In one embodiment, when performing the step of obtaining the preconfigured circular stripe pattern, the pattern generating module 702 is further configured to: according to a pre-calibrated projection matrix, determining the epipolar point of a projection plane of projection equipment, and calculating the nearest distance information and the farthest distance information between the epipolar point and the field of view of the projector; a circular fringe pattern is generated based on the pair pole, the closest distance information, and the farthest distance information.
In one embodiment, when performing the step of generating the circular stripe pattern according to the pair of poles, the closest distance information and the farthest distance information, the pattern generating module 702 is further configured to: carrying out normalization processing on the spatial frequency of the circular stripe pattern according to the poles, the nearest distance information and the farthest distance information; and determining a circular stripe pattern according to the pole, the nearest distance information, the farthest distance information, the highest frequency value and a preset phase shift value, wherein the higher the highest frequency value is, the higher the three-dimensional reconstruction accuracy is in a certain range.
In one embodiment, in performing the step of projecting the circular fringe pattern onto the object to be reconstructed and controlling the movement of the circular fringe pattern relative to the object to be reconstructed, the projection module 704 is further configured to: the circular fringe pattern is projected onto the object to be reconstructed by the projection device, and the movement of the circular fringe pattern relative to the object to be reconstructed is controlled by the projection device according to the phase shift value and the highest frequency value.
In one embodiment, when performing the step of acquiring the image set of the object to be reconstructed during the movement of the circular fringe pattern, the image acquisition module 706 is further configured to: during the movement of the circular fringe pattern, an image collection of the object to be reconstructed is acquired with an image acquisition device.
In one embodiment, when performing the step of three-dimensional reconstruction of the object to be reconstructed based on the image set to obtain the object model corresponding to the object to be reconstructed, the calculation module 708 is further configured to: determining a target reconstruction parameter corresponding to each pixel point in an object image based on a two-dimensional lookup table pre-configured by a projection matrix; the two-dimensional lookup table is used for representing the mapping relation between the phase and the three-dimensional coordinates; calculating the polar diameter information of the circular stripe pattern carried by the object image; the electrode diameter information is used for representing the distance between each pixel point in the object image and the opposite electrode point of the projection plane; and determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters and the polar diameter information corresponding to each pixel point.
In one embodiment, when performing the step of calculating the polar diameter information of the circular stripe pattern carried by the object image, the calculating module 708 is further configured to: determining a polar path expression according to the geometric relationship between each pixel point and the opposite polar point in the projection plane; and calculating the absolute phase corresponding to each pixel point according to a preconfigured absolute phase formula, and calculating the electrode path information according to the electrode path expression and the absolute phase.
In one embodiment, when performing the step of determining the object model corresponding to the object to be reconstructed according to the target reconstruction parameter and the polar diameter information corresponding to each pixel, the calculation module 708 is further configured to: according to the geometrical relationship between the plane formed by the optical center of the image acquisition equipment, the optical center of the projection equipment and the optical poles and the projection matrix, resolving and correlating the phase of the circular stripe pattern with the three-dimensional coordinates, and determining a point cloud resolving formula; and determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters, the polar diameter information and the point cloud analytic type corresponding to each pixel point.
The device provided by the embodiment of the present invention has the same implementation principle and technical effects as those of the foregoing method embodiment, and for the sake of brevity, reference may be made to the corresponding content in the foregoing method embodiment where the device embodiment is not mentioned.
The embodiment of the invention provides an electronic device, in particular to the electronic device, which comprises a processor, a memory, a structured light three-dimensional reconstruction device based on circular stripes and a bus, wherein the memory stores computer executable instructions capable of being executed by the processor, when the electronic device runs, the processor and the memory are communicated through the bus, and the processor executes the method of any one of the embodiments.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device 100 includes: the processor 80, the memory 81, the bus 82 and the communication interface 83, and the processor 80, the communication interface 83 and the memory 81 are connected through the bus 82; the processor 80 is arranged to execute executable modules, such as computer programs, stored in the memory 81.
The memory 81 may include a high-speed random access memory (RAM, random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. The communication connection between the system network element and at least one other network element is implemented via at least one communication interface 83 (which may be wired or wireless), and may use the internet, a wide area network, a local network, a metropolitan area network, etc.
Bus 82 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 8, but not only one bus or type of bus.
The memory 81 is used for storing a program, and the processor 80 executes the program after receiving the execution instruction, and the method executed by the apparatus for defining a flow in any of the foregoing embodiments of the present invention may be applied to the processor 80 or implemented by the processor 80.
The processor 80 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware or instructions in software in processor 80. The processor 80 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal processor (Digital Signal Processing, DSP for short), application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 81 and the processor 80 reads the information in the memory 81 and in combination with its hardware performs the steps of the method described above.
The computer program product of the readable storage medium provided by the embodiment of the present invention includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method in the foregoing method embodiment, and specific implementation may refer to the foregoing method embodiment and will not be described herein.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method of the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention for illustrating the technical solution of the present invention, but not for limiting the scope of the present invention, and although the present invention has been described in detail with reference to the foregoing examples, it will be understood by those skilled in the art that the present invention is not limited thereto: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (5)
1. A structured light three-dimensional reconstruction method based on circular fringes, characterized in that it is applied to a server, which is connected to a camera and a projector, the method comprising:
acquiring a preconfigured circular stripe pattern;
projecting the circular stripe pattern onto an object to be reconstructed, and controlling the circular stripe pattern to move relative to the object to be reconstructed;
acquiring an image set of the object to be reconstructed in the moving process of the circular stripe pattern; wherein the image set comprises a plurality of object images carrying the circular fringe pattern;
performing three-dimensional reconstruction on the object to be reconstructed based on the image set to obtain an object model corresponding to the object to be reconstructed;
determining a target reconstruction parameter corresponding to each pixel point in the object image based on a two-dimensional lookup table pre-configured by a projection matrix; the two-dimensional lookup table is used for representing the mapping relation between the phase and the three-dimensional coordinates; calculating the polar diameter information of the circular stripe pattern carried by the object image; the polar diameter information is used for representing the distance between each pixel point in the object image and the opposite polar point of the projection plane; determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters and the polar path information corresponding to each pixel point;
according to the geometrical relationship between the plane formed by the optical center of the image acquisition device, the optical center of the projection device and the optical poles and the projection matrix, resolving and correlating the phase of the circular stripe pattern with the three-dimensional coordinates, and determining a point cloud resolving formula; and determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters, the polar path information and the point cloud analytic type corresponding to each pixel point.
2. The method of claim 1, wherein the method is applied to a control terminal of a three-dimensional reconstruction device, the three-dimensional reconstruction device further comprising a projection device communicatively coupled to the control terminal;
the step of obtaining a preconfigured circular fringe pattern comprises the following steps:
determining the opposite poles of a projection plane of the projection equipment according to a pre-calibrated projection matrix, and calculating the nearest distance information and the farthest distance information between the opposite poles and the field of view of the projector;
generating a circular stripe pattern according to the pair pole, the nearest distance information and the farthest distance information;
the spatial frequency of the circular fringe pattern is normalized according to the pair pole, the nearest distance information and the farthest distance information; determining a circular stripe pattern according to the pair pole, the nearest distance information, the farthest distance information, the highest frequency value and a preset phase shift value;
wherein, in a certain range, the higher the highest frequency value is, the higher the three-dimensional reconstruction precision is;
the three-dimensional reconstruction device further comprises an image acquisition device which is in communication connection with the control end; the step of projecting the circular fringe pattern onto an object to be reconstructed and controlling the circular fringe pattern to move relative to the object to be reconstructed comprises the following steps: projecting the circular fringe pattern onto an object to be reconstructed by using the projection equipment, and controlling the circular fringe pattern to move relative to the object to be reconstructed by using the projection equipment according to the phase shift value and the highest frequency value;
wherein, during the moving process of the circular stripe pattern, the step of collecting the image set of the object to be reconstructed comprises the following steps: and acquiring an image set of the object to be reconstructed by using the image acquisition equipment in the moving process of the circular stripe pattern.
3. The method according to claim 1, wherein the step of calculating the polar diameter information of the circular fringe pattern carried by the object image includes:
determining a polar path expression according to the geometric relationship between each pixel point and the opposite polar point in the projection plane;
and calculating the absolute phase corresponding to each pixel point according to a preconfigured absolute phase formula, and calculating the electrode path information according to the electrode path expression and the absolute phase.
4. A structured light three-dimensional reconstruction device based on circular stripes, the device being applied to a server, the server being connected to a camera and a projector, the device comprising:
the pattern generation module is used for acquiring a preconfigured circular stripe pattern;
the projection module is used for projecting the circular stripe pattern onto an object to be reconstructed and controlling the circular stripe pattern to move relative to the object to be reconstructed;
the image acquisition module is used for acquiring an image set of the object to be reconstructed in the moving process of the circular stripe pattern; wherein the image set comprises a plurality of object images carrying the circular fringe pattern;
the computing module is used for carrying out three-dimensional reconstruction on the object to be reconstructed based on the image set to obtain an object model corresponding to the object to be reconstructed;
determining a target reconstruction parameter corresponding to each pixel point in the object image based on a two-dimensional lookup table pre-configured by a projection matrix; the two-dimensional lookup table is used for representing the mapping relation between the phase and the three-dimensional coordinates; calculating the polar diameter information of the circular stripe pattern carried by the object image; the polar diameter information is used for representing the distance between each pixel point in the object image and the opposite polar point of the projection plane; determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters and the polar path information corresponding to each pixel point;
according to the geometrical relationship between the plane formed by the optical center of the image acquisition device, the optical center of the projection device and the optical poles and the projection matrix, resolving and correlating the phase of the circular stripe pattern with the three-dimensional coordinates, and determining a point cloud resolving formula; and determining an object model corresponding to the object to be reconstructed according to the target reconstruction parameters, the polar path information and the point cloud analytic type corresponding to each pixel point.
5. An electronic device comprising a processor, a memory, a structured light three-dimensional reconstruction device based on circular stripes, and a bus, the memory storing computer executable instructions executable by the processor, the processor and the memory in communication via the bus when the electronic device is running, the processor executing the computer executable instructions to implement the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210361004.7A CN114708316B (en) | 2022-04-07 | 2022-04-07 | Structured light three-dimensional reconstruction method and device based on circular stripes and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210361004.7A CN114708316B (en) | 2022-04-07 | 2022-04-07 | Structured light three-dimensional reconstruction method and device based on circular stripes and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114708316A CN114708316A (en) | 2022-07-05 |
| CN114708316B true CN114708316B (en) | 2023-05-05 |
Family
ID=82173195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210361004.7A Active CN114708316B (en) | 2022-04-07 | 2022-04-07 | Structured light three-dimensional reconstruction method and device based on circular stripes and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114708316B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115379182B (en) * | 2022-08-19 | 2023-11-24 | 四川大学 | Bidirectional structure optical coding and decoding method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104406702A (en) * | 2014-11-18 | 2015-03-11 | 中国科学院上海光学精密机械研究所 | Method for detecting light beam collimation |
| CN110288642A (en) * | 2019-05-25 | 2019-09-27 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Three-dimension object fast reconstructing method based on camera array |
| CN113505626A (en) * | 2021-03-15 | 2021-10-15 | 南京理工大学 | Rapid three-dimensional fingerprint acquisition method and system |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018119771A1 (en) * | 2016-12-28 | 2018-07-05 | 深圳大学 | Efficient phase-three-dimensional mapping method and system based on fringe projection profilometry |
| US11605172B2 (en) * | 2017-12-08 | 2023-03-14 | Arizona Board Of Regents On Behalf Of The University Of Arizona | Digital fringe projection and multi-spectral polarization imaging for rapid 3D reconstruction |
| CN108413872B (en) * | 2018-04-10 | 2020-03-10 | 天津科技大学 | Three-dimensional size precision measurement method based on Fabry-Perot multi-beam interference |
| CN109489585B (en) * | 2018-12-06 | 2020-08-18 | 广西师范大学 | Three-dimensional measurement method based on improved multi-frequency stripe structured light |
| CN110310365B (en) * | 2019-06-27 | 2021-01-05 | 四川大学 | Three-dimensional reconstruction method and device |
| CN110285775B (en) * | 2019-08-02 | 2020-07-31 | 四川大学 | Three-dimensional reconstruction method and system based on structured light period coding pattern |
| CN110595389A (en) * | 2019-09-02 | 2019-12-20 | 深圳技术大学 | Monocular-lens-based acquisition device and three-dimensional reconstruction imaging system |
| CN110728177B (en) * | 2019-09-02 | 2022-06-14 | 华南理工大学 | Noise-like data low-frequency oscillation identification method based on double covariance random subspace |
| CN111242990B (en) * | 2020-01-06 | 2024-01-30 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | 360-degree three-dimensional reconstruction optimization method based on continuous phase dense matching |
| CN111721236B (en) * | 2020-05-24 | 2022-10-25 | 奥比中光科技集团股份有限公司 | Three-dimensional measurement system and method and computer equipment |
| CN112712585B (en) * | 2020-12-15 | 2024-02-09 | 四川川大智胜软件股份有限公司 | Three-dimensional imaging system and method based on arc binary coding phase shift fringe projection |
| CN112762859B (en) * | 2020-12-22 | 2022-08-09 | 四川大学 | High-precision three-dimensional measuring device for sine stripe structured light of non-digital optical machine |
-
2022
- 2022-04-07 CN CN202210361004.7A patent/CN114708316B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104406702A (en) * | 2014-11-18 | 2015-03-11 | 中国科学院上海光学精密机械研究所 | Method for detecting light beam collimation |
| CN110288642A (en) * | 2019-05-25 | 2019-09-27 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Three-dimension object fast reconstructing method based on camera array |
| CN113505626A (en) * | 2021-03-15 | 2021-10-15 | 南京理工大学 | Rapid three-dimensional fingerprint acquisition method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114708316A (en) | 2022-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200334840A1 (en) | Three-Dimensional Scanning System and Scanning Method Thereof | |
| CN108267097B (en) | Three-dimensional reconstruction method and device based on binocular three-dimensional scanning system | |
| CN111721236B (en) | Three-dimensional measurement system and method and computer equipment | |
| CN103236076A (en) | Three-dimensional object model reconstruction system and method based on laser images | |
| CN104596439A (en) | Speckle matching and three-dimensional measuring method based on phase information aiding | |
| CN101697233A (en) | Structured light-based three-dimensional object surface reconstruction method | |
| KR20130032368A (en) | Three-dimensional measurement apparatus, three-dimensional measurement method, and storage medium | |
| CN109307483A (en) | A kind of phase developing method based on structured-light system geometrical constraint | |
| CN114708316B (en) | Structured light three-dimensional reconstruction method and device based on circular stripes and electronic equipment | |
| CN114526692B (en) | Structured light three-dimensional measurement method and device based on defocusing unwrapping | |
| CN114792345B (en) | Calibration method based on monocular structured light system | |
| CN114612409A (en) | Projection calibration method and device, storage medium and electronic equipment | |
| CN104634323A (en) | Multi-stage photographing three-dimensional photographic system and method | |
| CN104200456A (en) | Decoding method for linear structure-light three-dimensional measurement | |
| CN116608794B (en) | Anti-texture 3D structured light imaging method, system, device and storage medium | |
| CN110141800B (en) | Accelerator optical distance scale equipment, calibration method and optical distance scale generation method | |
| CN110310492B (en) | Speed measuring method and device for mobile vehicle | |
| CN116993901A (en) | Three-dimensional image reconstruction method based on binocular structured light | |
| JP3370418B2 (en) | 3D shape measurement system | |
| CN112254653B (en) | Program control method for 3D information acquisition | |
| CN107547798A (en) | Shooting focusing control method, device and terminal device | |
| CN212843399U (en) | Portable three-dimensional measuring equipment | |
| CN115379182B (en) | Bidirectional structure optical coding and decoding method and device, electronic equipment and storage medium | |
| CN109785339A (en) | A kind of method of while segmentation and the accurate complex-curved interference fringe image of registration | |
| CN116558445B (en) | Phase decoding method of stripe projection and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |