CN114459380A - Method for acquiring folding phase, and method and system for three-dimensional reconstruction - Google Patents
Method for acquiring folding phase, and method and system for three-dimensional reconstruction Download PDFInfo
- Publication number
- CN114459380A CN114459380A CN202210085619.1A CN202210085619A CN114459380A CN 114459380 A CN114459380 A CN 114459380A CN 202210085619 A CN202210085619 A CN 202210085619A CN 114459380 A CN114459380 A CN 114459380A
- Authority
- CN
- China
- Prior art keywords
- image
- phase
- acquiring
- camera
- phase shift
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 230000010363 phase shift Effects 0.000 claims abstract description 74
- 238000003860 storage Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 abstract description 11
- 238000004422 calculation algorithm Methods 0.000 abstract description 7
- 238000009825 accumulation Methods 0.000 abstract description 6
- 230000000694 effects Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 12
- 238000007796 conventional method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000001360 synchronised effect Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000001186 cumulative effect Effects 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000004134 energy conservation Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/254—Projection of a pattern, viewing through a pattern, e.g. moiré
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The invention provides a method for acquiring a folding phase, a three-dimensional reconstruction method and a three-dimensional reconstruction system, wherein the method for acquiring the folding phase comprises the following steps: the projection unit projects the background light intensity to the object to be measured in a scanning mode, and a camera is used for collecting a background image; in the one-time exposure time of the camera, the projection unit respectively projects a plurality of different phase shift graphs to an object to be measured; carrying out light field superposition on a plurality of different phase shift graphs by using a camera to obtain a light field superposition graph, wherein the light field superposition graph is a numerator image or a denominator image; sequentially acquiring a plurality of molecular images and denominator images of fringe frequencies; and acquiring the folding phase of each fringe frequency by using the background image, the molecular image corresponding to each fringe frequency and the denominator image. The image collected by the camera in one exposure time comprises more than two phase shift patterns, so that the image collection time can be reduced; the camera has an addition effect on the light intensity, and the accumulation operation in the phase calculation expression is preposed in an image acquisition link, so that the calculation amount of a subsequent algorithm is omitted.
Description
Technical Field
The invention relates to the technical field of fringe projection, in particular to a method for acquiring a folding phase, a three-dimensional reconstruction method and a three-dimensional reconstruction system.
Background
In fringe projection profiling, the galvanometer is required to project multiple phase shift maps in order to acquire the folded phase. Common projection systems are Digital Micromirror Devices (DMDs), Liquid Crystal Displays (LCDs), and the like. Taking DMD as an example, assume that its sinusoidal fringe projection speed is about 120 fps. If double-frequency heterodyne 4-step phase shift is adopted for phase expansion, 8 phase shift graphs are required to be projected by a DMD projection system for one-time three-dimensional reconstruction. Then image acquisition alone limits the three-dimensional reconstruction speed to below 15 fps. Therefore, on one hand, the three-dimensional reconstruction speed cannot meet the real-time requirement, and on the other hand, the three-dimensional reconstruction precision is possibly not high enough due to the fact that the number of phase shift steps is small. At present, one-dimensional MEMS galvanometers and linear lasers are also adopted as projection systems. The MEMS galvanometer vibrating at high frequency rapidly projects a phase shift image to a measured object through reflected line laser. From the energy conservation analysis, the line laser is formed by the point laser passing through a Powell prism, and the total energy of the whole phase shift diagram is equal to the energy of the point laser. Generally, a pattern can be projected by a single scan of the MEMS galvanometer. When the pattern intensity is sufficiently large, it can be directly collected as a phase shift map using a camera. However, the working distance of the projection system directly affects the size of the projected pattern. As the working distance increases, the actual physical size of the projected pattern becomes larger and the average light intensity will also decrease. In addition, if the laser power is low and the scanning frequency of the MEMS galvanometer is high, the average light intensity is also low, resulting in low contrast of the acquired image, which directly affects the accuracy and correctness of three-dimensional reconstruction. To avoid capturing a darker image, the exposure time of the camera is typically much longer than the scan period of the high frequency projection system. In other words, the projection system tends to scan multiple times within one exposure time of the camera. In this case, the conventional fringe projection method projects the same phase shift map in multiple scans, and the image acquired by the camera is only a phase shift map containing one phase information.
As described above, with the conventional fringe projection method, the speed and accuracy of three-dimensional reconstruction may not be compatible at the same time.
The above background disclosure is only for the purpose of assisting understanding of the concept and technical solution of the present invention and does not necessarily belong to the prior art of the present patent application, and should not be used for evaluating the novelty and inventive step of the present application in the case that there is no clear evidence that the above content is disclosed at the filing date of the present patent application.
Disclosure of Invention
The invention provides a method for acquiring a folding phase, a method for three-dimensional reconstruction and a system for three-dimensional reconstruction, aiming at solving the existing problems.
In order to solve the above problems, the technical solution adopted by the present invention is as follows:
the invention provides a method for acquiring a folding phase, which comprises the following steps: s1: the projection unit projects the background light intensity to the object to be measured in a scanning mode, and a camera is used for collecting a background image; s2: in one exposure time of the camera, the projection unit respectively projects a plurality of different phase shift graphs to the object to be measured; s3: performing light field superposition on a plurality of different phase shift graphs by using the camera to obtain a light field superposition graph, wherein the light field superposition graph is a numerator image or a denominator image; s4: sequentially acquiring the numerator images and the denominator images of a plurality of fringe frequencies; s5: and acquiring the folding phase of each fringe frequency by using a background image, the molecular image and the denominator image corresponding to each fringe frequency.
Preferably, the expression of the folding phase is obtained as follows:
wherein ,
n is the total number of scans in one exposure time, and N represents the number of scans.
Preferably, at the nth scan, the projected background light intensity is expressed as:
to acquire the molecular image, at the nth scan, the phase shift map of the projection is represented as:
to acquire the denominator image, at the nth scan, the phase shift map of the projection is represented as:
where a is the background light intensity and b is the light intensity modulation degree.
Preferably, the light intensity modulation degree is equal to the background light intensity by controlling the projection unit when the phase shift map is projected.
Preferably, the numerator image, the denominator image, and the background image of one of the streak frequencies acquired by the camera are represented as:
preferably, the folding phase is acquired from the numerator image, the denominator image and the background image:
the invention also provides a three-dimensional reconstruction method, which comprises the following steps: acquiring the folding phase of each fringe frequency by adopting the method for acquiring the folding phase as described in any one of the above; unfolding the folding phase to obtain the absolute phase of the highest fringe frequency in the fringe frequencies; calibrating a three-dimensional reconstruction system consisting of the projection unit and the camera, and determining a phase-height conversion parameter; and converting the absolute phase of the highest frequency into height by using the conversion parameters to obtain the three-dimensional point cloud.
The present invention further provides a three-dimensional reconstruction system for implementing the three-dimensional reconstruction method, including: the high-frequency projection unit is used for projecting light intensity to the object to be measured; the image acquisition unit is used for acquiring the light intensity on the object to be detected; the scanning frequency of the high-frequency projection unit is greater than the frame rate of the image acquisition unit.
Preferably, the high-frequency projection unit is a fringe projection unit comprising a laser and a MEMS galvanometer; the laser is used for emitting laser outwards; and the MEMS galvanometer is used for continuously vibrating and scanning, reflecting the laser and projecting light intensity to an object to be measured.
The invention further relates to a computer-readable storage medium, in which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method according to any of the above.
The invention has the beneficial effects that: the method comprises the steps of projecting different phase shift patterns in multiple scanning within one exposure time, wherein the image acquired by a camera can comprise more than two phase shift patterns; furthermore, the camera has an addition effect on the light intensity, and the accumulation operation in the phase calculation expression is preposed in an image acquisition link, so that the calculation amount of a subsequent algorithm is omitted; in one exposure time, a numerator image and a denominator image in the phase solution expression can be directly acquired. The method is applied to the fringe projection and folding phase resolving link in the three-dimensional reconstruction based on the structured light, and the speed and the precision of the three-dimensional reconstruction are improved.
Drawings
Fig. 1 is a schematic diagram of a method for obtaining a folding phase according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a process of obtaining a folding phase according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a three-dimensional reconstruction method according to an embodiment of the present invention.
Fig. 4 is a schematic flow chart of three-dimensional reconstruction according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram of a three-dimensional reconstruction in an embodiment of the present invention.
Fig. 6(a) -6 (d) are schematic diagrams of a seed image, a denominator image, a background image and an acquired phase result, respectively, according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the embodiments of the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly on the other element or be indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element. In addition, the connection may be for either a fixing function or a circuit connection function.
It is to be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship indicated in the drawings for convenience in describing the embodiments of the present invention and to simplify the description, and are not intended to indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be in any way limiting of the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of the present invention, "a plurality" means two or more unless specifically limited otherwise.
As shown in fig. 1, a method for obtaining a folding phase of the present invention includes the following steps:
s1: the projection unit projects the background light intensity to the object to be measured in a scanning mode, and a camera is used for collecting a background image;
s2: in one exposure time of the camera, the projection unit respectively projects a plurality of different phase shift graphs to the object to be measured;
s3: performing light field superposition on a plurality of different phase shift graphs by using the camera to obtain a light field superposition graph, wherein the light field superposition graph is a numerator image or a denominator image;
s4: sequentially acquiring the numerator images and the denominator images of a plurality of fringe frequencies;
s5: and acquiring the folding phase of each fringe frequency by using a background image, the molecular image and the denominator image corresponding to each fringe frequency.
In order to acquire a folding phase, a conventional three-dimensional reconstruction method needs to acquire a plurality of phase shift maps, respectively, and in the prior art method: (1) each phase shift graph corresponds to one exposure time, so that the three-dimensional reconstruction speed cannot meet the real-time requirement due to the fact that the time for acquiring the images is too long; (2) the total number of different phase shift graphs is equal to the number of phase shift steps, and the larger the number of phase shift steps is, the higher the precision of the acquired folding phase is, and further the precision of three-dimensional reconstruction is. Because different phase shift graphs are respectively acquired, the phase shift steps are generally set manually in a small number in consideration of the three-dimensional reconstruction speed, and the three-dimensional reconstruction precision is not high enough.
Common conventional projection units are Digital Micromirror Devices (DMDs), Liquid Crystal Displays (LCDs), etc. Conventional projection units typically have a lower projection frequency than high frequency projection units, such as MEMS projection units. This makes conventional projection units generally unable to project multiple images within one exposure time of the camera. The following description is given by way of example of a MEMS projection unit, but does not represent that the invention can only be used with MEMS projection units.
For a high frequency MEMS projection unit, the period of scanning is very short, and multiple images can be projected within one exposure time. With the development of MEMS galvanometer technology, it is becoming the mainstream of fringe projection unit. There are two projection schemes for existing MEMS projection units: (1) by reducing the exposure time, the MEMS galvanometer is exposed once after scanning once, although the time for image acquisition is short, the acquired image is generally dark due to the short exposure time and low contrast, and the accuracy and correctness of three-dimensional reconstruction are directly influenced; (2) without reducing the exposure time, the projection unit scans a plurality of times, projecting the same phase shift map each time. The effect obtained in this way is consistent with the projection mode of conventional projection units.
The invention provides a strategy for projecting different phase shift graphs within one exposure time by utilizing the characteristic that an MEMS projection unit can scan for multiple times within one exposure time. However, the implementation of this strategy has certain difficulties: (1) the scanning frequency of the used projection unit is required to be greater than the frame rate of the camera, and the larger the difference between the scanning frequency and the frame rate, the better the difference between the scanning frequency and the frame rate, but the lower the projection frequency of the traditional projection unit is, the better the difference between the scanning frequency and the frame rate is, the lower the projection frequency is, the better the difference between the scanning frequency and the frame rate is; (2) different phase shift graphs are directly projected within one exposure time, and the folding phase is difficult to calculate, so that the phase shift graph projected each time needs to be further designed, so that the phase shift graph can be simply and conveniently used for calculating the folding phase; (3) according to the original formula for obtaining the folded phase, the projected phase shift diagram has a trigonometric function, so that the light intensity can be a negative value. The projection unit cannot directly project negative light intensity, and the method can be implemented only according to a deformation formula by carrying out deformation processing on the formula for obtaining the folding phase.
The method of the present invention projects different phase shift maps over multiple scans within one exposure time. The image captured by the camera may therefore contain more than two phase shift patterns; the camera has an addition effect on the light intensity, and the accumulation operation in the phase calculation expression is preposed in an image acquisition link, so that the calculation amount of a subsequent algorithm is omitted; in one exposure time, a numerator image and a denominator image in the phase solution expression can be directly acquired. The method is applied to the fringe projection and folding phase resolving links in the three-dimensional reconstruction based on the structured light, and the speed and the precision of the three-dimensional reconstruction are improved.
The invention provides a method for obtaining the expression of the folding phase, which comprises the following steps:
wherein ,
n is the total number of scans in one exposure time, and N represents the number of scans. N represents the total phase shift number, and the larger N is, the larger the noise suppression capability is, and the higher the three-dimensional reconstruction accuracy is.
The above formula is the deformation formula for obtaining the folding phase provided by the invention, and the original formula for obtaining the folding phase is
wherein ,InShowing the acquired nth phase shift diagram,
a represents the background light intensity and b represents the cosine modulation degree. I isnThe light intensity of (a) is not negative, and a is not less than b and not less than 0. The conventional method requires phase-shift of each phase map InRespectively collected, and then substituted into the original formula for obtaining the folding phase to calculate the folding phase. Direct projection of different I within one exposure timenIt is difficult to calculate the folded phase. The different phase shift diagrams projected by the invention thus contain trigonometric functions, i.e.
The projected phase shift diagram contains a trigonometric function term, the projection light intensity may be a negative value, and the projection unit cannot project the negative value light intensity, so the original formula for obtaining the folding phase needs to be deformed, and the method can be implemented only according to the deformation formula.
In order to solve the phase, the traditional method needs to combine N phase shift graphs InAll the samples were collected as (N-0, 1., N-1). Conventional methods acquire one phase shift map at a time for an exposure. Therefore, the conventional method requires N exposures in total in order to obtain N phase shift maps. It is noted that the high frequency projection system is scanned multiple times during one exposure time, as previously described. The multiple scans of the conventional method only project the same phase shift map within one exposure time. Due to the fact that the number of exposure times is large, the three-dimensional reconstruction speed cannot be guaranteed by the traditional method. Meanwhile, if the phase shift number N is low, the precision of three-dimensional reconstruction cannot be guaranteed.
The folded phase solution expression contains a fraction in which the numerator denominator requires the cumulative summation of all the phase shift maps.
As shown in fig. 2, the method of the present invention starts from a phase solution expression, and an image after summation operation can be directly obtained after one exposure, and the projected pattern is different in each scanning. That is, in one exposure time, the projection unit time-divisionally projects an accumulation term (i.e., a phase shift map) according to the number of scanning times, so that the image finally acquired by the camera is the superposition of all the phase shift maps. By the method, a numerator image and a denominator image can be respectively obtained, and finally, a folding phase can be directly obtained according to the two images
At the nth scan, the projected background intensity is expressed as:
to acquire the molecular image, at the nth scan, the phase shift map of the projection is represented as:
to acquire the denominator image, at the nth scan, the phase shift map of the projection is represented as:
wherein a is the background light intensity and b is the light intensity modulation degree.
In one embodiment of the present invention, by controlling the projection unit to make the light intensity modulation degree equal to the background light intensity, i.e. b ═ a, a greater contrast can be obtained and the light intensity of the projection pattern is not negative.
The numerator image, the denominator image, and the background image of one of the fringe frequencies acquired by the camera are represented as:
and respectively acquiring a numerator image and a denominator image of each frequency by projection. Although the background light intensity of each projection is the same, the background light intensity still needs to be projected N times in order to improve the brightness of the background image. The background images corresponding to different fringe frequencies are the same, so that the background images need only be acquired once for different fringe frequencies. The camera integration unit has an addition function on the light intensity, and different phase shift graphs in one exposure time can be accumulated by the camera integration unit. The method can lead the accumulation operation in the phase calculation expression to the image acquisition link, and can save the calculation amount of the subsequent algorithm.
Obtaining the folding phase according to the numerator image, the denominator image and the background image:
therefore, fringe projection and phase solution are completed, and the phase obtained by the method is a folded phase. For three-dimensional reconstruction, the folded phases also need to be unfolded subsequently. Taking the multi-frequency phase unwrapping technique as an example, the phase unwrapping can be performed only by repeatedly utilizing the method to calculate the folded phases with different frequencies. It is worth noting that: all frequencies correspond to the same background image IbTherefore, only one background image I needs to be acquiredb。
As shown in fig. 3, the present invention provides a method for three-dimensional reconstruction, comprising the following steps:
acquiring the folding phase of each fringe frequency by adopting the method for acquiring the folding phase as described in any one of the above;
unfolding the folding phase to obtain the absolute phase of the highest fringe frequency in the fringe frequencies;
calibrating a three-dimensional reconstruction system consisting of the projection unit and the camera, and determining a phase-height conversion parameter;
and converting the absolute phase of the highest frequency into height by using the conversion parameters to obtain the three-dimensional point cloud.
Fig. 4 is a schematic flow chart of three-dimensional reconstruction according to the present invention. The MEMS galvanometer is taken as an example in the invention, and the hardware parameter indexes required by the system of the invention are easily met by the current technology and production process.
The invention provides a three-dimensional reconstruction system, which is used for realizing the three-dimensional reconstruction method, and comprises the following steps:
the high-frequency projection unit is used for projecting light intensity to the object to be measured; the invention is premised on that the projection unit can project images for a plurality of times within one exposure time, namely the projection frequency of the projection unit is greater than the frame rate of the camera, and the larger the projection frequency is, the better the projection frequency is. Therefore, the preferred embodiment of the present invention is a MEMS projection unit, but the method is equally applicable to conventional projection units (DMD, LCD) with projection frequencies higher than the camera frame rate.
The image acquisition unit is used for acquiring the light intensity on the object to be detected;
the scanning frequency of the high-frequency projection unit is greater than the frame rate of the image acquisition unit.
In one embodiment, the high frequency projection unit is a fringe projection unit comprising a laser and a MEMS galvanometer; the laser is used for emitting laser outwards; in a specific embodiment, the laser is a line laser. From the energy conservation analysis, the line laser is formed by the point laser passing through a Powell prism, and the total energy of the whole phase shift diagram is equal to the energy of the point laser. As the working distance increases, the actual physical size of the projected pattern becomes larger and the image becomes darker. If the laser power is low or the working distance of the projection unit is long, the exposure time needs to be increased to improve the image brightness. In this case, the number of projections within one exposure time is also increased, and the use premise of the present invention is more easily satisfied. Therefore, the invention is suitable for low-power lasers and can be used for increasing the working distance of the projection unit; and the MEMS galvanometer is used for continuously vibrating and scanning, reflecting the laser and projecting light intensity to an object to be measured.
As shown in fig. 5, the FPGA control system controls the output power of the laser according to the angle pulse fed back by the MEMS galvanometer. The regulated linear laser is reflected by an MEMS galvanometer, projects a stripe pattern containing phase shift information to a measured object, and forms an image after one-time exposure of a camera.
Assuming that the resonance frequency of the selected MEMS galvanometer is 5000Hz, and the vibration angle range is ± 30, the maximum scanning angle range of the projection unit is ± 2 α ± 60, the scanning speed is 2f 10000Hz, i.e. the maximum pattern projection speed is 10000 fps. The camera maximum frame rate is 300fps, and the image resolution is 1024 × 768. Assuming that the camera can only acquire a bright enough image for every 20 times the projection unit projects the pattern, the camera exposure time is set to 2 ms.
In the conventional method, the projection unit needs to project the same phase shift map 20 times, and then integrate the intensity of the pattern 20 times at the camera end, so as to acquire a phase shift map with enough bright intensity. Different from the conventional method, in the 20-time pattern projection, the pattern projected at the nth time is related to the phase shift diagram at the nth time, and the camera end only needs to acquire 5 images to realize the dual-frequency 20-step phase unfolding method. Whereas the conventional projection scheme needs to acquire 8 images to realize dual-frequency 4-step phase shift. Therefore, the method provided by the invention greatly reduces the required image acquisition time, and simultaneously greatly improves the total phase shift step number, thereby improving the precision of three-dimensional reconstruction. The system can complete the acquisition of all data only by 10ms, and if the time consumption of a subsequent three-dimensional reconstruction algorithm is 10ms, the overall speed of three-dimensional reconstruction by adopting a double-frequency heterodyne 20-step phase shift method is about 50fps, so that the real-time requirement can be met.
At the time of camera acquisition, the method provided by the invention has completed the cumulative summation operation of 20 phase shift graphs through the integration of the light intensity pattern. Thus in the subsequent folding phase
The accumulation summation operation is not required to be executed in the obtaining stage, and the calculation amount of the algorithm is reduced.
Assuming that the speed of projecting 0-255 gray level images by a common projection unit is 120fps, the time for acquiring images required by double-frequency 4-step phase shift is 66 ms; the projection system based on the MEMS galvanometer needs 20 times of scanning and one-time exposure, and the time for acquiring the images required by the double-frequency 4-step phase shift by the traditional projection method is 16 ms; also based on the MEMS galvanometer, the time consumption of acquiring the images required by the double-frequency 20-step phase shift by the time-sharing multiplexing method is 10 ms. Comparing these several methods can find out: (1) the method provided by the invention improves the speed of the whole three-dimensional reconstruction by reducing the time for acquiring the image. (2) And as more phase shift steps are adopted, the precision of three-dimensional reconstruction is greatly improved.
The method of the present invention is used on the premise that the oscillating mirror can scan for many times within a single exposure time. Scanning multiple times superimposes dark patterns into one light pattern. When the working distance of the three-dimensional reconstruction is long or the laser power is low, the projected pattern is dark. Therefore, the method of the invention can be compatible with the use of low-power lasers and can also be used for improving the working distance of three-dimensional reconstruction.
In addition, when the image size is large, the calculation amount of the cumulative summation operation when the folding phase is acquired will not be negligible. The method of the invention is equivalent to complete the accumulative summation operation when the camera is exposed, and reduces the calculation amount of a subsequent software algorithm, thereby further improving the speed of three-dimensional reconstruction. The method provided by the invention has the advantages that the number of the collected images is less, the influence of the phase shift steps is avoided, and the space complexity of the algorithm is reduced.
As shown in fig. 6(a) -6 (d), the graphs are respectively a molecular image, a denominator image, a background image and an acquired phase result, which are acquired by a camera end through a 20-step phase shift method in a simulation experiment of the method of the present invention.
The invention fully utilizes the high-frequency advantage of the MEMS galvanometer, adopts the galvanometer with higher resonance frequency: the MEMS galvanometer with higher resonance frequency is selected to improve the scanning times in a single exposure time under the condition of not changing the exposure time of the camera. Based on the method of the present invention, the optimal solution will obtain more phase shift steps, and therefore the resolved phase will be more accurate. And finally, under the condition of not reducing the three-dimensional reconstruction speed, the optimal scheme can obtain higher precision. The optimal scheme fully utilizes the advantage of high-speed scanning of the MEMS galvanometer, and can easily meet the requirements of three-dimensional reconstruction real-time performance and high precision under the condition of low cost.
In other embodiments of the present invention, other types of projection systems such as DMD, LCD, etc. may also be used: the method according to the invention is still effective if the projection system can still scan or project several times within a single exposure time. Due to the lower frequency of the projections, the number of phase shift maps contained in the numerator image or denominator image will be reduced accordingly. The deformation scheme is still superior to the conventional method in speed and accuracy of three-dimensional reconstruction, but inferior to the optimal scheme. At the moment, parameters such as the type of a projection system, the projection frequency, the laser power, the exposure time and the like are designed in an integrated optimization mode, so that the deformation scheme can meet the requirements of real-time performance and high precision at the same time.
An embodiment of the present application further provides a control apparatus, including a processor and a storage medium for storing a computer program; wherein a processor is adapted to perform at least the method as described above when executing said computer program.
Embodiments of the present application also provide a storage medium for storing a computer program, which when executed performs at least the method described above.
Embodiments of the present application further provide a processor, where the processor executes a computer program to perform at least the method described above.
The storage medium may be implemented by any type of volatile or non-volatile storage device, or combination thereof. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Erasable Programmable Read-Only Memory (EEPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic Random Access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAMEN), Synchronous linked Dynamic Random Access Memory (DRAM), and Direct Random Access Memory (DRMBER). The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments.
Features disclosed in several of the product embodiments provided in the present application may be combined in any combination to yield new product embodiments without conflict.
The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several equivalent substitutions or obvious modifications can be made without departing from the spirit of the invention, and all the properties or uses are considered to be within the scope of the invention.
Claims (10)
1. A method of acquiring a folded phase, comprising the steps of:
s1: the projection unit projects the background light intensity to the object to be measured in a scanning mode, and a camera is used for collecting a background image;
s2: in one exposure time of the camera, the projection unit respectively projects a plurality of different phase shift graphs to the object to be measured;
s3: performing light field superposition on a plurality of different phase shift graphs by using the camera to obtain a light field superposition graph, wherein the light field superposition graph is a numerator image or a denominator image;
s4: sequentially acquiring the numerator images and the denominator images of a plurality of fringe frequencies;
s5: and acquiring the folding phase of each fringe frequency by using a background image, the molecular image and the denominator image corresponding to each fringe frequency.
2. The method of claim 1, wherein the folded phase is obtained by the expression:
wherein ,
n is the total number of scans in one exposure time, and N represents the number of scans.
3. A method of acquiring folding phase as claimed in claim 2, wherein at the nth scan, the projected background intensity is expressed as:
to acquire the molecular image, at the nth scan, the phase shift map of the projection is represented as:
to acquire the denominator image, at the nth scan, the phase shift map of the projection is represented as:
wherein a is the background light intensity and b is the light intensity modulation degree.
4. A method for obtaining a folding phase as claimed in claim 3, wherein the intensity modulation is made equal to the background intensity by controlling the projection unit when projecting the phase shift map.
5. The method for acquiring folding phase according to claim 3 or 4, wherein the numerator image, the denominator image, and the background image of one of the fringe frequencies acquired by the camera are represented as:
6. the method of claim 5, wherein the folding phase is obtained from the numerator image, the denominator image, and the background image:
7. a method of three-dimensional reconstruction, comprising the steps of:
acquiring the folding phase for each of the fringe frequencies using the method of acquiring folding phases of any of claims 1-6;
unfolding the folding phase to obtain the absolute phase of the highest fringe frequency in the fringe frequencies;
calibrating a three-dimensional reconstruction system consisting of the projection unit and the camera, and determining a phase-height conversion parameter;
and converting the absolute phase of the highest frequency into height by using the conversion parameters to obtain the three-dimensional point cloud.
8. A three-dimensional reconstruction system characterized by a method for implementing the three-dimensional reconstruction of claim 7, comprising:
the high-frequency projection unit is used for projecting light intensity to the object to be measured;
the image acquisition unit is used for acquiring the light intensity on the object to be detected;
the scanning frequency of the high-frequency projection unit is greater than the frame rate of the image acquisition unit.
9. The three-dimensional reconstruction system of claim 8 wherein said high frequency projection unit is a fringe projection unit comprising a laser and a MEMS galvanometer;
the laser is used for emitting laser outwards;
and the MEMS galvanometer is used for continuously vibrating and scanning, reflecting the laser and projecting light intensity to an object to be measured.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210085619.1A CN114459380B (en) | 2022-01-25 | 2022-01-25 | Method for acquiring folding phase, three-dimensional reconstruction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210085619.1A CN114459380B (en) | 2022-01-25 | 2022-01-25 | Method for acquiring folding phase, three-dimensional reconstruction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114459380A true CN114459380A (en) | 2022-05-10 |
| CN114459380B CN114459380B (en) | 2023-06-02 |
Family
ID=81412037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210085619.1A Active CN114459380B (en) | 2022-01-25 | 2022-01-25 | Method for acquiring folding phase, three-dimensional reconstruction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114459380B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6084712A (en) * | 1998-11-03 | 2000-07-04 | Dynamic Measurement And Inspection,Llc | Three dimensional imaging using a refractive optic design |
| JP2003042728A (en) * | 2001-07-27 | 2003-02-13 | Fuji Photo Optical Co Ltd | Phase unwrapping method for fringe image analysis |
| JP2008185370A (en) * | 2007-01-26 | 2008-08-14 | Matsushita Electric Works Ltd | Three-dimensional shape measuring device and method |
| CN105387819A (en) * | 2015-12-27 | 2016-03-09 | 河北工业大学 | Reflecting object three-dimensional shape measuring method and device based on fringe reflection method |
| CN106204732A (en) * | 2016-07-21 | 2016-12-07 | 深圳市易尚展示股份有限公司 | The three-dimensional rebuilding method of dynamic exposure and system |
| WO2017008226A1 (en) * | 2015-07-13 | 2017-01-19 | 深圳大学 | Three-dimensional facial reconstruction method and system |
| JP2017075899A (en) * | 2015-10-16 | 2017-04-20 | Ckd株式会社 | Three-dimensional measuring device |
| CN107885070A (en) * | 2017-06-27 | 2018-04-06 | 郑州大学 | A kind of incoherent digital hologram single exposure imaging method and system based on SLM |
| CN109523627A (en) * | 2018-10-24 | 2019-03-26 | 重庆大学 | A kind of three-dimensional rebuilding method based on Taylor's exponential expression moulding surface structure light |
| CN113155058A (en) * | 2021-04-29 | 2021-07-23 | 范金坪 | Multi-wavelength dynamic phase measurement device and method based on spatial domain carrier frequency phase shift |
-
2022
- 2022-01-25 CN CN202210085619.1A patent/CN114459380B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6084712A (en) * | 1998-11-03 | 2000-07-04 | Dynamic Measurement And Inspection,Llc | Three dimensional imaging using a refractive optic design |
| JP2003042728A (en) * | 2001-07-27 | 2003-02-13 | Fuji Photo Optical Co Ltd | Phase unwrapping method for fringe image analysis |
| JP2008185370A (en) * | 2007-01-26 | 2008-08-14 | Matsushita Electric Works Ltd | Three-dimensional shape measuring device and method |
| WO2017008226A1 (en) * | 2015-07-13 | 2017-01-19 | 深圳大学 | Three-dimensional facial reconstruction method and system |
| JP2017075899A (en) * | 2015-10-16 | 2017-04-20 | Ckd株式会社 | Three-dimensional measuring device |
| CN105387819A (en) * | 2015-12-27 | 2016-03-09 | 河北工业大学 | Reflecting object three-dimensional shape measuring method and device based on fringe reflection method |
| CN106204732A (en) * | 2016-07-21 | 2016-12-07 | 深圳市易尚展示股份有限公司 | The three-dimensional rebuilding method of dynamic exposure and system |
| CN107885070A (en) * | 2017-06-27 | 2018-04-06 | 郑州大学 | A kind of incoherent digital hologram single exposure imaging method and system based on SLM |
| CN109523627A (en) * | 2018-10-24 | 2019-03-26 | 重庆大学 | A kind of three-dimensional rebuilding method based on Taylor's exponential expression moulding surface structure light |
| CN113155058A (en) * | 2021-04-29 | 2021-07-23 | 范金坪 | Multi-wavelength dynamic phase measurement device and method based on spatial domain carrier frequency phase shift |
Non-Patent Citations (3)
| Title |
|---|
| LI, XINGHUI: "An improved phase measuring deflectometry method for defect detection of specular reflection surface", 《CONFERENCE ON ADVANCED OPTICAL IMAGING TECHNOLOGIES IV》 * |
| 周波;谢雪冬;陈新河;: "投射光栅的三维形体重建", 黑龙江科技学院学报, no. 05 * |
| 牛小兵: "光栅投射三维轮廓测量技术的研究", 《中国优秀博硕士学位论文全文数据库 (博士)工程科技Ⅰ辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114459380B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8184279B2 (en) | Fourier domain sensing | |
| JP5958779B2 (en) | Microscope and observation method | |
| US9506749B2 (en) | Structured light 3-D measurement module and system for illuminating an area-under-test using a fixed-pattern optic | |
| KR102372449B1 (en) | Method and system for adjusting light pattern for structured light imaging | |
| WO2018105549A1 (en) | Wavelength-swept light source, method for generating drive data for wavelength-swept light source, and optical deflector | |
| US20150292860A1 (en) | Method and apparatus for processing the signal in spectral domain interferometry and method and apparatus for spectral domain optical coherence tomography | |
| CN108982452A (en) | Multifocal spot scan three-D imaging method and system based on double helix point spread function | |
| CN111721236A (en) | Three-dimensional measurement system and method and computer equipment | |
| JPWO2017094380A1 (en) | Information processing apparatus, speckle imaging system, and information processing method | |
| KR20210051683A (en) | Point scan type imaging apparatus for imaging target object within media witch bring about aberration | |
| CN112146565B (en) | Scanner and three-dimensional scanning system | |
| CN115390087A (en) | Laser line scanning three-dimensional imaging system and method | |
| CN114459380A (en) | Method for acquiring folding phase, and method and system for three-dimensional reconstruction | |
| CN114485448A (en) | Self-adaptive speckle interferometry method and system | |
| CN114018554A (en) | Method and system for measuring stability of spectacle frame | |
| CN113899697A (en) | Fringe image-based crystal refractive index measuring method and device and storage medium | |
| JP7224708B2 (en) | Depth data measuring head, measuring device and measuring method | |
| EP3903284A1 (en) | Low-power surface reconstruction | |
| CN108616698A (en) | Imaging device | |
| CN109993784B (en) | Depth image construction method, device and system | |
| CN113411559A (en) | Modulation method, device and system for imaging scanning signal synchronization | |
| CN113959360B (en) | Method, device and medium for measuring three-dimensional surface shape based on phase shift and focal shift | |
| US20230281922A1 (en) | Non-Transitory Storage Medium Storing Thereon Optical Aberration Correction Program and Non-Transitory Storage Medium Storing Thereon Optical Wavefront Estimation | |
| CN114415364B (en) | Time division based multi-focus imaging system | |
| CN112802154A (en) | Test method and test system for obtaining phase modulation curve of spatial light modulator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |