CN116614713B - Self-adaptive multiple exposure method for three-dimensional morphology measurement - Google Patents
Self-adaptive multiple exposure method for three-dimensional morphology measurement Download PDFInfo
- Publication number
- CN116614713B CN116614713B CN202310860684.1A CN202310860684A CN116614713B CN 116614713 B CN116614713 B CN 116614713B CN 202310860684 A CN202310860684 A CN 202310860684A CN 116614713 B CN116614713 B CN 116614713B
- Authority
- CN
- China
- Prior art keywords
- exposure time
- image
- fringe
- exposure
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000005259 measurement Methods 0.000 title claims abstract description 39
- 230000004044 response Effects 0.000 claims abstract description 26
- 230000003287 optical effect Effects 0.000 claims abstract description 10
- 230000001186 cumulative effect Effects 0.000 claims abstract description 8
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000012876 topography Methods 0.000 claims description 14
- 230000003044 adaptive effect Effects 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 10
- 238000000691 measurement method Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 5
- 230000002194 synthesizing effect Effects 0.000 claims description 4
- 230000000873 masking effect Effects 0.000 claims description 3
- 238000007670 refining Methods 0.000 claims description 3
- 238000001308 synthesis method Methods 0.000 claims description 3
- 238000002310 reflectometry Methods 0.000 abstract description 28
- 229910052602 gypsum Inorganic materials 0.000 description 8
- 239000010440 gypsum Substances 0.000 description 8
- 238000007796 conventional method Methods 0.000 description 7
- 230000010363 phase shift Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 5
- 238000003786 synthesis reaction Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 239000002184 metal Substances 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000005316 response function Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000009827 uniform distribution Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 241001270131 Agaricus moelleri Species 0.000 description 1
- ZJPGOXWRFNKIQL-JYJNAYRXSA-N Phe-Pro-Pro Chemical compound C([C@H](N)C(=O)N1[C@@H](CCC1)C(=O)N1[C@@H](CCC1)C(O)=O)C1=CC=CC=C1 ZJPGOXWRFNKIQL-JYJNAYRXSA-N 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010587 phase diagram Methods 0.000 description 1
- 238000001314 profilometry Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration by the use of histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/30—Assessment of water resources
Abstract
The invention discloses a self-adaptive multiple exposure method for three-dimensional morphology measurement, and belongs to the technical field of optical three-dimensional reconstruction. Firstly, calculating reference exposure time according to a histogram cumulative function, and fusing a fringe pattern sequence under the reference exposure time into a reference image; and secondly, estimating a multi-exposure time sequence in an iterative mode according to a camera response curve and the pixel number of the reference image with high signal to noise ratio under different exposure times, and finally fusing the images under the multi-exposure time into a new stripe image sequence so as to carry out three-dimensional reconstruction. The invention can automatically reconstruct the three-dimensional surface object with high dynamic reflectivity by considering the whole and the part respectively, thereby effectively improving the application range of the fringe projection contour operation.
Description
Technical Field
The invention relates to a self-adaptive multiple exposure method for three-dimensional morphology measurement, and belongs to the technical field of optical three-dimensional reconstruction.
Background
With the development of science and technology, the three-dimensional morphology measurement technology is widely applied to different industries in the fields of automobile manufacturing, defect detection, virtual reality, medical cosmetology, cultural relics protection, recording and the like. The stripe projection profile technique (fringe projection profilometry, FPP) is developed rapidly due to the advantages of non-contact, high precision, flexible detection and the like, and the principle is that the grating stripe calculated by a computer is projected on the surface of a measured object through a projector, then a grating image of the surface of the measured object deformed due to the height change is acquired through a camera, an absolute phase diagram is obtained from the captured images by adopting a related algorithm, and finally three-dimensional data are calculated by combining calibration parameters. However, when the technology faces a weak reflection surface or a strong reflection surface, the acquired image can generate local oversaturation or over-darkness, and the calculation error of absolute phase can be increased no matter the information distortion caused by oversaturation or the low signal to noise ratio caused by over-darkness, so that the measurement accuracy of the measured object is finally affected.
In view of this problem, scholars at home and abroad propose a number of different solutions: the literature 'FENG S, ZHANG Y, CHEN Q, et al General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique [ J ]. Optics & Lasers in Engineering, 2014, 59:56-71' uses the trough of gray level histogram to divide pixels with different reflectivities into different clusters, calculates different cluster exposure time respectively, improves the rationality of exposure time selection to a certain extent, but the selection mode still needs to be selected manually, and the method is not very suitable when the histogram of the tested object does not have obvious peaks and troughs to present relatively uniform distribution. The literature Li Zhaojie, cui Haihua, liu Changyi et al discloses a method for measuring morphology based on structured light of an automatic multi-exposure surface, which is abbreviated as [ J ]. An optical report, 2018 "provides a method for automatically predicting exposure time, and the method can adapt to objects to be measured with different reflectivities by calculating exposure time and times required by current scene measurement according to a camera response curve and a reference image, but the prediction process needs to shoot a large number of pure white images, and the whole process is relatively complex and time-consuming. The literature "LI S, DA F, RAO L. Adaptive fringe projection technique for high-dynamic range three-dimensional shape measurement using binary search [ J ]. Optical Engineering, 2017, 56 (9): 1." proposes an adaptive fringe projection technique that acquires orthogonal fringes to obtain a coordinate match of a camera and projector, and then adaptively generates a new fringe pattern using least squares. The matching precision of the method is influenced by the orthogonal stripes of the primary projection, and the selection of the dichotomy threshold is not accurate enough. The literature LIN H. Automatic optimal projected light intensity control for digital fringe projection technique [ J ]. Optics Communications: A Journal Devoted to the Rapid Publication of Short Contributions in the Field of Optics and Interaction of Light with Matter, 2021, 484 (1), "proposes a method for automatically determining the optimal light intensity number and corresponding light intensity value of the fringe pattern projection, and different projected fringe intensity values are obtained by using a clustering method according to the reflectivity of the surface of the measured object. Document "CHUFAN, JIANG, TYLER, et al High dynamic range real-time 3D shape measurement [ J ]. Optics Express, 2016," add projection of an inverted set of fringes, which participate in absolute phase calculations for unsaturated pixel values, has good effect on areas where exposure is not severe, but is not suitable for areas where light is strongly reflected and too dark. Document "He Jiawei, xu Xinke, kongming, et al, research on a method for suppressing metallic surface high light based on surface structured light [ J ]. Chinese test, 2021, (010): 047" is effective in removing intense high light by adding a polarizing filter method, but the angle of the polarizing filter needs to be adjusted according to the optical path, and the intensity of the whole image is reduced to cause the signal-to-noise ratio of a partial region to be too low. Document "Chen Chaowen, xue Junpeng, zhang Qican, et al, high reflectance object surface three-dimensional topography measurement based on multi-view equations [ J ]. Optical journal, 2021, (022): 041." combining projector and binocular into a multi-view system solves the overexposure problem by increasing the viewing angle and corresponding pixel matching process, but it increases the complexity of hardware.
Disclosure of Invention
In order to solve the problems, the invention provides a self-adaptive multiple exposure method for three-dimensional morphology measurement based on multiple exposure technology, which can be suitable for three-dimensional morphology automatic reconstruction of a high dynamic range surface.
The reference exposure time is first calculated by a histogram integration function from the blank map. And screening out the maximum gray value of the same pixel position pixel by pixel from the fringe image sequence acquired by reference exposure to serve as a reference image, taking the low-reflectivity pixels in the reference image as an initial cluster, setting the minimum gray value of the high-signal-to-noise-ratio pixels, continuously increasing the exposure time according to a certain step length, counting the number of pixels reaching the high signal-to-noise ratio of the cluster under different exposure time predictions, wherein the optimal exposure time is the exposure time corresponding to the maximum number of pixels meeting the condition, updating the cluster participating in calculation, and obtaining a plurality of exposure time sequences through iteration. And finally, synthesizing the fringe patterns acquired by different exposure times in a linear interval of camera response to obtain a new fringe image sequence, and measuring the three-dimensional morphology after image synthesis. The method is considered from the whole and the part respectively, can automatically reconstruct the three-dimensional surface object with high dynamic reflectivity, and effectively improves the application range of the fringe projection contour operation.
A first object of the present invention is to provide an adaptive multiple exposure method for three-dimensional topography measurement, the method comprising:
step one: obtaining calibration parameters of a camera, and obtaining the linear response intercept of the camera through one calibrationGray response linear interval->；
Step two: based on the camera linear response intercept obtained in the step oneAnd the maximum desired pixel gray valueCalculating a reference exposure time;
step three: fusing the fringe pattern sequences acquired under the reference exposure time to obtain a reference image, dividing clusters based on the histogram of the reference image, and calculating the optimal exposure time of each cluster to obtain an optimal multiple exposure time sequence;
step four: respectively collecting fringe patterns under the optimal multiple exposure time sequence, and synthesizing the fringe patterns to obtain a new fringe image sequence;
the third step comprises the following steps:
step 31: fusing the fringe images acquired under the reference exposure time;
wherein ,representing the ith fringe pattern at coordinate +.>The gray value at which the color is to be changed,krepresenting the number of fringe patterns for a single exposure acquisition, +.>Representing the reference diagram at coordinates->Gray values at;
step 32: calculating a histogram of the fusion image, and refining the histogram:
wherein ,different gray values for gray histogram +.>Corresponding pixel number, < > and >>To refine the interval;
step 33: computing cumulative functions for refined histograms：
Continuously increase，/>Every increase->Dividing into a cluster;
step 34: calculating a predicted exposure time for each gray value of the cluster histogramAnd taking the gray level prediction as global optimal exposure time to perform gray level prediction on all pixels of the cluster, and counting to reach interval +.>Is +.>, wherein />Different gray values for the clusters;
step 35: the optimal exposure time of the single cluster is the exposure time with the largest number of pixels in the interval:
step 36: integrating the functionIs->Initializing the gray value corresponding to the optimal exposure time in the step 35, and repeating the steps 33-35 until the newly added cluster duty ratio is +.>Less than->。
Optionally, the calculating method of the reference exposure time is as follows:
wherein ,representing the gray value of the pixel at the initial exposure, is->Indicating the initial exposure time, +.>Representing the maximum desired pixel gray value that is set.
Optionally, the fourth step includes:
storing fringe patterns acquired at different exposure times into a sequenceIn order to generate a mask, a reference pattern sequence is synthesized from a fringe pattern of the same exposure time>The mask required for the synthesis method is defined as +.>The calculation formula is as follows:
in the formula ： and />Reference picture sequences->Middle->Image, th->The image being in coordinatesPixel gray value at +.>For the number of exposures, the brightest unsaturated threshold is set to be the maximum value of the linear response of the camera +.>；
Defining the synthesized fringe pattern sequence asThe calculation formula is as follows:
in the formula ：for masking image sequences->Middle->The image is at coordinates->The gray value of the pixel at that point,for the sequence of fringe patterns->First->The image is at coordinates->Pixel gray values at.
Alternatively, ifThe reference exposure time calculation reduces to:
。
optionally, the thinning interval。
Optionally, in the step 33Get->。
The second objective of the present invention is to provide a three-dimensional morphology measurement method, wherein a synthesized fringe pattern is obtained by adopting the adaptive multi-exposure method described in any one of the above, absolute phase information is calculated through the synthesized fringe pattern, and three-dimensional point cloud data is obtained by combining calibration parameters.
A third object of the present invention is to provide a three-dimensional topography measurement system, which uses the three-dimensional topography measurement method to measure, the system includes:
the digital projector is used for carrying out stripe projection on the object to be detected;
an industrial camera for acquiring a fringe image;
and the computer is used for carrying out three-dimensional morphology measurement on the object to be measured by adopting the three-dimensional morphology measurement method based on the acquired fringe image.
Optionally, the optical axes of both the industrial camera and the digital projector are parallel and perpendicular to the working plane.
The invention has the beneficial effects that:
the invention provides a self-adaptive multiple exposure method for three-dimensional morphology measurement, which can automatically predict a plurality of optimal exposure times for objects with wide-range reflectivity variation, and ensure that areas with different reflectivity of the objects can be captured and have good contrast ratio. Compared with manual selection of a plurality of exposures, the method reduces manual intervention and has high automation degree. Experimental results show that the method provided by the invention can adapt to scenes with larger reflectivity differences, can reconstruct the three-dimensional morphology of the object with high dynamic range better, and compared with the existing method, the reconstruction point cloud of the method provided by the invention is more complete and has higher precision.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a block diagram of a fringe generation and acquisition system of the present invention.
FIG. 2 is a flow chart of an adaptive multi-exposure method for three-dimensional topography measurement of the present invention.
Fig. 3 is a graph of a response experiment result of a camera according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating the prediction accuracy verification according to an embodiment of the present invention.
Fig. 5 is a schematic view of an art designing knife and a gypsum image according to an embodiment of the present invention.
FIG. 6 is a reference exposure time prediction diagram of an embodiment of the present invention.
FIG. 7 is an image acquired at different exposure times according to an embodiment of the present invention.
FIG. 8 is a graph comparing the measurement results of the conventional method and the method of the present invention.
FIGS. 9 (a) - (f) are graphs of error analysis of conventional methods and methods of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings.
Embodiment one:
the embodiment provides a self-adaptive multi-exposure method for three-dimensional morphology measurement, which comprises the following steps:
step one: obtaining calibration parameters of a camera, and obtaining the linear response intercept of the camera through one calibrationGray response linear interval->；
Step two: based on the camera linear response intercept obtained in the step oneAnd maximum desired pixel gray value +.>Calculating a reference exposure time;
step three: fusing the fringe pattern sequences acquired under the reference exposure time to obtain a reference image, dividing clusters based on a histogram of the reference image, and calculating the optimal exposure time of each cluster to obtain an optimal multiple exposure time sequence;
step four: and respectively acquiring fringe patterns under the optimal multiple exposure time sequence, and synthesizing the fringe patterns to obtain a new fringe image sequence.
Embodiment two:
the embodiment provides a self-adaptive multi-exposure method for three-dimensional morphology measurement, and a structured light sensing system designed by the embodiment consists of an industrial camera and a digital projector. The optical axes of both the camera and the projector are parallel and perpendicular to the working plane, the horizontal spacing is 70mm, and the working distance is 480mm. To reduce the effect of ambient light, the projector projects a blue sinusoidal stripe. The projector projects a pre-burnt sine fringe pattern, the camera collects the fringe pattern deformed by the high modulation of the object, absolute phase information strictly corresponding to pixel coordinates can be calculated through processing such as phase resolution and phase expansion, and the three-dimensional coordinates of the object are calculated by combining calibration parameters of the system.
Given an arbitrary object with unknown surface reflectivity, multiple optimal exposure times can be predicted. In order to be able to reach an optimal exposure time prediction of an unknown object, further analysis of the camera response function is required. The fringe pattern forming process of the fringe projection system is shown in fig. 1, the encoded fringe is generated by a computer and sent to a designated projector, the projector receives the rear projection pattern to the surface of the measured object, and the camera captures the projector light intensity reflected by the measured object and the surrounding environment light intensity. Typically cameras have a good linear photometric response, and the image captured by the camera can be expressed as:
(1)
in the formula ：is a parameter related to camera gain, aperture size, etc, +.>Is the camera exposure time, +.>For camera noise->Representing the light intensity of the object reflection projector.
Suppose the light of the projectorWith time, the ambient light and the light reflected from each other between objects are approximately constant>Relatively unchanged, ambient light actually entering the camera directly +.>Far less than->Can be ignored, let +.>For the tested objects with different reflectivity, <' > the test object is a light source>Remain relatively constant, so:
(2)
wherein ,、/>、/>respectively representing image columns, rows, exposure time, < >>Is expressed in exposure time +.>Pixel coordinates +.>Intensity of light at->Representing coordinates +.>Location and reflectionParameters related to the rate and the projected light intensity, +.>Representing coordinates +.>Parameters related to noise->Representing the camera linear response intercept.
The finishing formula can be obtained:
(3)
indicating that the camera is capturing the light intensity.
The overall flow of the method of this embodiment is shown in fig. 2. Firstly, obtaining calibration parameters, and obtaining the linear response intercept of the camera through one-time calibrationGray response linear interval->. Secondly, calculating reference exposure time, fusing fringe pattern sequences acquired by the reference exposure time to obtain a reference image, dividing clusters based on a histogram of the reference image, calculating local optimal exposure time, and dynamically obtaining a plurality of optimal exposure times by updating pixel clusters participating in calculation; finally, an exposure time sequence consisting of the reference exposure time and the iteratively calculated local optimum exposure time is output.
1) Obtaining calibration parameters
The actual camera is not strictly linear response, the camera response function needs to be fitted, and the measurement range is set according to the linear interval, and the specific flow is as follows: the projector projects blank images, color bars with different reflectivities are placed in the field of view of the camera, the exposure time of the camera is increased continuously by a certain step length, and an image is acquired under each exposure time until all areas are overexposed and acquired.
Taking the gray average value of each pixel area with different reflectivity as the gray response under the current exposure time, fitting a camera response curve, wherein the different curves represent the camera responses with different reflectivity and the camera responses are shown in figure 3Shows good linearity in gray scale interval, response curves with different reflectivities and +.>The axial intercept is->The intercept of the four different reflectivities can be obtained by fitting a curve to be respectively 0.3, 0.2, 0.1 and 0.08, which are all far smaller than 255, so the intercept of the embodiment is +.>The effect of (2) is negligible.
2) Calculating a reference exposure time
The main consideration of the reference exposure time is to increase the whole gray value of the acquired image as much as possible and prevent oversaturation of most pixel gray values. As can be seen from the formula (2), as the exposure time increases, the overall gray value of the image is continuously increased, and the pixel which reaches saturation first isPixels corresponding to regions having larger values. For a single exposure time +.>The larger area corresponds to a higher gray value for the pixel, so the reference exposure time can be confirmed by calculation of the maximum gray value in the given pixel.
In order to realize reference exposure time confirmation, the embodiment counts the accumulated distribution percentage of gray scales of the acquired image, obtains a given pixel point from the global image by setting a percentage threshold value, and finds the maximum gray scale value from the given pixel point.
First, a cumulative function is calculated:
(4)
wherein For the total number of pixels>For the image gray level histogram +.in 8-phase camera image>。
Then, by setting a percentage threshold valueSetting the maximum desired pixel gray value to +.>Initial exposure time +.>Reference exposure time->Can be determined by the following formula:
(5)
representing the gray value of the pixel at the initial exposure, is->Representing the initial exposure time, manually set, +.>Representing the maximum desired pixel gray value that is set.
The camera exposure time is continuously reduced until most pixels are not overexposed, at which time the exposure time is the initial exposure time, and in order to obtain the reference exposure time, the reference exposure time needs to be calculated based on the image acquired by the initial exposure time, and the reference exposure time is considered global and is taken as the global optimal exposure time.
Further, ifThe reference exposure time calculation can be reduced to:
(6)
3) Iterative calculation of multiple exposure times
And carrying out fringe pattern fusion on the images captured by the reference exposure time on the basis of the reference exposure, and automatically determining a plurality of optimal exposure times based on the fusion pattern. For equation (3), if parameters are acquired in advanceAnd corresponding to each pixelThe intensity value of each pixel at different exposure times can be predicted.
However, in practical application, the optimal exposure of each pixel cannot be acquired separately. Considering that a single exposure time can meet a part of the reflection area to reach a low-error gray scale interval, for a normal measurement scene, the measurement requirement can be met by a plurality of exposure times. In the actual calculation process, the number of exposure times required for objects with larger reflectivity change and uniform distribution is excessive, and the efficiency is reduced.
In the embodiment, from the consideration of reconstruction efficiency and quality, a strategy for dynamically calculating multiple optimal exposures is provided: first, a stripe image captured at a reference exposure time is subjected toAnd merging to obtain a reference image, and performing histogram dimension reduction on the reference image. Considering that the camera acquisition is discrete, the histogram is refined to reduce quantization errors caused by the discrete acquisition, so that the exposure time prediction is more accurate. Calculating cumulative function of refined histogram, dividing pixel number with gray value as characteristic into one cluster, calculating gray value predicted by different exposure time, setting gray lower limit=150, statistics reach gray lower limit +.>And maximum gray value +.>The number of pixels in between, wherein the exposure time with the largest number of pixels satisfying the interval is the optimal exposure time, and finally the multiple optimal exposure time sequence is iterated. The specific steps are summarized as follows:
step1: fusing the fringe images acquired by reference exposure;
(7)
wherein ,represent the firstiThe striped pattern is at coordinates->The gray value at which the color is to be changed,krepresenting the number of fringe patterns for a single exposure acquisition, +.>Representing the reference diagram at coordinates->Gray values at that point.
step2: computing a histogram of the fused image, since the camera samples are discreteIn order to alleviate the problem of inaccurate optimal exposure time prediction caused by discrete sampling, the histogram is subjected to refinement processing, whereinDifferent gray values for gray histogram +.>Corresponding pixel number, < > and >>To refine the interval.
(8)
step3: according to formula (4), calculating a cumulative function of the refined histogramContinuously increase +.>，/>Per incrementDivided into a cluster->Man-made definition->The smaller the exposure times, the more.
step4: calculating a predicted exposure time for each gray value of the cluster histogramAnd taking the gray level prediction as global optimal exposure time to perform gray level prediction on all pixels of the cluster, and counting to reach interval +.>Is +.>, wherein Different gray values for the clusters;
step5: the optimal exposure time of the single cluster is the exposure time with the largest number of pixels in the interval:
(9)
step6: of cumulative functionsInitializing the gray value corresponding to the optimal exposure time in step5, repeating step3-5 until the newly added cluster is occupied by +.>Less than->。
4) Multiple exposure image synthesis
For high dynamic range objects, the low reflection area has higher fringe contrast at high exposure times, while the high reflectivity area is acquired with low exposure, and the image can retain more detail. The multi-exposure stripe synthesis is adopted, the advantage of pixel-by-pixel three-dimensional reconstruction of a phase shift algorithm is fully utilized, and a stripe synthesis diagram with richer details and higher contrast is obtained.
The core idea of stripe synthesis is to select the brightest unsaturated pixel at each pixel position at different exposure times. Storing fringe patterns acquired at different exposure times into a sequenceIn order to generate a mask, it is necessary to fuse the fringe patterns of the same exposure time into a reference pattern sequence +.>The mask required for the synthesis method is defined as +.>The calculation formula is as follows:
(10)
in the formula ： and />Reference picture sequences->Middle->Image, th->The image being in coordinatesPixel gray value at +.>For the number of exposures, the brightest unsaturated threshold is set to be the maximum value of the linear response of the camera +.>。
Defining the synthesized fringe pattern sequence asThe calculation formula is that
(11)
in the formula ：for masking image sequences->Middle->The image is at coordinates->The gray value of the pixel at that point,for the sequence of fringe patterns->First->The image is at coordinates->Pixel gray values at.
Finally, absolute phase information can be calculated through the synthesized fringe pattern, three-dimensional point cloud data can be obtained by combining calibration parameters, so that a three-dimensional morphology measurement result can be obtained, and the reconstruction calculation process is as follows:
analysis with a lateral fringe pattern, for an N-step phase shift, any pixel on the camera acquired image satisfies the following expression
(12)
in the formula ：is->Pixel point in image>Gray values of (2); />Is pixel dot +.>A background gray value; />Is pixel dot +.>Gray scale modulation of (a); />Is pixel dot +.>The principal value of the phase that needs to be solved. The phase principal value of the N-step phase shift can be found using a least squares solution:
(13)
since equation (13) calculates the phase using the arctangent function, the calculated phase valueIs in the value rangeIn the interval, called wrapping phase, the wrapping phase loses the cycle information, so that the wrapping phase is unfolded to obtain the cycle number of the actual phase, in this embodiment, the main value of each pixel phase and the unique absolute phase of the whole field after being unfolded are calculated through a multi-frequency heterodyne method, and the complete phase value can be obtained through phase unfolding:
(14)
wherein ：representing fringe phase shift solutionsPhase principal value +_>For the actual number of cycles of the current pixel.
In order to further illustrate the beneficial effects of the present invention, experiments were conducted, and the experimental procedures and results were as follows:
the correctness of the proposed method is verified by using a built three-dimensional morphology measurement system which consists of a large constant gray-scale camera MER-1070-14U3M (with the resolution of 3840 multiplied by 2748), a computor lens with the focal length of 35mm, a digital projector (with the resolution of 1280 multiplied by 720) and a computer with the processor of i7-10700, and the display card of Injettia GeForce RTX 3060 and 32 GB memory. The four-step phase shift and multi-frequency heterodyne method is adopted for measurement, and the self-adaptive prediction multiple exposure method is required to project 1 blank image and 12 fringe images in total.
In order to ensure the measurement accuracy, the experimental measurement method provided by the invention is used for comparing the histogram prediction with the actual gray level histogram. As shown in fig. 4, images acquired with exposure time of 5000us and 10000us are acquired respectively and subjected to histogram analysis, the images acquired with exposure time of 10000us are subjected to histogram prediction by using the method of the invention with 5000us as a reference image, and the result shows that the gray level prediction of the method of the invention is relatively close to the actual gray level prediction, and the prediction and the actual deviation are acceptable in consideration of factors such as sensor noise, discrete sampling errors and the like.
In order to verify the effectiveness of the method, a gypsum image with larger reflectivity difference and an art designing knife are selected as measuring objects, a physical diagram is shown in fig. 5, the gypsum image has higher reflectivity, in order to ensure that the acquired image is not overexposed, the exposure time of a camera is reduced, the pixels in a high-reflectivity area can be well captured by the camera and cannot be oversaturated, but the light intensity of the pixels in a low-reflectivity area is too low to cause the signal-to-noise ratio to be too low, and the phase unwrapping is wrong. If the exposure time is gradually increased, the brightness of the low-reflection area can be improved, the signal to noise ratio of the low-reflection area is improved, but the high-reflection area is too bright due to high exposure, the acquired image light intensity exceeds the intensity range captured by the camera, and the problem of camera supersaturation is caused. Obviously, a single exposure time cannot simultaneously meet the reconstruction requirements of workpieces with different reflectivities. Therefore, the invention automatically predicts a plurality of different exposure times to adapt to different reflectivities of the surface of the measured object.
The first step is to obtain a reference exposure time: reducing the exposure time of the camera to 10So that the acquired blank pattern does not have overexposure phenomenon, and the cumulative function of the blank pattern is calculated, as shown in figure 6, and the percentage threshold is set to enable +.>Calculated to obtainThe maximum expected pixel gray value is set to 240, and the reference exposure time is calculated to be 19.83 according to the formula (5)。
The second step automatically determines a plurality of optimal exposure times: fusing the fringe patterns acquired by reference exposure to obtain a reference image, performing histogram dimension reduction on the reference image, and performing histogram dimension reduction on the reference image by usingRefining the histogram for interval to +.>The pixel number is a cluster, and a gray lower limit is set>For 150, iteratively calculating an optimal exposure time sequence, respectively obtaining the following optimal exposure times: 13.60, 3.62, 2.75, 2.38, 1.98 ∈>。
The image acquisition is performed according to several optimal exposure times, fig. 7 (a) -7 (e) show fringe patterns acquired by different exposure times, fig. 7 (f) shows a fusion of fringe patterns acquired by different exposure times, and as can be seen from fig. 7, different reflectivity areas are acquired by different exposure times, the brightness of the low-reflectivity area is improved, the high-reflectivity area is not overexposed, and the overall brightness and contrast are improved.
Fig. 8 shows a comparison of the measurement results of the conventional method and the method of the present invention, wherein fig. 8 (a) is a point cloud image of three-dimensional reconstruction under the premise that the gypsum image stripes are not overexposed, the quality of gypsum image reconstruction is better, but the problem that the point cloud is inconsistent with the actual point cloud due to the fact that the single exposure time of the art knife is too low, as shown in fig. 8 (b), the floating error point cloud can be found above the knife handle. Fig. 8 (c) is a diagram showing that the exposure time is increased and the problem of phase resolution errors in the low reflection area of the utility knife is reduced based on fig. 8 (a), but the enlarged gypsum image detail diagram of fig. 8 (d) shows that the facial point cloud presents water wave shape errors, so that the conventional method cannot reconstruct the object with high dynamic range and high quality. Fig. 8 (e) shows an oblique view of the 3D reconstruction effect of the method of the present invention, and in order to better display the quality of data, fig. 8 (f) and fig. 8 (h) show close-up views of the utility knife and the gypsum image respectively, and it can be seen from the figures that the utility knife is complete in reconstruction, no error point cloud occurs, meanwhile, the reconstruction details are clear, the gypsum image 3D reconstruction has no water wave phenomenon, and the overall reconstruction quality is better. The object with high dynamic range can be reconstructed with high quality without manual intervention. Experiments show that the method can automatically predict multiple optimal exposures, and the 3D reconstruction result is satisfactory.
In order to further verify the effectiveness of the three-dimensional morphology measurement method, the method is applied to blades and metal flat plates with larger curvature change, as shown in fig. 9 (a), the curvature change of the blades is large, the light intensity collected in the middle of a curved surface is larger than that of the edge of the curved surface, absolute phase solution is carried out by adopting a traditional method and the method, absolute phase calculated by 20 steps of phase shift is taken as an ideal phase to calculate phase errors, as can be seen from fig. 9 (b), the edge part of the curved surface is a low-reflectivity area, the phase errors calculated by the traditional method are larger, the phase errors calculated by the method are larger, and the phase errors of other areas except the boundary and shadow area errors are smaller. Fig. 9 (d) is an image of a measured metal flat plate, three-dimensional reconstruction is performed by adopting a conventional method and the method of the invention, the generated point cloud is led into geomic, fig. 9 (e) and fig. 9 (f) are deviations between the metal flat plate to be measured and a geomic fitting plane, wherein fig. 9 (e) represents a reconstruction deviation representation of the conventional method, fig. 9 (f) represents a reconstruction deviation representation of the method of the invention, it can be seen that a plurality of dark spots are distributed at a high reflection position on the right side of the metal flat plate in fig. 9 (e), the darker color represents the larger deviation, and the dark spots do not appear in fig. 9 (f), so compared with the conventional method, the method provided by the invention can effectively reduce the reconstruction deviation, and can better finish three-dimensional morphology measurement of a high reflection workpiece.
Some steps in the embodiments of the present invention may be implemented by using software, and the corresponding software program may be stored in a readable storage medium, such as an optical disc or a hard disk.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (9)
1. An adaptive multi-exposure method for three-dimensional topography measurement, the method comprising:
step one: obtaining calibration parameters of a camera, and obtaining the linear response intercept of the camera through one calibrationGray response linear interval->；
Step two: based on the camera linear response intercept obtained in the step oneAnd maximum desired pixel gray value +.>Calculating a reference exposure time;
step three: fusing the fringe pattern sequences acquired under the reference exposure time to obtain a reference image, dividing clusters based on the histogram of the reference image, and calculating the optimal exposure time of each cluster to obtain an optimal multiple exposure time sequence;
step four: respectively collecting fringe patterns under the optimal multiple exposure time sequence, and synthesizing the fringe patterns to obtain a new fringe image sequence;
the third step comprises the following steps:
step 31: fusing the fringe images acquired under the reference exposure time;
wherein ,representing the ith fringe pattern at coordinate +.>The gray value at which the color is to be changed,krepresenting the number of fringe patterns for a single exposure acquisition, +.>Representing the reference diagram at coordinates->Gray values at;
step 32: calculating a histogram of the fusion image, and refining the histogram:
wherein ,different gray values for gray histogram +.>Corresponding pixel number, < > and >>To refine the interval;
step 33: computing cumulative functions for refined histograms：
Continuously increase，/>Every increase->Divided into a cluster->Man-made definition->The smaller the exposure times, the more;
step 34: calculating a predicted exposure time for each gray value of the cluster histogramAnd taking the gray level prediction as global optimal exposure time to perform gray level prediction on all pixels of the cluster, and counting to reach interval +.>Is +.>, wherein />Different gray values for the clusters;
step 35: the optimal exposure time of the single cluster is the exposure time with the largest number of pixels in the interval:
step 36: integrating the functionIs->Initializing the gray value corresponding to the optimal exposure time in the step 35, and repeating the steps 33-35 until the newly added cluster duty ratio is +.>% is less than->。
2. The adaptive multi-exposure method for three-dimensional topography measurement according to claim 1, wherein the reference exposure time calculation method is as follows:
wherein ,representing the gray value of the pixel at the initial exposure, is->Indicating the initial exposure time, +.>Representing the maximum desired pixel gray value that is set.
3. The adaptive multi-exposure method for three-dimensional topography measurement according to claim 2, wherein the fourth step comprises:
storing fringe patterns acquired at different exposure times into a sequenceIn order to generate a mask, a reference pattern sequence is synthesized from a fringe pattern of the same exposure time>The mask required for the synthesis method is defined as +.>The calculation formula is as follows:
in the formula ： and />Reference picture sequences->Middle->Image, th->The image is at coordinates->Pixel gray value at +.>For the number of exposures, the brightest unsaturated threshold is set to be the maximum value of the linear response of the camera +.>；
Defining the synthesized fringe pattern sequence asThe calculation formula is as follows:
in the formula ：for masking image sequences->Middle->The image is at coordinates->The gray value of the pixel at that point,for the sequence of fringe patterns->First->The image is at coordinates->Pixel gray values at.
4. The adaptive multiple exposure method for three-dimensional topography measurement according to claim 2, wherein ifThe reference exposure time calculation reduces to:
。
5. the adaptive multiple exposure method for three-dimensional topography measurement according to claim 1, wherein the refinement interval。
6. The adaptive multi-exposure method for three-dimensional topography measurement according to claim 1, wherein in step 33Get->。
7. The three-dimensional morphology measurement method is characterized in that the three-dimensional morphology measurement method firstly adopts the self-adaptive multiple exposure method for three-dimensional morphology measurement according to any one of claims 1-6 to obtain a synthesized fringe pattern, then calculates absolute phase information through the synthesized fringe pattern, and combines calibration parameters to obtain three-dimensional point cloud data.
8. A three-dimensional topography measurement system, wherein the system measures using the three-dimensional topography measurement method of claim 7, the system comprising:
the digital projector is used for carrying out stripe projection on the object to be detected;
an industrial camera for acquiring a fringe image;
a computer for performing three-dimensional morphology measurement on the object to be measured using the three-dimensional morphology measurement method of claim 7 based on the acquired fringe image.
9. The three-dimensional topography measurement system of claim 8, wherein the optical axes of both the industrial camera and the digital projector are parallel and perpendicular to the working plane.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310860684.1A CN116614713B (en) | 2023-07-14 | 2023-07-14 | Self-adaptive multiple exposure method for three-dimensional morphology measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310860684.1A CN116614713B (en) | 2023-07-14 | 2023-07-14 | Self-adaptive multiple exposure method for three-dimensional morphology measurement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116614713A CN116614713A (en) | 2023-08-18 |
| CN116614713B true CN116614713B (en) | 2023-10-27 |
Family
ID=87678484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310860684.1A Active CN116614713B (en) | 2023-07-14 | 2023-07-14 | Self-adaptive multiple exposure method for three-dimensional morphology measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116614713B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104019767A (en) * | 2014-05-29 | 2014-09-03 | 南京理工大学 | Self-adaptive high-dynamic-range three-dimensional measuring method based on gray scale histogram |
| CN108827184A (en) * | 2018-04-28 | 2018-11-16 | 南京航空航天大学 | A kind of structure light self-adaptation three-dimensional measurement method based on camera response curve |
| CN115170832A (en) * | 2022-07-25 | 2022-10-11 | 江南大学 | Weak texture surface microstructure feature extraction method based on visible light single image |
| CN115937404A (en) * | 2022-09-27 | 2023-04-07 | 江苏聚目科技有限公司 | Grating three-dimensional reconstruction system and method based on multi-view shadow segmentation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107044833B (en) * | 2017-03-24 | 2019-03-05 | 南京理工大学 | A kind of ultrafast 3 D measuring method and its system based on improved Fourier transform outline technology |
-
2023
- 2023-07-14 CN CN202310860684.1A patent/CN116614713B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104019767A (en) * | 2014-05-29 | 2014-09-03 | 南京理工大学 | Self-adaptive high-dynamic-range three-dimensional measuring method based on gray scale histogram |
| CN108827184A (en) * | 2018-04-28 | 2018-11-16 | 南京航空航天大学 | A kind of structure light self-adaptation three-dimensional measurement method based on camera response curve |
| CN115170832A (en) * | 2022-07-25 | 2022-10-11 | 江南大学 | Weak texture surface microstructure feature extraction method based on visible light single image |
| CN115937404A (en) * | 2022-09-27 | 2023-04-07 | 江苏聚目科技有限公司 | Grating three-dimensional reconstruction system and method based on multi-view shadow segmentation |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于自动多次曝光面结构光的形貌测量方法;李兆杰;崔海华;刘长毅;程筱胜;张小迪;;光学学报(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116614713A (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107607040B (en) | Three-dimensional scanning measurement device and method suitable for strong reflection surface | |
| Song et al. | A high dynamic range structured light means for the 3D measurement of specular surface | |
| CN112106105B (en) | Method and system for generating three-dimensional image of object | |
| CN109477710B (en) | Reflectance map estimation for point-based structured light systems | |
| CN108827184B (en) | Structured light self-adaptive three-dimensional measurement method based on camera response curve | |
| CN105651203A (en) | High-dynamic-range three-dimensional shape measurement method for self-adaptation fringe brightness | |
| CN107592449B (en) | Three-dimensional model establishing method and device and mobile terminal | |
| CN110425998A (en) | The components three-dimensional measurement method of gray level image coupling feature point height | |
| CN107483815B (en) | Method and device for shooting moving object | |
| CN111563952A (en) | Method and system for realizing stereo matching based on phase information and spatial texture characteristics | |
| CN114972633A (en) | Rapid scanning point cloud interpolation method under cross laser line constraint | |
| Khan et al. | High-density single shot 3D sensing using adaptable speckle projection system with varying preprocessing | |
| CN116608794B (en) | Anti-texture 3D structured light imaging method, system, device and storage medium | |
| CN116614713B (en) | Self-adaptive multiple exposure method for three-dimensional morphology measurement | |
| CN115290004B (en) | Underwater parallel single-pixel imaging method based on compressed sensing and HSI | |
| Fu et al. | High Dynamic Range Structured Light 3-D Measurement Based on Region Adaptive Fringe Brightness | |
| CN114387387A (en) | High-reflectivity surface color three-dimensional reconstruction method, system, terminal and medium | |
| CN107610079B (en) | Image processing method and apparatus, electronic apparatus, and computer-readable storage medium | |
| KR101904170B1 (en) | Coding Device and Method for Depth Information Compensation by Sphere Surface Modeling | |
| CN114731373A (en) | Method for determining one or more sets of exposure settings for a three-dimensional image acquisition process | |
| Gao et al. | An investigation of high dynamic range imaging of highly reflective surfaces based on deep learning 3D fringe projection | |
| CN116824069B (en) | Self-adaptive stripe method for detecting saturation point by using high-frequency signal | |
| Wang et al. | Optimal exposure selection of high dynamic range-based reflection compensation for printed circuit board reconstruction using structured light 3D measurement system | |
| CN114596351A (en) | Data set generation method for phase estimation network training | |
| d'Angelo | 3D scene reconstruction by integration of photometric and geometric methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |