CN116642895B - Five-face imaging 3D reconstruction method for focal plane sweep - Google Patents
Five-face imaging 3D reconstruction method for focal plane sweep Download PDFInfo
- Publication number
- CN116642895B CN116642895B CN202310931474.7A CN202310931474A CN116642895B CN 116642895 B CN116642895 B CN 116642895B CN 202310931474 A CN202310931474 A CN 202310931474A CN 116642895 B CN116642895 B CN 116642895B
- Authority
- CN
- China
- Prior art keywords
- focal plane
- imaging
- pixels
- target
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 104
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000006073 displacement reaction Methods 0.000 claims abstract description 34
- 238000010408 sweeping Methods 0.000 claims description 9
- HXGPBZKGPPZSGX-UHFFFAOYSA-N 2,7-bis[fluoro(dipyridin-2-yl)methyl]-1,8-naphthyridine Chemical compound C=1C=CC=NC=1C(C=1N=C2N=C(C=CC2=CC=1)C(F)(C=1N=CC=CC=1)C=1N=CC=CC=1)(F)C1=CC=CC=N1 HXGPBZKGPPZSGX-UHFFFAOYSA-N 0.000 claims description 5
- 101150056585 DACT2 gene Proteins 0.000 claims description 5
- 102100025691 Dapper homolog 1 Human genes 0.000 claims description 5
- 101100031414 Dictyostelium discoideum psmE3 gene Proteins 0.000 claims description 5
- 101000708874 Homo sapiens Zinc finger protein ubi-d4 Proteins 0.000 claims description 5
- 101100120432 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) RAM1 gene Proteins 0.000 claims description 5
- 102100029859 Zinc finger protein neuro-d4 Human genes 0.000 claims description 5
- 102100032701 Zinc finger protein ubi-d4 Human genes 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 5
- 101150113198 dact1 gene Proteins 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 17
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000000007 visual effect Effects 0.000 abstract description 3
- 230000007547 defect Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 239000000463 material Substances 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000004377 microelectronic Methods 0.000 description 1
- 238000000386 microscopy Methods 0.000 description 1
- 230000010287 polarization Effects 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Chemical & Material Sciences (AREA)
- Biochemistry (AREA)
- Pathology (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Immunology (AREA)
- Analytical Chemistry (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Microscoopes, Condenser (AREA)
Abstract
The invention belongs to the technical field of visual imaging detection, and particularly relates to a focal plane scanning five-face imaging 3D reconstruction method. The method utilizes the characteristic of shallow depth of field under microscopic imaging conditions, and makes the 5-plane imaging microscopic camera move from far to near once in the imaging process, so that the focal plane finishes scanning imaging on 5 planes (1 front face and 4 side faces of the imaging object), thereby generating image sequences with different displacement depths, and then using a 3D reconstruction image processing flow to extract a well focused imaging part in the image sequences, and combining displacement depth information to finish 3D splicing and reconstruction on the 5 planes of the imaging object. The invention utilizes the combination of microscopic imaging focal plane scanning and 5-plane imaging technology, and based on the depth information of displacement depth, can conveniently complete the 3D reconstruction of an imaging object in one scanning imaging, and provides an effective means for the 3D imaging and detection of a tiny object.
Description
Technical Field
The invention belongs to the technical field of visual imaging detection, and particularly relates to a focal plane scanning five-face imaging 3D reconstruction method.
Background
In the production process of tiny components, defect detection needs to be carried out on the surfaces of the components so as to ensure that the components meet the use standard. In order to realize defect detection of components, image acquisition is required to be carried out on a plurality of surfaces of the components, and the problems of large use quantity of cameras, large occupied space of equipment and interference of ambient light exist in the existing five-face imaging device.
The existing penta-surface imaging technology mainly focuses on obtaining penta-surface imaging through adjustment of an angle position structure of a reflecting prism, but in obtaining penta-surface imaging, high-quality sample pictures need to be extracted due to interference of environmental factors such as natural light, and the aplanatic penta-surface visual detection technology (CN 200910056450) is provided with a structure consisting of a polarization beam splitter prism, so that an image is improved, but the structure is complex, and the cost is high.
Disclosure of Invention
Aiming at the problems, the invention provides a five-face imaging 3D reconstruction method for focal plane scanning, which utilizes reflection mirror refraction to enable a camera to acquire imaging of four sides of a component at one time, obtains imaging of the component with different depths through focal plane scanning, extracts pixels with good focus through a focus judgment algorithm, splices the good pixels through a splicing algorithm, and finally realizes 3D reconstruction of the component through a splicing and fusion algorithm, only uses one camera in a mirror reflection imaging mode, reduces equipment occupation control, ensures imaging definition through focal plane scanning, and effectively improves detection efficiency and detection precision. The method overcomes the defect that the high-quality sample image is obtained under the interference of ambient light, solves the problem of large detection volume of the existing industrial defects, realizes engineering, miniaturization and light weight, and can be installed anywhere and anytime.
The technical scheme of the invention is as follows:
a focal plane swept five-sided imaging 3D reconstruction method comprising the steps of:
s1, carrying out front focal plane scanning on a target to be detected, wherein the method specifically comprises the following steps: after fixing a target to be measured, setting imaging equipment to be close to the target to be measured from far to near along the vertical direction, and finishing focal plane scanning of the front face of the target to be measured to obtain a plurality of front face focal plane scanning pictures;
s2, carrying out side focal plane sweep on a target to be detected, wherein the side focal plane sweep specifically comprises the following steps: setting a reflecting mirror surface, forming a mirror image of a target to be detected in the reflecting mirror surface, enabling the imaging equipment to be far from the target to be detected and near to the target to be detected along the vertical direction, and finishing focal plane scanning of the mirror image of the target to be detected to obtain a plurality of side face focal plane scanning pictures;
and S3, performing 3D reconstruction by using the obtained multiple front face focal plane sweep pictures and the obtained multiple side face focal plane sweep pictures to obtain a 3D image of the target to be detected.
Further, in S1, when the front focal plane is swept by the target to be measured, as shown in fig. 2, a position where the focal plane is tangent to the target to be measured is taken as a starting position of the front focal plane sweep, and the starting position is taken as a reference, so as to obtain a displacement depth of each front focal plane sweep picture.
Further, in S2, when the side focal plane of the target to be measured is swept, as shown in fig. 3, a position where the focal plane is tangential to the mirror image is taken as a starting position of the side focal plane sweep, and the starting position is taken as a reference, so as to obtain a displacement depth of each side focal plane sweep picture.
Further, as shown in fig. 5, the specific method of S3 is:
the front 3D reconstruction is carried out on the target to be detected by utilizing the obtained multiple front focal plane sweep pictures, and the specific method comprises the following steps: the front focal plane sweep pictures obtained by definition are PF1, PF2, … … and PFN respectively, the sharpness values of each picture at each point are obtained by calculating the longitudinal gradient and the transverse gradient of N pictures through a focusing judgment algorithm, the pixels with good focus are judged through the sharpness values, and the x, y and z position information of the pixels with good focus is obtained, so that the pixels DPF1, DPF2, … … and DPFN with good focus at the displacement depth corresponding to each picture are obtained, and the focusing algorithm formula is as shown in the following formula 1:
(equation 1)
In equation 1:representing an imageCorresponding pixel pointIs used for the gray-scale value of (c),the result is calculated for the image sharpness.
And then, carrying out point cloud calculation and registration on the well-focused pixels through smoothing processing by a top view stitching algorithm, fusing the registered point cloud data to obtain a reconstruction model, and finally generating the surface of the object to be detected to obtain a top view 3D map, wherein the top view stitching algorithm is as follows in formula 2 and formula 3:
(equation 2)
In equation 2:for the size of the sliding window it is,for image size, pairThe number of rows of the device is,and summing the well focused pixels of the columns to obtain the well focused pixels at the displacement depth corresponding to each picture.
(equation 3)
In equation 3:wherein pixels that are well focused during focal plane sweeping,point cloud data for well focused pixels, whereFor the coordinates of well focused pixels in the reference coordinate system,time information for a well focused pixel when sweeping the focal plane.By findingAnd fusing the point cloud data to generate a top view 3D map.
The obtained multiple side focal plane sweep pictures are utilized to carry out side 3D reconstruction on the object to be detected, and the specific method comprises the following steps: the side focal plane scanned pictures are PR1, PR2, … … and PRM respectively, the sharpness values of each picture at each point are obtained by calculating the longitudinal gradient and the transverse gradient of M pictures through a focusing judgment algorithm, the pixels with good focus are judged through the sharpness values, and the x, y and z position information of the pixels with good focus is obtained, so that good focus pixels DPR1, DPR2, … … and DPRM at the displacement depth corresponding to each picture are obtained, and the focusing algorithm is as shown in the formula 1.
And then, carrying out smoothing treatment on the well-focused pixels through a side view stitching algorithm, carrying out point cloud calculation and registration on the well-focused pixels, fusing the registered point cloud data to obtain a reconstruction model, finally generating a surface to obtain a side view 3D image, and obtaining the side view 3D image by adopting formulas such as the formula 2 and the formula 3.
And obtaining the 3D image by a stitching and fusion algorithm based on the displacement depth information by using the top view 3D image and the side view 3D image.
The beneficial effects of the invention are as follows: (1) The invention uses the reflecting mirror surface to form a five-sided imaging reflecting mirror structure, and uses the principle of specular reflection to enable the camera to obtain imaging of four sides of the component at one time, thereby reducing the production cost and realizing engineering, miniaturization and light weight. (2) According to the five-face imaging 3D reconstruction technology for focal plane scanning, component imaging with different depths is obtained by using a focal plane scanning method, pixels with good focusing are extracted, and 3D reconstruction is completed through splicing and fusion.
Drawings
FIG. 1 is a block diagram of a focal plane swept five-sided imaging 3D reconstruction technique;
FIG. 2 is a schematic diagram of a front focal plane swept 3D imaging process;
FIG. 3 is a schematic diagram of a side focal plane swept 3D imaging process;
FIG. 4 is a five-sided imaging microscopy camera view;
FIG. 5 is a block diagram of a five-sided imaging mirror;
FIG. 6 is a block diagram of a 3D reconstructed image processing flow;
fig. 7 is a schematic structural diagram of the present example.
Detailed Description
The technical scheme of the invention is described in detail below with reference to the accompanying drawings and the embodiments.
As shown in fig. 1, the embodiment of the present invention can be described as the following steps:
(1) The microscopic camera moves along the main shaft at a constant speed;
(2) Performing front focal plane swept 3D imaging;
(3) Performing side focal plane swept 3D imaging;
(4) Ending the scanning;
(5) 3D reconstruction.
The specific implementation mode of the step (1) is as follows:
controlling the microscopic camera to move at a constant speed along the main shaft, so that the focal plane of the microscopic camera sequentially passes through the imaging object and the imaging object mirror surface image obtained by the five-surface imaging mirror structure; the five-sided imaging reflecting mirror structure is formed by splicing a reflecting mirror surface 1, a reflecting mirror surface 2, a reflecting mirror surface 3 and a reflecting mirror surface 4; the included angle between the reflecting mirror and the main shaft is 45 degrees.
As shown in fig. 2, the specific implementation manner of the step (2) is as follows:
when the focal plane passes through the imaging object, front focal plane scanning 3D imaging is started on the imaging object, and a front focal plane scanning picture N is obtained, wherein the front focal plane scanning picture PF1, the front focal plane scanning pictures PF2 and … … and the front focal plane scanning picture PFN are included.
As shown in fig. 3, the specific implementation manner of the step (3) is as follows:
(3.1) forming a mirror image of the imaging object through the mirror surface of the reflecting mirror, wherein a point A of the imaging object corresponds to a point A ' of the mirror image of the imaging object, a point B of the imaging object corresponds to a point B ' of the mirror image of the imaging object, and a point C of the imaging object corresponds to a point C ' of the mirror image of the imaging object;
(3.2) acquiring a side focal plane sweep graph when the focal plane of the microscope camera passes through a tangent point of a mirror image of an imaging object imaged by reflection of the reflecting mirror surface;
(3.3) continuing to move upwards to obtain a side focal plane sweep image, wherein the focal plane moves upwards to pass through points A ', B ', C ' of the mirror image of the imaging object to obtain a side focal plane sweep image M, including side focal plane sweep image PR1, side focal plane sweep images PR2 and … … and side focal plane sweep image PRM, so as to complete the focal plane sweep of the side face of the whole element.
The specific implementation mode of the step (5) is as follows:
(5.1) carrying out a focusing judgment algorithm on a front focal plane scanned picture PF1, front focal plane scanned pictures PF2 and … … and a front focal plane scanned picture PFN obtained by scanning and photographing a microscopic camera, wherein the focusing judgment algorithm respectively calculates the longitudinal gradient and the transverse gradient of the front focal plane scanned picture PF1, the front focal plane scanned picture PF2 and … … and the front focal plane scanned picture PFN to obtain sharpness values of each picture at each point, judges pixels with good focusing through the sharpness values, obtains x, y and z position information of the pixels with good focusing, and outputs good focusing pixels at a displacement depth DPF1, good focusing pixels at a displacement depth DPF2, … … and good focusing pixels at a displacement depth DPFN;
(5.2) carrying out a focusing judgment algorithm on a side face focus plane scanned picture PR1, side face focus plane scanned pictures PR2 and … … and a side face focus plane scanned picture PRM obtained by scanning and photographing a microscopic camera, wherein the focusing judgment algorithm respectively calculates the longitudinal gradient and the transverse gradient of the side face focus plane scanned picture PR1, the side face focus plane scanned picture PR2 and … … and the side face focus plane scanned picture PRM to obtain sharpness values of each picture at each point, judges pixels with good focus according to the sharpness values, obtains x, y and z position information of the pixels with good focus, and outputs good focus pixels at a displacement depth DPR1, good focus pixels at a displacement depth DPR2, … … and good focus pixels at a displacement depth DPRM;
(5.3) performing a top view stitching algorithm on the output good focusing pixels at the displacement depth DPF1, the good focusing pixels at the displacement depth DPF2, … … and the good focusing pixels at the displacement depth DPFN, performing point cloud computing and registration on the good focusing pixels after smooth processing by the top view stitching algorithm, fusing registered point cloud data to obtain a reconstruction model, and finally generating a surface to obtain a top view 3D map;
(5.4) the output good focusing pixels at the displacement depth DPR1, the good focusing pixels at the displacement depth DPR2, … … and the good focusing pixels at the displacement depth DPRM are subjected to side view stitching algorithm, the side view stitching algorithm carries out point cloud calculation and registration on the good focusing pixels after smooth processing, the registered point cloud data are fused to obtain a reconstruction model, and finally a side view 3D image is generated on the surface to obtain;
(5.5) obtaining the 3D image by a stitching and fusion algorithm based on the displacement depth information by using the top view 3D image and the side view 3D image.
Examples
As shown in fig. 7, this example is a five-sided imaging 3D reconstruction technology apparatus designed based on the method of the present invention, including two sets of image acquisition apparatuses, where the first set of image acquisition apparatuses includes: the device comprises an XYZ three-axis sliding table 41, an upper surface scanning microscope camera 42, a lens 43 connected with the upper surface scanning microscope camera 42, a lens shockproof fixing 44 arranged at the bottom of the lens 43, an annular light source 45 arranged below the lens 43, a suction nozzle 46 for extracting targets arranged beside the camera, a material tray 47 to be detected arranged below the collecting device, a material supplementing tray 48 and a waste bin 49. Another set of image acquisition devices is mounted below the five-sided imaging mirror 410, including: the second annular light source 411 and the second lens are fixed 412 in a vibration-proof way, and are connected with the pentahedral imaging lens 413, the pentahedral imaging microscope 414 and the upper computer 415.
In this embodiment, the upper surface scanning microscope camera is connected to the XYZ three-axis sliding table, and the upper surface scanning microscope camera can move along with the XYZ three-axis sliding table.
In the embodiment, firstly, scanning and photographing are carried out on the upper surface of a component to be tested in a material tray to be tested through an upper surface scanning camera, and defect detection is carried out on the upper surface; if the defect exists, the component is moved to a waste bin through a suction nozzle, and a new component is sucked from a feeding disc to be placed in a to-be-detected feeding disc for waiting detection; and if the defect does not exist, sucking the element to be detected into a pentahedral imaging reflecting mirror, and carrying out pentahedral detection by scanning the pentahedral imaging 3D reconstruction through a focal plane.
As shown in fig. 4, the microscopic camera focal plane sweeps, the imaging object reflects the mirror image of the imaging object through the reflecting mirror 1, the microscopic camera focal plane moves upwards, and the imaging object and the mirror image of the imaging object are swept in the focal plane.
As shown in fig. 5, the five-sided imaging mirror is composed of a mirror surface 1, a mirror surface 2, a mirror surface 3 and a mirror surface 4; the reflecting mirror surface forms an included angle of 45 degrees with the main shaft.
In this example, the five-sided imaging 3D reconstruction is performed by focal plane scanning, as shown in fig. 6, and specifically the following steps are performed:
s1, inputting a starting signal;
s2, controlling the microscopic camera to move along the main shaft at a constant speed through the upper computer according to the starting signal;
s3, performing front focal plane scanning 3D imaging on the imaging object by the microscope camera from bottom to top at constant speed, and obtaining a front focal plane scanning picture PF1, front focal plane scanning pictures PF2 and … … and a front focal plane scanning picture PFN;
s301, photographing an imaging object by the tangent of a focal plane of a microscopic camera and the imaging object;
s302, a focal plane of the microscopic camera continues to move upwards to pass through a point A of an imaging object, and a front focal plane sweep picture is obtained;
and S303, the focal plane of the microscope camera continues to move upwards through the point B of the imaging object until the focal plane of the front surface of the whole element is swept.
S4, continuing upward uniform motion of the microscope camera to carry out side face focal plane scanning 3D imaging on the imaging object, and obtaining a side face focal plane scanning picture PR1, side face focal plane scanning pictures PR2 and … … and a side face focal plane scanning picture PRM;
s401, forming a mirror image of an imaging object through a mirror surface of a reflecting mirror, wherein a point A of the imaging object corresponds to a point A ' of the mirror image of the imaging object, a point B of the imaging object corresponds to a point B ' of the mirror image of the imaging object, and a point C of the imaging object corresponds to a point C ' of the mirror image of the imaging object;
s402, a focal plane of a microscopic camera passes through a point of contact of a mirror image of an imaging object imaged by reflection of a reflecting mirror surface of the imaging object, and a focal plane sweep picture is obtained;
s403, a focal plane of the microscope camera continues to move upwards to pass through points A ', B' of a mirror image of the imaging object to obtain a focal plane sweep picture;
s404, the focal plane of the microscopic camera continues to move upwards to pass through the points A ', B ', C ' of the mirror image of the imaging object to complete the focal plane sweep of the side surface of the whole element.
S5, controlling the microscopic camera through the upper computer to finish focal plane scanning, and outputting a result to the upper computer for processing to finish five-sided imaging 3D reconstruction of the focal plane scanning of the imaging object;
s501, obtaining pixels with good focus in front focal plane sweeping imaging through a focus judgment algorithm by using a front focal plane sweeping picture PF1, front focal plane sweeping pictures PF2 and … … and a front focal plane sweeping picture PFN;
s502, obtaining pixels with good focus in side face focus plane scanning imaging through a focus judgment algorithm by using a side face focus plane scanning picture PR1, side face focus plane scanning pictures PR2 and … … and a side face focus plane scanning picture PRM;
s503, obtaining a top view 3D image through a top view stitching algorithm by using the good focusing pixels at the displacement depth DPF1, the good focusing pixels at the displacement depth DPF2, … … and the good focusing pixels at the displacement depth DPFN;
s504, obtaining a side view 3D image through a side view stitching algorithm by using the good focusing pixels at the displacement depth DPR1, the good focusing pixels at the displacement depth DPR2, … … and the good focusing pixels at the displacement depth DPRM;
and S505, splicing and fusing the top view 3D image and the side view 3D image by using a splicing and fusing algorithm to obtain a 3D image of the final imaging object.
According to the invention, the focal plane is used for sweeping so as to generate image sequences with different moving depths, and 3D reconstruction images are used for completing 3D splicing reconstruction of 5 surfaces of an object, so that the image sequences obtained by the focal plane sweeping can be used for extracting a good focusing imaging part in the image sequences through 3D reconstruction image processing, 3D reconstruction of five-surface imaging is completed, and consistency of imaging and an original is ensured. Meanwhile, as the 5-surface 3D reconstruction of an imaging object is realized without adding detection hardware for depth detection, the invention is also a low-cost 3D imaging technology path, and the invention is applied to microscopic vision detection procedures of micro products such as electronic components, micro-electronic mechanical devices, integrated circuits and the like, can effectively improve detection efficiency and detection precision, and has wide industrial application prospect.
Claims (2)
1. A focal plane swept five-sided imaging 3D reconstruction method, comprising the steps of:
s1, carrying out front focal plane scanning on a target to be detected, wherein the method specifically comprises the following steps: after fixing a target to be measured, setting imaging equipment to be close to the target to be measured from far to near along the vertical direction, and finishing focal plane scanning of the front face of the target to be measured to obtain a plurality of front face focal plane scanning pictures; when a front focal plane is swept by a target to be detected, taking the tangential position of the focal plane and the target to be detected as the initial position of the front focal plane sweep, and taking the initial position as a reference, thereby obtaining the displacement depth of each front focal plane sweep picture;
s2, carrying out side focal plane sweep on a target to be detected, wherein the side focal plane sweep specifically comprises the following steps: setting a reflecting mirror surface, forming a mirror image of a target to be detected in the reflecting mirror surface, enabling the imaging equipment to be far from the target to be detected and near to the target to be detected along the vertical direction, and finishing focal plane scanning of the mirror image of the target to be detected to obtain a plurality of side face focal plane scanning pictures; when a side focal plane is swept by a target to be detected, taking the position of the focal plane tangent to the mirror image as the initial position of the side focal plane sweep, and taking the initial position as a reference, thereby obtaining the displacement depth of each side focal plane sweep picture;
s3, performing 3D reconstruction by using the obtained multiple front face focal plane sweep pictures and the obtained multiple side face focal plane sweep pictures to obtain a 3D image of the target to be detected; the specific method comprises the following steps:
the front 3D reconstruction is carried out on the target to be detected by utilizing the obtained multiple front focal plane sweep pictures, and the specific method comprises the following steps: the front focal plane sweep pictures obtained by definition are PF1, PF2, … … and PFN respectively, the sharpness values of each picture at each point are obtained by calculating the longitudinal gradient and the transverse gradient of N pictures through a focusing judgment algorithm, the pixels with good focus are judged through the sharpness values, and the x, y and z position information of the pixels with good focus is obtained, so that the pixels DPF1, DPF2, … … and DPFN with good focus at the displacement depth corresponding to each picture are obtained, and the focusing algorithm formula is as shown in the following formula 1:
(equation 1)
In equation 1:representation of image->Corresponding pixel dot +.>Gray value of +.>Calculating a result for the image definition;
and then, carrying out point cloud calculation and registration on the well-focused pixels through smoothing processing by a top view stitching algorithm, fusing the registered point cloud data to obtain a reconstruction model, and finally generating the surface of the object to be detected to obtain a top view 3D map, wherein the top view stitching algorithm is as follows in formula 2 and formula 3:
(equation 2)
In equation 2:for the size of the sliding window +.>For the image size, for->Go (go)/(go)>Summing the focused good pixels of the columns to obtain the good focused pixels at the displacement depth corresponding to each picture;
(equation 3)
In equation 3:wherein pixels which are well focused during focal plane sweeping are +.>Point cloud data for focusing good pixels, wherein +.>Coordinates of pixels well focused in the reference coordinate system, +.>Time information of a well focused pixel when a focal plane is scanned is fused with the found point cloud data to generate a top view 3D map;
the obtained multiple side focal plane sweep pictures are utilized to carry out side 3D reconstruction on the object to be detected, and the specific method comprises the following steps: defining the obtained side focal plane sweep pictures as PR1, PR2, … … and PRM respectively, calculating the longitudinal gradient and the transverse gradient of M pictures through a focusing judgment algorithm to obtain the sharpness value of each picture at each point, judging pixels with good focusing through the sharpness value, obtaining the x, y and z position information of the pixels with good focusing, so as to obtain good focusing pixels DPR1, DPR2, … … and DPRM at the displacement depth corresponding to each picture, carrying out smoothing treatment on the good focusing pixels through a side view stitching algorithm, carrying out point cloud calculation and registration on the good focusing pixels, fusing the registered point cloud data to obtain a reconstruction model, adopting formulas as formula 2 and formula 3, and finally generating the surface of an object to be detected to obtain a side view 3D picture;
and obtaining the 3D image by a stitching and fusion algorithm based on the displacement depth information by using the top view 3D image and the side view 3D image.
2. A method of five-sided imaging 3D reconstruction of a focal plane sweep according to claim 1, characterized in that the set mirror surfaces are in particular five-sided imaging mirrors, each at an angle of 45 ° to the vertical midline of the imaging device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310931474.7A CN116642895B (en) | 2023-07-27 | 2023-07-27 | Five-face imaging 3D reconstruction method for focal plane sweep |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310931474.7A CN116642895B (en) | 2023-07-27 | 2023-07-27 | Five-face imaging 3D reconstruction method for focal plane sweep |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116642895A CN116642895A (en) | 2023-08-25 |
| CN116642895B true CN116642895B (en) | 2023-09-29 |
Family
ID=87625198
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310931474.7A Active CN116642895B (en) | 2023-07-27 | 2023-07-27 | Five-face imaging 3D reconstruction method for focal plane sweep |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116642895B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108377658A (en) * | 2014-11-28 | 2018-08-07 | 台湾爱美科股份有限公司 | Autofocus system in digital holography and method |
| CN111256616A (en) * | 2020-03-30 | 2020-06-09 | 阳宇春 | Metering-level 3D super-depth-of-field microscopic system and detection method |
| CN113204107A (en) * | 2021-04-28 | 2021-08-03 | 武汉理工大学 | Three-dimensional scanning microscope with double objective lenses and three-dimensional scanning method |
-
2023
- 2023-07-27 CN CN202310931474.7A patent/CN116642895B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108377658A (en) * | 2014-11-28 | 2018-08-07 | 台湾爱美科股份有限公司 | Autofocus system in digital holography and method |
| CN111256616A (en) * | 2020-03-30 | 2020-06-09 | 阳宇春 | Metering-level 3D super-depth-of-field microscopic system and detection method |
| CN113204107A (en) * | 2021-04-28 | 2021-08-03 | 武汉理工大学 | Three-dimensional scanning microscope with double objective lenses and three-dimensional scanning method |
Non-Patent Citations (1)
| Title |
|---|
| 基于光场成像的三维粒子追踪测速技术;刘慧芳;周骛;蔡小舒;周雷;郭延昂;;光学学报(01);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116642895A (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110763152B (en) | Underwater active rotation structure light three-dimensional vision measuring device and measuring method | |
| CN109919911B (en) | Mobile three-dimensional reconstruction method based on multi-view photometric stereo | |
| WO2012053521A1 (en) | Optical information processing device, optical information processing method, optical information processing system, and optical information processing program | |
| CN114998499B (en) | Binocular three-dimensional reconstruction method and system based on line laser galvanometer scanning | |
| CN103728315B (en) | Heavy-calibre element surface detection apparatus and damage accordingly method for rapidly positioning | |
| JP2012013660A (en) | Point group data processing device, point group data processing system, point group data processing method, and point group data processing program | |
| CN112067233B (en) | Six-degree-of-freedom motion capture method for wind tunnel model | |
| CN109883391B (en) | Monocular distance measurement method based on digital imaging of microlens array | |
| CN111189415B (en) | Multifunctional three-dimensional measurement reconstruction system and method based on line structured light | |
| CN113267141B (en) | Microscopic three-dimensional information acquisition equipment | |
| CN101930606A (en) | Field depth extending method for image edge detection | |
| CN109974659A (en) | A kind of embedded range-measurement system based on binocular machine vision | |
| CN101033954A (en) | Liquid assisted section scanning measurement method for three-dimensional form | |
| CN109781745A (en) | A kind of optical element surface defects detection Atomatic focusing method and device | |
| Liu et al. | Real-time 3D surface measurement in additive manufacturing using deep learning | |
| CN201007646Y (en) | Liquid auxiliary dislocation scanning three-dimensional shape measuring apparatus | |
| Walecki et al. | Super-resolution 3-D laser scanning based on interval arithmetic | |
| CN116642895B (en) | Five-face imaging 3D reconstruction method for focal plane sweep | |
| CN111340959B (en) | Three-dimensional model seamless texture mapping method based on histogram matching | |
| CN105651699B (en) | It is a kind of based on the dynamic of area array cameras with burnt method | |
| Lachat et al. | Assessment of the accuracy of 3D models obtained with DSLR camera and Kinect v2 | |
| Lin et al. | Depth from motion and defocus blur | |
| Loktev et al. | Image Blur Simulation for the Estimation of the Behavior of Real Objects by Monitoring Systems. | |
| Georgopoulos | Photogrammetric automation: is it worth? | |
| CN114463405A (en) | Method, device and system for accelerating surface scanning line laser 3D camera and FPGA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |