CN114636385B - Three-dimensional imaging method and system based on light field camera and three-dimensional imaging measurement production line - Google Patents
Three-dimensional imaging method and system based on light field camera and three-dimensional imaging measurement production line Download PDFInfo
- Publication number
- CN114636385B CN114636385B CN202011475925.3A CN202011475925A CN114636385B CN 114636385 B CN114636385 B CN 114636385B CN 202011475925 A CN202011475925 A CN 202011475925A CN 114636385 B CN114636385 B CN 114636385B
- Authority
- CN
- China
- Prior art keywords
- light field
- image
- field camera
- linear array
- spliced
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 42
- 238000005259 measurement Methods 0.000 title claims abstract description 13
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 11
- 230000007246 mechanism Effects 0.000 claims abstract description 14
- 230000000007 visual effect Effects 0.000 claims description 16
- 238000009826 distribution Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 9
- 238000000034 method Methods 0.000 description 15
- 239000011159 matrix material Substances 0.000 description 10
- 230000007547 defect Effects 0.000 description 7
- 238000001514 detection method Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000009792 diffusion process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000009877 rendering Methods 0.000 description 4
- 230000008030 elimination Effects 0.000 description 3
- 238000003379 elimination reaction Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- ATJFFYVFTNAWJD-UHFFFAOYSA-N Tin Chemical compound [Sn] ATJFFYVFTNAWJD-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 229910000679 solder Inorganic materials 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/557—Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0075—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for altering, e.g. increasing, the depth of field or depth of focus
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10052—Images from lightfield camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/957—Light-field or plenoptic cameras or camera modules
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a three-dimensional imaging method and system based on a light field camera and a three-dimensional imaging measurement production line, a moving mechanism, a light source, the light field camera or a linear array light field camera. According to the invention, the images are respectively obtained through multiple scans of the light field camera or the linear array light field camera, and then the high-resolution light field multi-view images are obtained through splicing, so that a better three-dimensional imaging result is obtained, and the measurement accuracy is improved.
Description
Technical Field
The invention relates to the technical field of three-dimensional photoelectric measurement of conventional objects, in particular to a three-dimensional imaging system based on a linear array light field camera scanning device.
Background
The light field camera is a novel camera which appears in recent years, a micro lens array is added between a sensor and a main lens of a conventional camera, the propagation direction of light rays is recorded, a unique light field image coded by the lens array is formed, and the light field image is processed and rendered, so that three-dimensional information can be obtained.
The prior art has solved the technical problem of how to calibrate a light field camera.
Patent document CN106303175a discloses a virtual reality three-dimensional data acquisition method based on multiple perspectives of a single light field camera, comprising the steps of: s101, acquiring a microlens calibration map by using a single light field camera; s102, positioning the center position of the micro lens by using a calibration chart; s103, acquiring a light field picture; s104, selecting a certain pixel with the same relative position from each microlens in the light field image; s105, using the selected pixels as sampling points, solving pixel values embedded in the square matrix, and further forming an image of a visual angle; s106, selecting pixels at different positions, and repeating the steps S103 to S105 until all the pixels are selected.
Patent document CN111351446a discloses a light field camera calibration method for three-dimensional morphology measurement, which calibrates calibration plates at different positions in space and corresponding light field original images to obtain a corresponding relationship between a light field parallax image and three-dimensional space information; shooting a plurality of defocused soft light pure color calibration plates by using a light field camera to obtain a light field white image; calculating to obtain a vignetting elimination matrix according to the white image of the light field camera; iterative calculation is carried out to obtain a sub-pixel level central coordinate matrix of the micro lens of the light field camera; shooting a plurality of round dot calibration plates with known three-dimensional space positions by a light field camera and performing vignetting elimination treatment; the method can efficiently and accurately convert the parallax image of the light field into the three-dimensional space information without main lens distortion.
Patent document CN111340888A discloses a method and a system for calibrating a light field camera without white image, which firstly obtain an electronic checkerboard light field original image shot by the light field camera, and then calibrate a microlens array according to the light field original image to generate a calibration result of the microlens array and a central point grid of the microlens array; and extracting line characteristics of the original image of the light field by adopting a template matching method, and calibrating internal and external parameters of a projection model of the light field camera by taking the line characteristics as calibration data. The method is independent of white images, and can obtain the central point grid of the micro lens, the array gesture and the internal and external parameters of the camera projection model only by processing the original checkerboard light field, and has the characteristics of high accuracy of light field camera calibration and wide application range.
However, because resolution in the x and y directions is sacrificed when imaging a light field camera, the resolution of the resulting light field multi-view image remains low even with higher resolution light field cameras.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a three-dimensional imaging method and system based on a light field camera and a three-dimensional imaging measurement production line.
The three-dimensional imaging method based on the light field camera provided by the invention comprises the following steps:
a photographed image acquisition step: acquiring a plurality of groups of images to be spliced; wherein: each group of images to be spliced comprises a light field depth image and a light field multi-view image which are obtained by shooting an object to be detected at shooting positions by a linear array light field camera, and the distribution of the shooting positions forms a linear array parallel to the width direction of an image sensor of the linear array light field camera;
and (3) image stitching: splicing the plurality of groups of images to be spliced to obtain spliced images; the spliced image comprises a spliced light field depth image and a spliced light field multi-view image;
three-dimensional information acquisition: and obtaining three-dimensional imaging information of the object to be detected according to the spliced image.
Preferably, the resolution of the linear array light field camera is n×m, and the moving distance of the linear array light field camera between adjacent shooting positions is m pixels; after the linear array light field camera moves for N times, the size of the generated spliced image is N x m x (n+1).
Preferably, the three-dimensional information acquisition step includes:
a center view image acquisition step: splicing to obtain a center visual angle image;
a step of obtaining a depth image of a central visual field: splicing to obtain a central visual angle light field depth image;
and a point cloud information acquisition step: and obtaining the point cloud information and the size information of the object to be measured according to the center view angle graph and the center view angle light field depth image.
According to the invention, a three-dimensional imaging system based on a light field camera comprises:
a shooting image acquisition module: acquiring a plurality of groups of images to be spliced; wherein: each group of images to be spliced comprises a light field depth image and a light field multi-view image which are obtained by shooting an object to be detected at shooting positions by a linear array light field camera, and the distribution of the shooting positions forms a linear array parallel to the width direction of an image sensor of the linear array light field camera;
and the image splicing module is used for: splicing the plurality of groups of images to be spliced to obtain spliced images; the spliced image comprises a spliced light field depth image and a spliced light field multi-view image;
three-dimensional information acquisition module: and obtaining three-dimensional imaging information of the object to be detected according to the spliced image.
Preferably, the resolution of the linear array light field camera is n×m, and the moving distance of the linear array light field camera between adjacent shooting positions is m pixels; after the linear array light field camera moves for N times, the size of the generated spliced image is N x m x (n+1).
Preferably, the three-dimensional information acquisition module includes:
a center view image acquisition module: splicing to obtain a center visual angle image;
the central visual field depth image acquisition module is used for: splicing to obtain a central visual angle light field depth image;
the point cloud information acquisition module is used for: and obtaining the point cloud information and the size information of the object to be measured according to the center view angle graph and the center view angle light field depth image.
According to the present invention there is provided a computer readable storage medium storing a computer program which when executed by a processor implements the steps of the light field camera based three-dimensional imaging method.
According to the light field camera scanning device provided by the invention,
comprising the following steps: the device comprises a moving mechanism, a light source and a linear array light field camera; the linear array light field camera is arranged on the moving mechanism, and the moving mechanism drives the light field camera to move to each shooting position for scanning; the distribution of each shooting position forms a linear array parallel to the width direction of the image sensor of the linear array light field camera; the light source irradiates light to an object to be measured; the linear array light field camera shoots light field images of an object to be detected at all shooting positions.
Preferably, it comprises: a controller;
the controller includes the computer readable storage medium storing the computer program.
The three-dimensional imaging measurement production line comprises the three-dimensional imaging system based on the light field camera;
or, a computer-readable storage medium including the computer program stored thereon;
or, the linear array light field camera scanning device is included.
Compared with the prior art, the invention has the following beneficial effects:
1. according to the invention, the light field camera is utilized for scanning, so that a high-resolution light field multi-view image is obtained, a better three-dimensional imaging result is obtained, the measurement precision is improved, and the high-resolution imaging of the three-dimensional morphology of the object to be measured is realized.
2. Compared with a common light field camera, the image of the light field scanned by the linear array light field camera has larger resolution, the obtained light field multi-view image has larger resolution compared with the common light field camera, the defect that the light field camera sacrifices resolution in the x and y directions is well solved, and a more accurate three-dimensional image can be obtained through calculation by the light field multi-view image with high resolution.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, given with reference to the accompanying drawings in which:
FIG. 1 is a flow chart of a three-dimensional imaging process according to one of the embodiments of the present invention.
Fig. 2 is a schematic diagram of a linear array light field camera scanning shooting a medium under illumination of a light source according to an embodiment of the present invention.
Fig. 3 is a partial region of a light field image of a chip obtained by one scanning by a linear array light field camera according to an embodiment of the present invention, where the resolution of the light field image obtained at one time is 7912×10.
Fig. 4 shows a light field image partial region obtained by splicing after multiple scans of a linear array light field camera, and the resolution of the light field image obtained after multiple scans is 7912×5430.
Fig. 5 is a view image of the light field center of the chip.
Fig. 6 is a light field depth image corresponding to fig. 5.
Fig. 7 is a three-dimensional point cloud image corresponding to fig. 5.
The figure shows:
linear array light field camera 100
Moving mechanism 101 of linear array light field camera
First light source 201 (Red light)
Second light source 202 (Green light)
Third light source 203 (blue light)
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the present invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications could be made by those skilled in the art without departing from the inventive concept. These are all within the scope of the present invention.
The three-dimensional imaging method based on the light field camera provided by the invention comprises the following steps:
a light field camera calibration step: calibrating a light field camera in the linear array light field camera; the image sensor of the linear array light field camera and the image sensor of the linear array light field camera are different in types, and the image sensor of the linear array light field camera is an image sensor with the width being the diameter of the micro lens in the single-row micro lens and the length being the length of the single-row micro lens (namely the sum of the diameters of all the micro lenses in the single-row micro lens). The sensor of the area array light field camera is the same as the image sensor of the common area array 2D camera.
Shooting: the method comprises the steps that a light field camera obtains all groups of images to be spliced according to all shooting positions of an area array, or a linear array light field camera obtains all groups of images to be spliced according to all shooting positions of a linear array;
each group of images to be spliced comprises a light field depth image and a light field multi-view image which are obtained by shooting an object to be detected at shooting positions by a linear array light field camera, and the distribution of the shooting positions forms a linear array parallel to the width direction of an image sensor of the linear array light field camera;
and (3) image stitching: splicing the plurality of groups of images to be spliced to obtain spliced images; the spliced image comprises a spliced light field depth image and a spliced light field multi-view image;
three-dimensional information acquisition: and obtaining three-dimensional imaging information of the object to be detected according to the spliced image.
The light field camera calibration step comprises the following steps:
step A1: adjusting the focal length and aperture of the linear array light field camera, scanning and shooting a plurality of defocused soft light pure color calibration plates, and obtaining the white of the light field cameraAn image; and calculating according to the white image of the light field camera to obtain a vignetting matrix and a sub-pixel level center coordinate matrix of the micro lens of the light field camera. Specifically, the main lens and the aperture of the linear array light field camera are adjusted to enable the micro lens array of the original light field white image of the light field camera to be just or approximately tangential; the linear array light field camera scans and shoots a plurality of defocused soft light pure color calibration plate images, the calibration plate is positioned on a pure color background plate with uniform light intensity at the defocused position of the light field camera, wherein, the deghosting matrix is a matrix obtained by averaging and normalizing a plurality of original light field white images W (u, v)
The central coordinate matrix of the micro-lens sub-pixel level of the linear array light field camera is a light field white image +.>
And after carrying out local maximum processing on each microlens, carrying out iterative optimization processing to obtain the center coordinates of the sub-pixel level microlenses. Where u represents the coordinates of the light field white image in the x direction and v represents the coordinates of the light field white image in the y direction. More specifically, an optical lens suitable for focal length and magnification is selected as the lens of the microlens according to the area size and the measurement depth range of the object to be measured. The lens aperture of the micro lens is adjusted to be matched with the aperture of the main lens of the light field camera, namely the micro lens aperture is matched with the aperture of the main lens, and the method is specifically characterized in that the linear array light field camera scans and shoots defocused soft light pure color calibration plate images, and a micro lens array in the images is just or nearly in a tangential state. The light field white image or the light field camera white image refers to a pure white background image shot by the light field camera, and the shape of the micro lens array is particularly obvious on the image. The aperture can thus be adjusted based on the image, ensuring that the microlens images are exactly tangential. After the adjustment is finished, shooting a plurality of pure-color background plates with uniform light intensity, namely defocused soft light pure-color calibration plates, which are positioned at defocused positions of the linear array light field camera. Averaging and normalizing the multiple light field white images to obtain a vignetting elimination matrix +.>
All light field original images shot later need to be divided by the vignetting matrix, so that light field white image calibration is completed. The original image of the light field refers to the image of the light field which is not processed by the multi-view image algorithm of the light field. After the light field package image calibration step is completed, processing the light field white image by using a filter, removing light field white image noise, and carrying out non-maximum suppression on the filtered light field image; taking a local maximum value according to the processed image, wherein the maximum value is just the integer-level center of the micro lens of the light field camera; and taking the integer-level microlens center as an initial iteration value, iteratively optimizing the microlens arrangement grid, and finally obtaining the angle and the interval of microlens arrangement to obtain the sub-pixel-level microlens center.
Step A2: and shooting a plurality of dot calibration plates with known space three-dimensional positions by using the light field camera, and establishing a light field mathematical model from three-dimensional coordinates to parallax to finish the scale calibration of the light field camera. Specifically, the three-dimensional coordinates of each dot on the dot calibration plate are known, the dispersion degree of the dot of the calibration plate and the corresponding parallax value are obtained after the calibration plate is shot by a light field camera, and then the relationship between the parallax value and the three-dimensional coordinates is obtained through fitting and calibration. More specifically, the linear array light field camera scale calibration step requires assembling a displacement table and a scale calibration plate: firstly, a fixed-scale calibration plate moves the calibration plate to a fixed space distance from a focal plane in a focal plane area of a linear array light field camera continuously, scanning shooting is carried out, and the space position of a point on the calibration plate is known, so that the space position of the point on the calibration plate in the whole moving process can be obtained. The dot calibration points form a diffusion circle on the light field image, the diameter of the diffusion circle is obtained through processing, the parallax value of the diffusion circle and the pixel coordinate of the diffusion circle are obtained through calculation, and the relation between the three-dimensional coordinate in space and the pixel coordinate and the parallax value of the light field camera is obtained through fitting according to the linear array light field camera scale calibration model.
Step A3: and irradiating the light source on the object to be detected, and obtaining a shooting image of the object to be detected under the light source. Specifically, the angle of the light source is set, so that the light source light can clearly irradiate an object to be detected, and the linear array light field camera is ensured to image the object to be detected under multispectral light.
In the shooting step, taking the size of the designed linear array light field camera as 7912×10 as an example, in the scanning shooting process, in order to ensure that the light field pictures scanned once can be well spliced, a moving mechanism of the linear array camera moves for 10 times of the distance of pixels each time, so that no overlapped part exists between the images to be spliced.
In the step of acquiring the shooting image, specifically, after the light field camera shoots an object to be detected, light field multi-view rendering is performed to obtain a light field multi-view image and a light field parallax image, and the parallax image is converted into a light field depth image through the conversion relation of parallax and three-dimensional coordinates obtained through calibration. More specifically, conventional light field rendering and depth estimation are performed based on the original light field image. Firstly, performing light field multi-view rendering to obtain a light field multi-view image of an object to be detected; and then further calculating to obtain a light field parallax image, and converting the light field parallax image into a light field depth image according to a light field camera scale calibration result, wherein the depth information of the same object to be detected in the light field depth image is obtained, so that the object to be detected can be subjected to three-dimensional imaging.
In the image stitching step, stitching the plurality of groups of images to be stitched to obtain stitched images; the spliced image comprises a spliced light field depth image and a spliced light field multi-view image. Specifically, because the distance of each movement of the moving mechanism driving the linear array light field camera to move is m pixels (if the resolution of the linear array light field camera is n×m), after N times of movements, the generated spliced image is n×m×n+1, and the quality of the spliced image depends on the accuracy of the transmission mechanism.
In the three-dimensional information obtaining step, specifically, the light field depth image is subjected to three-dimensional mapping based on the scale calibration result, and three-dimensional point cloud information of the object to be measured is obtained. Specifically, the object to be detected can be an object with a larger area and smaller defects, and the method has better application to detection of the object. Such as glass defect depth detection, screen defect depth detection, semiconductor gold wire and solder ball detection, etc.
The invention provides a three-dimensional imaging system based on a light field camera, which can be realized by executing the step flow of the three-dimensional imaging method based on the light field camera, namely the three-dimensional imaging method based on the light field camera is a preferable example of the three-dimensional imaging system based on the light field camera.
According to the invention, a three-dimensional imaging system based on a light field camera comprises:
a shooting image acquisition module: acquiring a plurality of groups of images to be spliced; wherein: each group of images to be spliced comprises a light field depth image and a light field multi-view image which are obtained by shooting an object to be detected at shooting positions by a light field camera, and the distribution of the shooting positions forms an area array; or each group of images to be spliced comprises a light field depth image and a light field multi-view image which are obtained by shooting an object to be detected at shooting positions by a linear array light field camera, and the distribution of the shooting positions forms a linear array parallel to the width direction of an image sensor of the linear array light field camera;
and the image splicing module is used for: splicing the plurality of groups of images to be spliced to obtain spliced images; the spliced image comprises a spliced light field depth image and a spliced light field multi-view image;
three-dimensional information acquisition module: and obtaining three-dimensional imaging information of the object to be detected according to the spliced image.
Preferably, the resolution of the linear array light field camera is n×m, and the moving distance of the linear array light field camera between adjacent shooting positions is m pixels; after the linear array light field camera moves for N times, the size of the generated spliced image is N x m x (n+1).
Preferably, the three-dimensional information acquisition module includes:
a center view image acquisition module: splicing to obtain a center visual angle image;
the central visual field depth image acquisition module is used for: splicing to obtain a central visual angle light field depth image;
the point cloud information acquisition module is used for: and obtaining the point cloud information and the size information of the object to be measured according to the center view angle graph and the center view angle light field depth image.
The invention provides a light field camera scanning device, comprising: the device comprises a moving mechanism, a light source and a linear array light field camera; the linear array light field camera is arranged on the moving mechanism, and the moving mechanism drives the light field camera to move to each shooting position for scanning; the distribution of each shooting position forms a linear array parallel to the width direction of the image sensor of the linear array light field camera; the light source irradiates light to an object to be measured; the linear array light field camera shoots light field images of an object to be detected at all shooting positions.
According to the present invention there is provided a computer readable storage medium storing a computer program which when executed by a processor implements the steps of the light field camera based three-dimensional imaging method. The computer readable storage medium may be a read-only or read-write memory of a magnetic disk, an optical disk, etc. in a single chip microcomputer, a DSP, a processor, a data center, a server, a PC, an intelligent terminal, a special purpose machine, a light field camera, etc.
According to the invention, the three-dimensional measurement production line comprises the three-dimensional imaging system based on the light field camera or comprises the computer readable storage medium storing the computer program. The three-dimensional measurement production line can be a glass defect depth detection production line, a screen defect depth detection production line, a semiconductor gold wire and tin ball detection production line and the like.
The present invention is described in more detail below in connection with specific application scenarios, including chip three-dimensional imaging embodiments.
The specific process of the embodiment is as follows:
the linear array light field camera scanning device adopts a light field camera to match with a lens with 4 times of magnification for scanning shooting according to the size and the height of a chip; the light field camera is matched with a micro lens with proper aperture and focal length, and then scans and shoots a defocused soft light pure color calibration plate to calibrate light field white images and micro lens centers; the light field camera scans and shoots a plurality of scale calibration plates with different spatial positions to calibrate the scales of the light field camera; the chip is polished by matching with a light source, so that good imaging can be performed on a light field camera; performing light field multi-view rendering and depth calculation on the light field image obtained each time to finally obtain a center view image (figure 5) of the corresponding chip and a corresponding light field depth image (figure 6); finally, the point cloud information and the size information of the chip are obtained (fig. 7).
Those skilled in the art will appreciate that the systems, apparatus, and their respective modules provided herein may be implemented entirely by logic programming of method steps such that the systems, apparatus, and their respective modules are implemented as logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc., in addition to the systems, apparatus, and their respective modules being implemented as pure computer readable program code. Therefore, the system, the apparatus, and the respective modules thereof provided by the present invention may be regarded as one hardware component, and the modules included therein for implementing various programs may also be regarded as structures within the hardware component; modules for implementing various functions may also be regarded as being either software programs for implementing the methods or structures within hardware components.
The foregoing describes specific embodiments of the present invention. It is to be understood that the invention is not limited to the particular embodiments described above, and that various changes or modifications may be made by those skilled in the art within the scope of the appended claims without affecting the spirit of the invention. The embodiments of the present application and features in the embodiments may be combined with each other arbitrarily without conflict.
Claims (6)
1. A three-dimensional imaging method based on a light field camera, comprising:
a photographed image acquisition step: acquiring a plurality of groups of images to be spliced; wherein: each group of images to be spliced comprises a light field depth image and a light field multi-view image which are obtained by shooting an object to be detected at shooting positions by a linear array light field camera, and the distribution of the shooting positions forms a linear array parallel to the width direction of an image sensor of the linear array light field camera;
and (3) image stitching: splicing the plurality of groups of images to be spliced to obtain spliced images; the spliced image comprises a spliced light field depth image and a spliced light field multi-view image;
three-dimensional information acquisition: according to the spliced image, three-dimensional imaging information of the object to be detected is obtained;
the resolution of the linear array light field camera is n x m, and the moving distance of the linear array light field camera between adjacent shooting positions is m pixels; after the linear array light field camera moves for N times, the size of the generated spliced image is N x m x (n+1);
the three-dimensional information acquisition step includes:
a center view image acquisition step: splicing to obtain a center visual angle image;
a step of obtaining a depth image of a central visual field: splicing to obtain a central visual angle light field depth image;
and a point cloud information acquisition step: and obtaining the point cloud information and the size information of the object to be measured according to the center view angle graph and the center view angle light field depth image.
2. A three-dimensional imaging system based on a light field camera, comprising:
a shooting image acquisition module: acquiring a plurality of groups of images to be spliced; wherein: each group of images to be spliced comprises a light field depth image and a light field multi-view image which are obtained by shooting an object to be detected at shooting positions by a linear array light field camera, and the distribution of the shooting positions forms a linear array parallel to the width direction of an image sensor of the linear array light field camera;
and the image splicing module is used for: splicing the plurality of groups of images to be spliced to obtain spliced images; the spliced image comprises a spliced light field depth image and a spliced light field multi-view image;
three-dimensional information acquisition module: according to the spliced image, three-dimensional imaging information of the object to be detected is obtained;
the resolution of the linear array light field camera is n x m, and the moving distance of the linear array light field camera between adjacent shooting positions is m pixels; after the linear array light field camera moves for N times, the size of the generated spliced image is N x m x (n+1);
the three-dimensional information acquisition module comprises:
a center view image acquisition module: splicing to obtain a center visual angle image;
the central visual field depth image acquisition module is used for: splicing to obtain a central visual angle light field depth image;
the point cloud information acquisition module is used for: and obtaining the point cloud information and the size information of the object to be measured according to the center view angle graph and the center view angle light field depth image.
3. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the light field camera based three-dimensional imaging method of claim 1.
4. A light field camera scanning device for implementing the light field camera-based three-dimensional imaging method of claim 1, comprising: the device comprises a moving mechanism, a light source and a linear array light field camera; the linear array light field camera is arranged on the moving mechanism, and the moving mechanism drives the light field camera to move to each shooting position for scanning; the distribution of each shooting position forms a linear array parallel to the width direction of the image sensor of the linear array light field camera; the light source irradiates light to an object to be measured; the linear array light field camera shoots light field images of an object to be detected at all shooting positions.
5. The light field camera scanning apparatus of claim 4 comprising: a controller;
the controller comprising the computer readable storage medium of claim 3 storing a computer program.
6. A three-dimensional imaging measurement production line comprising the light field camera-based three-dimensional imaging system of claim 2;
alternatively, a computer-readable storage medium comprising the computer program of claim 3;
alternatively, a light field camera scanning device comprising the light field camera of claim 4 or 5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011475925.3A CN114636385B (en) | 2020-12-15 | 2020-12-15 | Three-dimensional imaging method and system based on light field camera and three-dimensional imaging measurement production line |
| PCT/CN2021/079837 WO2022126870A1 (en) | 2020-12-15 | 2021-04-23 | Three-dimensional imaging method and method based on light field camera and three-dimensional imaging measuring production line |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011475925.3A CN114636385B (en) | 2020-12-15 | 2020-12-15 | Three-dimensional imaging method and system based on light field camera and three-dimensional imaging measurement production line |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114636385A CN114636385A (en) | 2022-06-17 |
| CN114636385B true CN114636385B (en) | 2023-04-28 |
Family
ID=81944865
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011475925.3A Active CN114636385B (en) | 2020-12-15 | 2020-12-15 | Three-dimensional imaging method and system based on light field camera and three-dimensional imaging measurement production line |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114636385B (en) |
| WO (1) | WO2022126870A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115908242B (en) * | 2022-09-22 | 2023-10-03 | 深圳市明测科技有限公司 | Chip gold thread whole line detection method and system for multichannel image fusion |
| CN116952845A (en) * | 2023-09-19 | 2023-10-27 | 宁德时代新能源科技股份有限公司 | Battery sealing nail welding detection system and method |
| CN117036175B (en) * | 2023-10-08 | 2024-01-09 | 之江实验室 | Linear array image splicing method, device, medium and equipment |
| CN117053690A (en) * | 2023-10-10 | 2023-11-14 | 合肥联宝信息技术有限公司 | Imaging method and device for to-be-positioned piece, electronic equipment and storage medium |
| CN117491355B (en) * | 2023-11-06 | 2024-07-02 | 广州航海学院 | Visual detection method for abrasion loss of three-dimensional curved surface of rake teeth type large component |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103884650B (en) * | 2014-03-28 | 2016-08-24 | 北京大恒图像视觉有限公司 | A kind of multiple light courcess linear array imaging system and method |
| CN106331439B (en) * | 2015-07-10 | 2019-10-25 | 深圳超多维科技有限公司 | Microlens array imaging device and imaging method |
| CN105806249A (en) * | 2016-04-15 | 2016-07-27 | 南京拓控信息科技股份有限公司 | Method for achieving image collection and depth measurement simultaneously through a camera |
| CN106303175A (en) * | 2016-08-17 | 2017-01-04 | 李思嘉 | A kind of virtual reality three dimensional data collection method based on single light-field camera multiple perspective |
| WO2018140348A1 (en) * | 2017-01-26 | 2018-08-02 | The Charles Stark Draper Laboratory, Inc. | Method and apparatus for light field generation |
| CN107977998B (en) * | 2017-11-30 | 2021-01-26 | 浙江大学 | Light field correction splicing device and method based on multi-view sampling |
| CN108175535A (en) * | 2017-12-21 | 2018-06-19 | 北京理工大学 | A kind of dentistry spatial digitizer based on microlens array |
| CN110648345B (en) * | 2019-09-24 | 2023-03-31 | 中国烟草总公司郑州烟草研究院 | Method and system for detecting flow of tobacco shred materials on conveying belt based on light field imaging |
| CN211825732U (en) * | 2019-12-20 | 2020-10-30 | 武汉精立电子技术有限公司 | Non-contact film pressing backlight panel detection device and backlight panel automatic detection line |
| CN111161404B (en) * | 2019-12-23 | 2023-05-09 | 华中科技大学鄂州工业技术研究院 | Annular scanning morphology three-dimensional reconstruction method, device and system |
| CN111340888B (en) * | 2019-12-23 | 2020-10-23 | 首都师范大学 | Light field camera calibration method and system without white image |
| CN111351446B (en) * | 2020-01-10 | 2021-09-21 | 奕目(上海)科技有限公司 | Light field camera calibration method for three-dimensional topography measurement |
| CN111538223B (en) * | 2020-04-30 | 2022-02-11 | 北京大学 | Holographic projection method based on light beam deflection |
| CN111982921B (en) * | 2020-05-21 | 2023-11-03 | 北京安视中电科技有限公司 | Method and device for detecting hole defects, conveying platform and storage medium |
| CN112040140A (en) * | 2020-09-02 | 2020-12-04 | 衢州光明电力投资集团有限公司赋腾科技分公司 | Wide-view-field high-resolution hybrid imaging device based on light field |
-
2020
- 2020-12-15 CN CN202011475925.3A patent/CN114636385B/en active Active
-
2021
- 2021-04-23 WO PCT/CN2021/079837 patent/WO2022126870A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| CN114636385A (en) | 2022-06-17 |
| WO2022126870A1 (en) | 2022-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114636385B (en) | Three-dimensional imaging method and system based on light field camera and three-dimensional imaging measurement production line | |
| US10127682B2 (en) | System and methods for calibration of an array camera | |
| US10043290B2 (en) | Image processing to enhance distance calculation accuracy | |
| CN110689581B (en) | Structured light module calibration method, electronic device and computer readable storage medium | |
| US20190268586A1 (en) | System and Methods for Calibration of an Array Camera | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| WO2021129437A1 (en) | Method and system for light calibration field camera without requiring white image | |
| CN111080705B (en) | Calibration method and device for automatic focusing binocular camera | |
| JP2013192177A (en) | Solid-state image pickup device and mobile information terminal | |
| CN112816493A (en) | Chip routing defect detection method and device | |
| JP7056131B2 (en) | Image processing system, image processing program, and image processing method | |
| CN114693583A (en) | Defect layering detection method and system based on light field camera and detection production line | |
| CN118333914B (en) | High dynamic range light field data white image correction method, system and medium | |
| CN112747670A (en) | BGA packaging solder ball detection system and method | |
| CN111127379B (en) | Rendering method of light field camera 2.0 and electronic equipment | |
| CN112747822A (en) | Three-dimensional imaging system and method | |
| CN113678060A (en) | Interchangeable lens, information processing device, information processing method, and program | |
| CN113923445B (en) | Light field camera calibration method and system under shift imaging condition | |
| CN112489141B (en) | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera | |
| CN115514877A (en) | Apparatus and method for noise reduction from multi-view image | |
| JP7312594B2 (en) | Calibration charts and calibration equipment | |
| CN117615257B (en) | Imaging method, device, medium and equipment | |
| CN114689506A (en) | Chip detection system and detection production line based on light field camera and two-dimensional camera | |
| CN110728714B (en) | Image processing method and device, storage medium and electronic equipment | |
| Liu | Camera System Characterization with Uniform Illuminate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: Room 102, 1st Floor, Building 98, No. 1441 Humin Road, Minhang District, Shanghai, 2019; Room 302, 3rd Floor, Building 98; Room 402, 4th Floor, Building 98 Patentee after: Yimu (Shanghai) Technology Co.,Ltd. Address before: 201100 room 1206, building 1, No. 951, Jianchuan Road, Minhang District, Shanghai Patentee before: Yimu (Shanghai) Technology Co.,Ltd. |