CN112710276A - Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS - Google Patents

Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS Download PDF

Info

Publication number
CN112710276A
CN112710276A CN202011481812.4A CN202011481812A CN112710276A CN 112710276 A CN112710276 A CN 112710276A CN 202011481812 A CN202011481812 A CN 202011481812A CN 112710276 A CN112710276 A CN 112710276A
Authority
CN
China
Prior art keywords
frequency domain
ccd
cmos
binocular
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011481812.4A
Other languages
Chinese (zh)
Inventor
汪凯巍
袁利
王立
武延鹏
郑然�
王苗苗
朱临轩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202011481812.4A priority Critical patent/CN112710276A/en
Publication of CN112710276A publication Critical patent/CN112710276A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00Measuring distances in line of sight; Optical rangefinders
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/02Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

本发明涉及一种基于像素频域标定矫正CCD/CMOS的双目测距方法及系统,属于光学成像、图像处理、以及信号处理技术领域。系统包含杨氏动态干涉模块以及由一对参数相似的相机组成的双目测距系统。其中杨氏动态干涉模块主要用于标定双目测距系统中所使用的CCD/CMOS传感器的频域响应函数的参数,并将参数保存在计算机中,每次从相机获取到图像后,使用上述标定频域响应函数获取入射光场的频域分布,作傅立叶逆变换后获取入射光场的空间分布,再对之按理想CCD/CMOS传感器像元分布及所需分辨率重新采样获得更精确的图像后,输入到双目测距系统的匹配算法中,获取更精准,分辨率更高的测量结果。

Figure 202011481812

The invention relates to a binocular ranging method and system based on pixel frequency domain calibration and correction CCD/CMOS, belonging to the technical fields of optical imaging, image processing and signal processing. The system includes Young's dynamic interference module and a binocular ranging system composed of a pair of cameras with similar parameters. Among them, the Young's dynamic interference module is mainly used to calibrate the parameters of the frequency domain response function of the CCD/CMOS sensor used in the binocular ranging system, and save the parameters in the computer. The frequency domain distribution of the incident light field is obtained by calibrating the frequency domain response function, and the spatial distribution of the incident light field is obtained after inverse Fourier transform, and then resampled according to the ideal CCD/CMOS sensor pixel distribution and required resolution to obtain a more accurate After the image is input, it is input into the matching algorithm of the binocular ranging system to obtain more accurate and higher resolution measurement results.

Figure 202011481812

Description

Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS
Technical Field
The invention relates to the technical field of optical imaging, image processing and signal processing, in particular to a binocular distance measuring method and system based on pixel frequency domain calibration correction CCD/CMOS.
Background
At present, CCD/CMOS is widely applied in the fields of digital photography, aerospace, astronomical detection, spectrum telescope and the like, and is industrially used for measuring size, collecting images and detecting high temperature. The CCD/CMOS can directly convert the optical signal into a digital electric signal to realize the acquisition, storage, transmission, processing and reproduction of the image.
The binocular ranging is a depth information acquisition mode based on a triangular ranging principle, and compared with other depth information acquisition systems, the binocular ranging system has the advantages of simple structure, good environmental adaptability, dense depth information and the like, and is widely applied to the fields of autonomous navigation, obstacle detection and the like.
The binocular ranging system mainly comprises a pair of CCD/CMOS cameras with fixed pose relationship and an operation platform for processing images. The measurement principle is as follows: the same object point in the scene is imaged at different positions of a CCD/CMOS image surface due to the optical geometric relation, the position difference is called parallax, the distance of the object point can be calculated by calculating the parallax and combining the parameters of the pair of cameras, and the depth information can be obtained. According to the measurement principle of the binocular ranging system, the measurement accuracy thereof depends on the accuracy of parallax.
The current measurement algorithm is based on the assumption that CCD/CMOS pixels are uniformly distributed, and does not consider the fact that the actual CCD/CMOS pixels have a certain micro-deviation of a geometric position from the theoretical position, so that a certain error is introduced; in addition, the measurement resolution of the binocular ranging system is limited by the resolution of parallax, and the resolution of parallax is determined by the pixel size of the CCD/CMOS sensor, so the most direct method for improving the measurement resolution of the binocular ranging system is to replace an image sensor with a smaller pixel size.
However, in the case of the CCD/CMOS sensor, the sensitivity of the pixel is easily reduced due to the gradual reduction of the size of the pixel, and at the same time, the CCD/CMOS sensor poses a great challenge to the manufacturing process and is easy to create a technical barrier. In view of this situation, researchers expect that the method of dynamic young's interference correction and pixel frequency domain response function reconstruction CCD/CMOS can be applied to image preprocessing of the binocular ranging system to improve the measurement accuracy of the binocular ranging system.
The CCD/CMOS technology for dynamic Young's interference correction mainly aims at calculating the tiny deviation of the position of a pixel set, and is limited by a CCD/CMOS processing technology, so that the position of a pixel of a CCD/CMOS usually has a certain deviation from a preset integer coordinate position, and the position of an output image is easy to be inaccurate.
Disclosure of Invention
The invention aims to provide a binocular distance measurement method and system based on pixel frequency domain calibration correction CCD/CMOS, which utilizes a dynamic Young's interference method to calibrate the frequency domain response function of a CCD/CMOS sensor and combines a binocular distance measurement technology to ensure that the measurement precision is better and the resolution is higher.
In order to achieve the above object, in a first aspect, the present invention provides a binocular ranging method based on pixel frequency domain calibration correction CCD/CMOS, comprising the steps of:
1) the laser output by the light source is divided into two beams of light by the beam splitter, and after the two beams of light are respectively subjected to frequency modulation by the two acousto-optic modulators, the two beams of light have certain frequency difference and are respectively coupled into the optical fibers;
2) fixing an optical fiber bearing two beams of light on a multi-channel optical fiber fixing base, wherein the two beams of light interfere with each other to form a dynamic interference fringe at a far distance;
3) using a CCD/CMOS sensor to be calibrated to acquire continuous fringe images of two optical fibers at different transverse intervals and longitudinal intervals, wherein the different transverse intervals and the longitudinal intervals correspond to different wave vector directions of an incident light field;
4) calculating each parameter of the pixel frequency domain response function through fitting of a plurality of groups of collected continuous images in different wave vector directions, and storing the parameters for later use;
5) selecting optical lenses with the same parameters from a pair of CCD/CMOS sensors calibrated in the mode to form a binocular camera, performing Fourier transform on images acquired by the CCD/CMOS sensors, combining each parameter of the frequency domain response function in the step 4), solving the frequency domain distribution of an incident light field, obtaining the spatial distribution of the incident light field after inverse Fourier transform, and then performing resampling to obtain an image with ideal pixel position and higher resolution;
6) and taking the processed image as the input of a binocular ranging system to further acquire the measured distance.
In the above technical solution, the frequency domain response function of each pixel of the CCD/CMOS sensor can be calibrated by the dynamic young's interference correction technique, and then the frequency domain response function of the CCD/CMOS sensor is obtained, and according to the fourier optical theory, the output of the CCD/CMOS sensor is the convolution of the incident optical field distribution and the response function in the spatial domain, and is the product of the incident optical field distribution and the response function in the frequency domain. Therefore, after knowing the frequency domain response function of the CCD/CMOS sensor, the Fourier transform result of the incident light field can be obtained from the output of the CCD/CMOS sensor, and after inverse Fourier transform, the accurate incident light field distribution can be recovered, and the sampling is carried out again according to the mode of uniform distribution of pixels and the required resolution (pixel size), thereby improving the measurement precision and resolution of the binocular ranging system.
In step 1), a HeNe laser is used as a light source.
In the step 3), the transverse interval and the longitudinal interval of the two bundles of optical fibers can be changed by replacing the positions of the two bundles of optical fibers in the multichannel optical fiber fixing base.
And 4) in the step 4), fitting and calculating parameters of the pixel frequency domain response function by using a four-step phase method.
In step 5), the image collected by the CCD/CMOS sensor is the product of the frequency domain distribution of the incident light field and the frequency domain response function on the frequency domain.
In step 6), according to the basic formula of the binocular ranging system
Figure BDA0002837806650000041
d is depth, f is cameraThe focal length of the lens, L is the base line distance of the used camera pair, D is the parallax, after the image processed in the step 5) is used, the obtained parallax is more accurate, and meanwhile, the resolution ratio is higher, so that the measurement result is more accurate and the resolution ratio is higher.
In a second aspect, the binocular ranging system based on the pixel frequency domain calibration correction CCD/CMOS provided by the present invention is used for implementing the binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS, and includes:
the CCD/CMOS sensors are respectively used for acquiring images in front of two optical lenses of the binocular camera;
the Young dynamic interference module is used for calibrating parameters of a frequency domain response function of the CCD/CMOS sensor;
and the computer is used for fitting and calculating each parameter of the pixel frequency domain response function and storing the parameters.
The Young dynamic interference module comprises a HeNe laser, a beam splitter, a pair of light shields, a pair of polarization controllers, a pair of acousto-optic modulators and an optical fiber base; the beam splitter divides laser emitted by the HeNe laser into two beams of light which respectively enter the light chopper, the polarization controller and the acousto-optic modulator in sequence along two paths.
Compared with the prior art, the invention has the advantages that:
the invention can calibrate the frequency domain response function of the CCD/CMOS sensor, thereby recovering the accurate incident light field distribution, and resampling the incident light field distribution with the required resolution (pixel size) according to the mode of uniform distribution of the pixels, thereby improving the measurement precision and resolution of the binocular ranging system.
Drawings
FIG. 1 is a diagram of a binocular ranging system based on a pixel frequency domain calibration correction CCD/CMOS in an embodiment of the present invention;
FIG. 2 is a schematic diagram of a binocular ranging system according to an embodiment of the present invention;
FIG. 3 is a flowchart of a binocular ranging method based on pixel frequency domain calibration correction CCD/CMOS in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described with reference to the following embodiments and accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments without any inventive step, are within the scope of protection of the invention.
Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. The use of the word "comprise" or "comprises", and the like, in the context of this application, is intended to mean that the elements or items listed before that word, in addition to those listed after that word, do not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
Examples
The binocular distance measuring system based on the pixel frequency domain calibration correction CCD/CMOS comprises a pair of CCD/CMOS sensors 7, a pair of optical lenses 8 with the same parameters and a CCD/CMOS dynamic Young's interference calibration pixel deviation and frequency domain response function reconstruction module. The CCD/CMOS dynamic Young's interference calibration pixel shift and frequency domain response function reconstruction module comprises a HeNe laser 1, a beam splitter 2, a pair of light shields 3, a pair of polarization controllers 4, a pair of acousto-optic modulators (AOMs) 5 and an optical fiber base 6.
Firstly, calibrating a CCD/CMOS sensor in a binocular ranging system. As shown in figure 1, a CCD/CMOS dynamic Young interference is used for calibrating a pixel offset and frequency domain response function reconstruction module, laser generated by a HeNe laser 1 is coupled into an optical fiber, and then split into two beams of light beams 2, the two beams of light beams are respectively modulated by an acousto-optic modulator 5 to generate a certain frequency difference, and then the two beams of optical fibers are respectively fixed at different positions of a multichannel optical fiber base, so that the two beams of light interfere in space. The emergent light intensity of the two optical fibers is measured by using the light chopper 3 to shade one optical fiber light beam and measuring the emergent light intensity of the other optical fiber light beam. The shutter 3 is opened, the CCD/CMOS sensor 7 to be calibrated is used for recording an interference fringe image for a period of time, and the interference fringe image is stored. The fixed position of the fiber is changed, the interference fringes within a period of time are recaptured, and the same operation is repeated for a plurality of times. And all the captured interference fringe images, the CCD/CMOS sensor 7 and the relevant parameters of the laser used are input into a computer program to obtain the frequency domain response function of the CCD/CMOS sensor 7.
Referring to fig. 2, a pair of calibrated CCD/CMOS sensors and optical lenses with the same parameters are used to form a binocular camera, and a scene to be measured is photographed to obtain an image. The image collected by the sensor is subjected to Fourier transform, and then the frequency domain distribution of the incident light field can be obtained by combining with the calibrated CCD/CMOS frequency domain response function, after the Fourier inverse transformation, the spatial distribution of the incident light field can be obtained, then the resampling is carried out, the image with ideal pixel position and higher resolution is obtained, the image is input into a binocular matching algorithm for calculation, the depth of the scene to be measured is obtained, and the overall flow is shown in figure 3. After the CCD/CMOS sensor is calibrated, a more accurate imaging position of the object point to be measured can be obtained, and therefore a more accurate distance measurement result of the object point to be measured is obtained.

Claims (8)

1. A binocular distance measurement method based on pixel frequency domain calibration correction CCD/CMOS is characterized by comprising the following steps:
1) the laser output by the light source is divided into two beams of light by the beam splitter, and after the two beams of light are respectively subjected to frequency modulation by the two acousto-optic modulators, the two beams of light have certain frequency difference and are respectively coupled into the optical fibers;
2) fixing an optical fiber bearing two beams of light on a multi-channel optical fiber fixing base, wherein the two beams of light interfere with each other to form a dynamic interference fringe at a far distance;
3) using a CCD/CMOS sensor to be calibrated to acquire continuous fringe images of two optical fibers at different transverse intervals and longitudinal intervals, wherein the different transverse intervals and the longitudinal intervals correspond to different wave vector directions of an incident light field;
4) calculating each parameter of the pixel frequency domain response function through fitting of a plurality of groups of collected continuous images in different wave vector directions, and storing the parameters for later use;
5) selecting optical lenses with the same parameters from a pair of CCD/CMOS sensors calibrated in the mode to form a binocular camera, performing Fourier transform on images acquired by the CCD/CMOS sensors, combining each parameter of the frequency domain response function in the step 4), solving the frequency domain distribution of an incident light field, obtaining the spatial distribution of the incident light field after inverse Fourier transform, and then performing resampling to obtain an image with ideal pixel position and higher resolution;
6) and taking the processed image as the input of a binocular ranging system to further acquire the measured distance.
2. The binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 1), HeNe laser is used as light source.
3. The binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 3), the lateral spacing and the longitudinal spacing of the two optical fibers can be changed by replacing the positions of the two optical fibers in the multi-channel optical fiber fixing base.
4. The binocular distance measuring method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in the step 4), the parameters of the pixel frequency domain response function are calculated by using a four-step phase method in a fitting manner.
5. The binocular distance measuring method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 5), the image collected by the CCD/CMOS sensor is the product of the frequency domain distribution of the incident light field and the frequency domain response function in the frequency domain.
6. The binocular distance measuring method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 6), according to the basic formula of the binocular distance measuring system
Figure FDA0002837806640000021
D is the depth, f is the focal length of the camera lens, L is the baseline distance of the used camera pair, D is the parallax, after the image processed in the step 5) is used, the acquired parallax is more accurate, and meanwhile, the resolution ratio is higher, so that the measurement result is more accurate and the resolution ratio is higher.
7. A binocular ranging system based on a pixel frequency domain calibration correction CCD/CMOS is used for realizing the binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in any one of claims 1-6, and is characterized by comprising the following steps:
the CCD/CMOS sensors are respectively used for acquiring images in front of two optical lenses of the binocular camera;
the Young dynamic interference module is used for calibrating parameters of a frequency domain response function of the CCD/CMOS sensor;
and the computer is used for fitting and calculating each parameter of the pixel frequency domain response function and storing the parameters.
8. The binocular ranging system based on the pixel frequency domain calibration rectification CCD/CMOS as claimed in claim 7, wherein the Young's dynamic interference module comprises a HeNe laser, a beam splitter, a pair of shutters, a pair of polarization controllers, a pair of acousto-optic modulators and a fiber base;
the beam splitter divides laser emitted by the HeNe laser into two beams of light which respectively enter the light chopper, the polarization controller and the acousto-optic modulator in sequence along two paths.
CN202011481812.4A 2020-12-15 2020-12-15 Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS Pending CN112710276A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011481812.4A CN112710276A (en) 2020-12-15 2020-12-15 Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011481812.4A CN112710276A (en) 2020-12-15 2020-12-15 Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS

Publications (1)

Publication Number Publication Date
CN112710276A true CN112710276A (en) 2021-04-27

Family

ID=75542178

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011481812.4A Pending CN112710276A (en) 2020-12-15 2020-12-15 Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS

Country Status (1)

Country Link
CN (1) CN112710276A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20000040404A (en) * 1998-12-18 2000-07-05 이구택 Method and system for compensating measured error when measuring width using ccd camera
CN104320598A (en) * 2014-11-06 2015-01-28 中国科学院光电研究院 Aberration-free image reconstruction method based on CCD array pixel response function frequency domain calibration
CN105841925A (en) * 2016-03-22 2016-08-10 中国科学院国家空间科学中心 Detector pixel response Fourier spectrum acquisition-based image reconstruction method
CN108245788A (en) * 2017-12-27 2018-07-06 苏州雷泰医疗科技有限公司 A kind of binocular range unit and method, the accelerator radiotherapy system including the device
US20190098283A1 (en) * 2017-09-15 2019-03-28 Elphel, Inc. Method for the FPGA-Based Long Range Multi-View Stereo with Differential Image Rectification
CN111664870A (en) * 2020-06-04 2020-09-15 北京控制工程研究所 Dynamic Young laser interference fringe calibration system and detector pixel geometric position deviation calibration method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20000040404A (en) * 1998-12-18 2000-07-05 이구택 Method and system for compensating measured error when measuring width using ccd camera
CN104320598A (en) * 2014-11-06 2015-01-28 中国科学院光电研究院 Aberration-free image reconstruction method based on CCD array pixel response function frequency domain calibration
CN105841925A (en) * 2016-03-22 2016-08-10 中国科学院国家空间科学中心 Detector pixel response Fourier spectrum acquisition-based image reconstruction method
US20190098283A1 (en) * 2017-09-15 2019-03-28 Elphel, Inc. Method for the FPGA-Based Long Range Multi-View Stereo with Differential Image Rectification
CN108245788A (en) * 2017-12-27 2018-07-06 苏州雷泰医疗科技有限公司 A kind of binocular range unit and method, the accelerator radiotherapy system including the device
CN111664870A (en) * 2020-06-04 2020-09-15 北京控制工程研究所 Dynamic Young laser interference fringe calibration system and detector pixel geometric position deviation calibration method

Similar Documents

Publication Publication Date Title
CN102494609B (en) Three-dimensional photographing process based on laser probe array and device utilizing same
KR101590778B1 (en) Method and camera for the real-time acquisition of visual information from three-dimensional scenes
KR970001140B1 (en) Distance measuring method and device
CN108171758B (en) Multi-camera calibration method based on minimum optical path principle and transparent glass calibration plate
CN109307483B (en) Phase unwrapping method based on geometric constraint of structured light system
CN103727891B (en) Synchronize three-dimensional speckle interferometer measuration system and measuring method
CN113205592B (en) A three-dimensional reconstruction method and system of light field based on phase similarity
Gao et al. Stereo camera calibration for large field of view digital image correlation using zoom lens
JP5440615B2 (en) Stereo camera device
CN113643436B (en) Depth data splicing and fusion method and device
CN111971956B (en) Method and system for dynamic stereo calibration
JPH102722A (en) Three-dimensional position-measuring apparatus
CN115451820B (en) Three-channel polarization information acquisition system
CN110476043B (en) Method and optical system for acquiring a tomographic profile of a wavefront of an electromagnetic field
CN114252025B (en) Multi-parallel line laser object three-dimensional contour measuring device and measuring method
CN112710276A (en) Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS
US12205314B2 (en) Image processing apparatus for a plurality of timeseries images acquired by an endoscope, image processing method, and recording medium
de Souza et al. Combining 3D models with 2D infrared images for medical applications
US20230274444A1 (en) Measuring device, moving device, measuring method, and storage medium
Chen et al. A structured-light-based panoramic depth camera
CN112393694B (en) A Measurement Method Based on Pixel Frequency Domain Calibration to Improve the Precision of Photoelectric Autocollimator
JP4651550B2 (en) Three-dimensional coordinate measuring apparatus and method
JPH05306916A (en) Stripe phase distribution analysis method and stripe phase distribution analyzer
CN109470143B (en) External light source high-resolution stereo vision measuring system and method
Zhao et al. High throughput system for plant height and hyperspectral measurement

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210427