CN112731443A - Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image - Google Patents
Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image Download PDFInfo
- Publication number
- CN112731443A CN112731443A CN202110170666.1A CN202110170666A CN112731443A CN 112731443 A CN112731443 A CN 112731443A CN 202110170666 A CN202110170666 A CN 202110170666A CN 112731443 A CN112731443 A CN 112731443A
- Authority
- CN
- China
- Prior art keywords
- laser
- single photon
- photon detector
- infrared image
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/481—Constructional features, e.g. arrangements of optical elements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The invention belongs to the technical field of imaging laser radars, and particularly relates to a three-dimensional imaging system and method for fusing a single photon laser radar and a short wave infrared image. The system comprises a laser light source, a single photon detector, a light path system, a signal control unit and a short wave infrared camera; the laser light source comprises a laser controller, and the laser controller is connected with the single photon detector; the optical path system is a receiving and transmitting coaxial optical path and comprises a transmitting optical path and a receiving optical path; the signal control unit is connected with the single photon detector and the laser light source and used for sending out control signals. The system adopts a multi-echo ranging mode through the InGaAs-SPAD type single photon detector, so that the problem of distance loss of a distant object caused by object shielding can be solved; the weak light detection sensitivity of the imaging laser radar is improved. The two data of the single photon laser radar and the short wave infrared image are fused, and a three-dimensional image with high transverse resolution and texture can be obtained under the condition that the transverse resolution of the laser radar is low.
Description
Technical Field
The invention belongs to the technical field of imaging laser radars, and particularly relates to a three-dimensional imaging system and method for fusing a single photon laser radar and a short wave infrared image.
Background
The imaging laser radar technology is the most reliable distance measurement imaging technology at present, and is widely applied to the fields of robot navigation, automatic driving, topographic mapping, marine organism detection and the like. The imaging laser radar directly or indirectly measures the flight time of laser in an active laser illumination mode, and finally obtains the distance of an echo target through calculation and restores the distance information of a scene. The imaging laser radar has three imaging modes of single-point type, linear array type and planar array type. The single-point mode adopts a point-by-point distance measurement mode, and the distance measurement direction is controlled through the two-dimensional galvanometer, so that complete distance information is spliced. Compared with a single-point type, the line array type has the advantages that the number of the optical detectors is increased, and the distance measurement can be carried out on multiple directions simultaneously, so that the imaging time is shortened; the number of area array type optical detectors is further increased to form a detector area array, and distance measurement imaging can be carried out on the whole scene by emitting laser once. The lateral resolution refers to a resolution in a direction perpendicular to the detection direction, and includes a horizontal resolution and a vertical resolution. The imaging laser radar has great advantages in ranging, can measure the accurate distance of an object, but is slightly insufficient in imaging, and has the defects of low transverse resolution, missing texture and the like. The single-point imaging laser radar is flexible, and the sampling point number is increased by reducing the step value of the galvanometer, so that the effect of improving the resolution ratio is achieved. However, the time of single-point imaging is positively correlated with the number of sampling points, so that the imaging time is greatly increased while the high resolution is achieved, the real-time performance is poor, and the method is not suitable for application of dynamic scenes. The line array type imaging laser radar can realize relatively good real-time performance, but the vertical resolution is only related to the number of detection elements and cannot be adjusted by controlling the galvanometer. The area array type imaging laser radar has the best real-time performance, but the transverse resolution is limited by the number of detection elements, and under the existing technical conditions, the detector is difficult to realize a large area array or even an ultra-large area array, so that the transverse resolution is more difficult to improve, and meanwhile, the area array type imaging laser radar has higher requirements on the power of a light source and is difficult to realize remote detection under the condition that the power of the light source is limited.
The optical image has the characteristics of high resolution, abundant texture details and the like, has great advantages in the field of target identification, but can only obtain the reflection characteristic of a target and cannot obtain the distance information of the target. One common approach is to fuse the lidar with image data to improve the lidar's lateral resolution. In the prior art, an imaging laser radar based on a silicon-based avalanche photodiode is generally adopted to be fused with a visible light image, and in the technology, the imaging laser radar starts to time each sampling direction after emitting laser and stops after receiving a first target echo. According to the method, the range image of a target scene can be obtained and then is fused with the image, so that the transverse resolution of the imaging laser radar is improved. In a close scene, the scheme has a good effect because the target is single. However, in a long-distance scene, objects often have a serious occlusion problem, for example, a nearby tree can occlude a distant building, in this case, the lateral resolution of the imaging lidar is low, the distance information of the distant building can be lost, and the problem is particularly serious at the edge of an object with discontinuous depth, which can cause the serious loss of the distance information of the distant object.
Disclosure of Invention
The invention aims to solve the problems of the existing imaging laser radar in the ranging imaging process, and provides a three-dimensional imaging system and a three-dimensional imaging method for fusing a single photon laser radar and a short wave infrared image.
In order to achieve the purpose, the invention adopts the technical scheme that: a three-dimensional imaging system integrating a single photon laser radar and a short wave infrared image comprises a laser light source, a single photon detector, a light path system, a signal control unit and a short wave infrared camera; the laser light source comprises a laser controller, and the laser controller is connected with the single photon detector; the optical path system is a receiving and transmitting coaxial optical path and comprises a transmitting optical path and a receiving optical path; the signal control unit is connected with the single photon detector and the laser light source and used for sending out control signals.
As a preferred mode of the present invention, the emission light path includes a half-wave plate, a laser shaping light path, a beam expanding and collimating light path, a polarization beam splitter, and a scanning galvanometer, which are sequentially arranged.
Further preferably, the laser shaping optical path sequentially comprises a first lens and a line mirror from front to back.
Further preferably, the receiving optical path includes a scanning galvanometer, a polarization beam splitter and a fourth lens, which are sequentially arranged along the optical path, and the return beam of the scanning galvanometer passes through the polarization beam splitter and is imaged on the detection head of the single photon detector through the fourth lens.
As a preferable mode of the invention, the single-photon detector is an InGaAs-SPAD type single-photon detector.
The invention also provides a three-dimensional imaging method for fusing the single photon laser radar and the short wave infrared image, which comprises the following steps:
the single-photon detector acquires optical flight time data and transmits the optical flight time data to the computer, and meanwhile, the short-wave infrared camera transmits short-wave infrared image data of a corresponding scene to the computer;
fusing the two obtained data, wherein the fusing steps sequentially comprise: the method comprises the steps of flight time histogram statistics, point cloud extraction, point cloud denoising, data calibration, point cloud edge-preserving interpolation and image gray mapping to be point cloud colors.
Further preferably, the flow of acquiring the optical time-of-flight data by the single-photon detector is as follows:
the signal control unit controls the scanning galvanometer to do transverse reciprocating scanning movement;
the laser light source emits laser pulses which are shaped by the emission light path and then spread to a far position by a divergent longitudinal straight line; when laser pulse is emitted every time, the laser controller emits a laser synchronous signal to the single-photon detector, and the single-photon detector enters a detection state;
the transmitted longitudinal linear light beam meets a target object in the transmission process to generate laser echo, and echo photons are received by a detection element of a corresponding single photon detector after being received by a receiving light path;
the single photon detector obtains the time difference between laser receiving and laser emitting through an internal circuit, and the time difference is used as the flight time of light.
Further preferably, the point cloud extraction process sequentially comprises logarithm processing, threshold calculation and clustering combination; the clustering combination method is based on Gaussian distribution of target echo signals, each distance is used as a clustering reference by calculating a mean value and a standard deviation, and whether the current distance belongs to the previous data class or a new data class is to be created is determined according to the 3Sigma principle of the Gaussian distribution.
Further preferably, the point cloud denoising process comprises correlation denoising; the relevance denoising method is characterized in that relevance is provided based on adjacent detection directions, the relevance is determined through the target echo distance difference of the adjacent detection directions, when distance data exists in the adjacent directions of certain distance data, the distance difference between the distance data and the target echo distance difference is smaller than a preset threshold value, the certain distance data is considered to be valid, otherwise, the certain distance data is considered to be noise, and the data is rejected.
Further preferably, the point cloud edge-preserving interpolation is used for simultaneously judging whether the corresponding position of each point in the point cloud and the corresponding position of each point in the short-wave infrared image are positioned at the edge of the object; and linearly interpolating the distance between the non-edge point and the adjacent point of the adjacent detection direction.
Compared with the prior art, the invention has the following beneficial effects: the InGaAs-SPAD type single photon detector is used as a laser echo detection device, a ranging mode of multiple echoes is adopted, depth information of a partially shielded target is reserved, more complete distance information can be obtained, the problem of distance loss of a distant object caused by shielding of the object is eliminated, meanwhile, weak light detection sensitivity of the imaging laser radar is improved, and compared with a general laser radar, detection distance is remarkably improved. The imaging laser radar and the short wave infrared image are fused, so that the vertical resolution of the scanning imaging laser radar is improved, and the distance information of a target and the high-resolution short wave infrared reflection characteristic can be obtained simultaneously.
Drawings
FIG. 1 is a schematic structural connection diagram of a three-dimensional imaging system for fusing a single photon laser radar and a short wave infrared image provided in an embodiment of the present invention;
fig. 2 is a flowchart of a three-dimensional imaging method for fusing a single photon laser radar and a short wave infrared image according to an embodiment of the present invention.
Detailed Description
In order to facilitate an understanding of the invention, the invention is described in more detail below with reference to the accompanying drawings and specific examples. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
The three-dimensional imaging system for fusing the single photon laser radar and the short wave infrared image, which is provided by the embodiment, has a structure shown in fig. 1, and mainly comprises a laser light source, an InGaAs-SPAD type single photon detector, a light path system, a signal control unit, a short wave infrared camera and a computer.
The laser light source selects a STA-01-5 type YAG laser of the standard, and comprises a laser controller 1 and a laser head 2. The laser controller 1 is provided with an external trigger signal input interface and a laser synchronous signal output interface, and the laser synchronous signal output interface is connected with a laser synchronous signal input interface of the InGaAs-SPAD single photon detector. The light output end of the laser controller 1 is connected with the laser head 2 through an optical fiber. The laser outputs laser wavelength 1064nm, repetition frequency 10kHz, pulse width 0.7 ns.
The optical path system is divided into a transmitting optical path and a receiving optical path, the transmitting optical path is used for shaping a light source to generate linear laser required by active laser illumination, the receiving optical path is used for receiving echo photons, and the transmitting optical path and the receiving optical path are combined together through a polarization beam splitter to form a transmitting-receiving coaxial system. The emitted light path consists of a half-wave plate 3, a laser shaping light path, a beam expanding and collimating light path, a polarization beam splitter 10 and a scanning galvanometer 9. The laser shaping light path is the first lens 5 from front to back in proper order: a positive lens with a focal length of 75mm, a straight-line mirror 4: the main device for generating linear laser adjusts the Gaussian circular spot into uniform linear shape by adjusting the longitudinal divergence angle of the laser spot. The beam expanding collimation light path sequentially comprises the following steps from front to back: second lens 7: a negative lens with a focal length of 50mm, a third lens 8: a positive lens with a focal length of 300 mm. The beam expanding collimation light path is matched with the laser shaping light path for use, and is used for adjusting the transverse divergence angle and the energy uniformity of the laser ray. After the laser is shaped, the polarization direction of the laser is changed through the polarization beam splitter 10, so that the optical power output by the polarization beam splitter can be adjusted. Finally, after being reflected by the scanning galvanometer 9, a longitudinal uniform laser line is formed in a far field.
The receiving light path is composed of a scanning galvanometer 9, a polarization beam splitter 10, a fourth lens 13 and a narrow-band filter 14. The fourth lens 13 is a positive lens with a focal length of 50 mm. A narrow-band filter 14 with the center wavelength of 1064nm is arranged in front of a detection head of the single-photon detector and used for removing noise optical signals. The short wave infrared camera 11 is placed in an off-axis mode, and the view field consistency of the laser radar and the short wave infrared camera is kept through scene calibration. The received light is reflected by the scanning galvanometer 9, then is separated from the emitted light by the polarization beam splitter 10, and is imaged on a probe head 15 of the single photon detector through a fourth lens 13.
In order to prevent the transmission light path and the receiving light path from interfering with each other, a Polarization Beam Splitter (PBS) 10 is added to the light path system as a relay, and a part of light is transmitted by the PBS 10 and the other part of light is reflected by the PBS 10. The proportion of the reflected light and the transmitted light can be adjusted by rotating the half-wave plate 3 at the entrance of the emission light path, and the reflected light is output as a light source, so that the proportion of the reflected light is maximized, and the maximum light energy utilization rate is obtained. A light barrier is added at the transmission end of the polarization beam splitter 10, and the surface material of the light barrier is a black plastic film with a low reflection coefficient, so as to prevent stray light of the transmission part from influencing the main light path.
The receiving light path takes the scanning galvanometer 9 as an inlet, the longitudinal linear light beam generates diffuse reflection when meeting an obstacle target, and the return light beam is reflected by the scanning galvanometer 9 to enter the receiving light path. The emitted laser light is depolarized at the target surface due to the occurrence of diffuse reflection, so the returning light enters the polarization beam splitter 10 in a natural light state, and the light transmitted through the polarization beam splitter 10 is imaged on a detection head 15 of the InGaAs-SPAD single photon detector through a fourth lens 13.
The InGaAs-SPAD type single photon detector consists of a detector power supply 16, a detector host 17 and a 19X 1 detector head 15. The front end of the detector head 15 is coupled to the microlens array so that as many photons as possible can be received. 19 ports of the detection head 15 are respectively connected with 19 detection units of the detector host 17 through optical fibers, and finally returned photons reach a photosensitive surface of the InGaAs-SPAD for photoelectric conversion to form effective detection. The single photon detector works in a free running mode, and enters an avalanche state after receiving photons to form an electrical signal capable of being detected; meanwhile, the quenching circuit starts to work, so that the InGaAs-SPAD is rapidly restored to a usable state, and the next photon incident event is detected, namely the multi-echo recording function is realized.
The signal control unit consists of a computer 19, a CSC-USB control card 18 and a drive board 12 and is responsible for the work flow control of the whole three-dimensional imaging system. The computer 19 is connected with the CSC-USB control card 18 through a USB port, a scanning head control interface of the CSC-USB control card 18 is connected with the driving plate 12, and the output end of the driving plate 12 is connected with the scanning galvanometer 9. The Laser control mode of the CSC-USB control card 18 selects a YAG Laser mode, a Laser control interface Laser FREQ of the CSC-USB control card 18 is connected with an external trigger signal input interface of a Laser controller, and a Laser control interface Laser ON of the CSC-USB control card 18 is connected with a frame synchronization signal input interface of an InGaAs-SPAD single photon detector. The InGaAs-SPAD single photon detector is connected with a computer 19 through a usb3.0 interface and used for data transmission and recording.
The computer sends out scanning signal to control the rotation of the scanning galvanometer and sends out laser starting signal to the laser when each frame of scanning starts. With the rotating scanning of the scanning galvanometer, the laser emits narrow pulse light signals at a repetition frequency of 10 kHz. The laser emits a synchronous electric signal to the single-photon detector when emitting a light signal every time, and the single-photon detector records the time as the laser emitting time. After a short time delay, the single photon detector enters a detection state and starts to receive the echo photons. The emitted optical signal propagates in a longitudinal line to a distance and is reflected by an obstacle during flight to generate an echo signal. After each laser pulse is sent out, each detection element respectively records the laser return time with the same number as that of the obstacles on the detection point, the subtraction of the laser return time and the laser sending time is the round-trip flight time of the light between the imaging laser radar device and the obstacles, and finally the distance information of all the obstacles can be obtained through calculation.
The method for three-dimensional imaging by adopting the three-dimensional imaging system fusing the single photon laser radar and the short wave infrared image has the flow as shown in figure 2, and comprises the following specific steps and principles:
the start command of the system is sent by the computer 19, and the CSC-USB control card 18 and the InGaAs-SPAD single photon detector start to work. The CSC-USB control card 18 sends scanning signals to the driving plate 12, and the driving plate 12 controls the scanning galvanometer 9 to do transverse reciprocating scanning motion. The scanning galvanometer 9 is specified to scan once from left to right or from right to left for one frame. When the scanning galvanometer 9 reaches the leftmost end or the rightmost end of the scanning range, the Laser ON output port of the CSC-USB control card 18 generates a rising edge, and the rising edge is captured by the single-photon detector to be used as a frame starting signal. Then, the laser emits Gaussian circular spot laser pulses with the pulse width of 0.7ns and the wavelength of 1064nm at the repetition frequency of 10kHz, and the laser is shaped by an emission light path and then propagates to a far position by a divergent longitudinal straight line. When laser pulse is emitted every time, the laser controller emits a laser synchronous signal to the single-photon detector, the single-photon detector records the moment as a laser starting moment, and after a short fixed delay, all 19 detection elements enter a detection state. If the transmitted longitudinal linear light beam encounters an obstacle in the propagation process, diffuse reflection occurs on the surface of the obstacle and echo photons are generated, the echo photons are received by the corresponding detection elements after being received by the receiving light path, the time when the echo photons are received is recorded as the laser return time, the flight time of the laser in the process can be obtained after being processed by a time digital converter in the single photon detector, finally, the flight time and the azimuth data are transmitted to a notebook computer in real time through a usb3.0 data interface, and corresponding distance information can be obtained through calculation. When the target is partially shielded, the laser which is not shielded continues to propagate forwards until the next obstacle reflection is met, and the single-photon detector can receive multiple echoes, so that even if a part of a rear object is shielded by a front object, the corresponding distance information can be detected.
After each laser pulse is emitted, the single photon detector obtains the distance information of the scene corresponding to the whole linear laser with the resolution of 19 multiplied by 1. With the rotation of the scanning galvanometer, each linear laser pulse will appear at a distance, and if each frame has M laser lines, because the detector is a 19 × 1 linear array, 19 detection points will be established to complete distance detection on a scene on a longitudinal line after each laser signal is sent out. With the transverse rotation scanning of the galvanometer, the distance detection of 19 multiplied by M detection points can be completed on the target scene. Meanwhile, a short-wave infrared image of the corresponding scene is acquired through the short-wave infrared camera.
The detection data obtained by the single photon detector is calculated to obtain target distance information, and the specific process is as follows:
the number of echo photons at each moment is counted, the time resolution is set to 1152 picoseconds (ps), and finally, a distribution histogram of the echo photons along with time in each detection direction can be obtained. To make the data easy to process, the number of photons of the histogram is logarithmized. To further improve the signal-to-noise ratio, a sliding summation is performed over a window of 5 time resolution cells in width. The distance of the non-zero counts is then averaged as a threshold reference, and when the number of photons exceeds 1.5 times the threshold, the data is considered valid, representing that the photon at that time was returned by the target. The method comprises the steps that time jitter exists in the whole system under the influence of noise, the time jitter can enable the flight time of photons to be widened, in order to eliminate the influence, extracted effective time data are clustered, data with close distances are clustered into the same class, the average value is used as the center of the class, the standard deviation of 3 times is used as the data radius of the class, whether the next distance data belong to the current class or a new class is needed to be created according to the 3Sigma principle of Gaussian distribution, and therefore the influence caused by the time jitter is eliminated. At this time, the target distance in each detection direction is extracted.
The process of fusing the obtained target distance information and the short wave infrared image comprises the following steps:
in order to obtain better data for fusion, the obtained point cloud data needs to be denoised. Firstly, by calculating the correlation between a certain sampling direction and a target distance in the adjacent sampling direction, wherein the correlation is reflected by the absolute difference of the distances, when an absolute difference is smaller than a preset threshold value, the point is considered to be a non-noise point, otherwise, the point is a noise point and should be removed from the data.
According to the prior condition that scene depth discontinuity often exists at the edge of an object, edge-preserving interpolation can be carried out on the three-dimensional target distance information. And (4) carrying out nearest neighbor scaling on the short wave infrared image data to make the short wave infrared image data consistent with the transverse resolution of the laser point cloud. And at the moment, the target distance in each point cloud corresponds to a pixel position in the short-wave infrared image, the target distance in each detection direction in the point cloud is traversed, if the adjacent detection direction has a distance close to the target distance, linear interpolation is carried out between the two distances, otherwise, the pixel gray levels in the short-wave infrared images corresponding to the two distances are continuously compared, if the gray levels are close to each other, the current distance is taken as the interpolation distance, nearest neighbor interpolation is carried out towards the adjacent detection direction of the current point, and interpolation operation is carried out in the horizontal direction and the vertical direction in sequence, so that the transverse resolution of the laser point cloud is improved. And finally, giving the gray value of the short wave infrared image to the three-dimensional point cloud according to the position mapping relation so as to increase textures to realize three-dimensional imaging, thereby obtaining high-resolution target reflectivity information and distance information.
Claims (10)
1. A three-dimensional imaging system of single photon laser radar and short wave infrared image fusion is characterized in that: the system comprises a laser light source, a single photon detector, a light path system, a signal control unit and a short wave infrared camera; the laser light source comprises a laser controller, and the laser controller is connected with the single photon detector; the optical path system is a receiving and transmitting coaxial optical path and comprises a transmitting optical path and a receiving optical path; the signal control unit is connected with the single photon detector and the laser light source and used for sending out control signals.
2. The three-dimensional imaging system for fusion of the single photon lidar and the shortwave infrared image of claim 1, which is characterized in that: the emission light path comprises a half-wave plate, a laser shaping light path, a beam expanding collimation light path, a polarization beam splitter and a scanning galvanometer which are arranged in sequence.
3. The three-dimensional imaging system for fusion of the single photon lidar and the shortwave infrared image of claim 2, which is characterized in that: the laser shaping light path comprises a first lens and a linear lens from front to back.
4. The three-dimensional imaging system for fusion of the single photon lidar and the shortwave infrared image of claim 1, which is characterized in that: the receiving light path comprises a scanning galvanometer, a polarization beam splitter and a fourth lens which are sequentially arranged along the light path, and a return light beam of the scanning galvanometer penetrates through the polarization beam splitter and is imaged on a detection head of the single photon detector through the fourth lens.
5. The three-dimensional imaging system for fusion of single photon lidar and short wave infrared images of any one of claims 1 to 4, wherein: the single-photon detector is a multi-component InGaAs-SPAD type single-photon detector.
6. A three-dimensional imaging method for fusing a single photon laser radar and a short wave infrared image is characterized by comprising the following steps:
the single-photon detector acquires optical flight time data and transmits the optical flight time data to the computer, and meanwhile, the short-wave infrared camera transmits short-wave infrared image data of a corresponding scene to the computer;
fusing the two obtained data, wherein the fusing steps sequentially comprise: the method comprises the steps of flight time histogram statistics, point cloud extraction, point cloud denoising, data calibration, point cloud edge-preserving interpolation and image gray mapping to be point cloud colors.
7. The three-dimensional imaging method for fusing the single photon laser radar and the shortwave infrared image as claimed in claim 6, which is characterized in that: the flow of the single-photon detector for acquiring the optical flight time data is as follows:
the signal control unit controls the scanning galvanometer to do transverse reciprocating scanning movement;
the laser light source emits laser pulses which are shaped by the emission light path and then spread to a far position by a divergent longitudinal straight line; when laser pulse is emitted every time, the laser controller emits a laser synchronous signal to the single-photon detector, and the single-photon detector enters a detection state;
the transmitted longitudinal linear light beam meets a target object in the transmission process to generate laser echo, and echo photons are received by a detection element of a corresponding single photon detector after being received by a receiving light path;
the single photon detector obtains the time difference between laser receiving and laser emitting through an internal circuit, and the time difference is used as the flight time of light.
8. The three-dimensional imaging method for fusing the single photon laser radar and the shortwave infrared image as claimed in claim 6, which is characterized in that: the point cloud extraction process sequentially comprises logarithm processing, threshold calculation and clustering combination; the clustering combination method is based on Gaussian distribution of target echo signals, each distance is used as a clustering reference by calculating a mean value and a standard deviation, and whether the current distance belongs to the previous data class or a new data class is to be created is determined according to the 3Sigma principle of the Gaussian distribution.
9. The three-dimensional imaging method for fusing the single photon laser radar and the shortwave infrared image as claimed in claim 6, which is characterized in that: the point cloud denoising process comprises correlation denoising; the relevance denoising method is characterized in that relevance is provided based on adjacent detection directions, the relevance is determined through the target echo distance difference of the adjacent detection directions, when distance data exists in the adjacent directions of certain distance data, the distance difference between the distance data and the target echo distance difference is smaller than a preset threshold value, the certain distance data is considered to be valid, otherwise, the certain distance data is considered to be noise, and the data is rejected.
10. The three-dimensional imaging method for fusing the single photon laser radar and the shortwave infrared image as claimed in claim 6, which is characterized in that: the point cloud edge-preserving interpolation is used for simultaneously judging whether each point in the point cloud and the corresponding position of each point in the short-wave infrared image are positioned at the edge of an object; and linearly interpolating the distance between the non-edge point and the adjacent point of the adjacent detection direction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110170666.1A CN112731443B (en) | 2021-02-08 | 2021-02-08 | Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110170666.1A CN112731443B (en) | 2021-02-08 | 2021-02-08 | Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112731443A true CN112731443A (en) | 2021-04-30 |
| CN112731443B CN112731443B (en) | 2022-11-25 |
Family
ID=75596507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110170666.1A Active CN112731443B (en) | 2021-02-08 | 2021-02-08 | Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112731443B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113238202A (en) * | 2021-06-08 | 2021-08-10 | 上海海洋大学 | Coordinate system point cloud computing method of photon laser three-dimensional imaging system and application thereof |
| CN113296075A (en) * | 2021-07-28 | 2021-08-24 | 中国科学院西安光学精密机械研究所 | Method and system for automatically extracting target information of single photon imaging in strong noise environment |
| CN113687383A (en) * | 2021-08-23 | 2021-11-23 | 阿波罗智联(北京)科技有限公司 | Detection device containing radar and camera, roadside sensing equipment and intelligent traffic system |
| CN114545643A (en) * | 2022-04-22 | 2022-05-27 | 北京航空航天大学 | Single photon imaging system based on polarization noise reduction |
| CN115616608A (en) * | 2022-12-19 | 2023-01-17 | 中国科学院西安光学精密机械研究所 | Single-photon three-dimensional imaging distance super-resolution method and system |
| CN116165682A (en) * | 2023-03-14 | 2023-05-26 | 兰州大学 | Portable atmosphere monitoring laser radar system |
| CN117111093A (en) * | 2023-10-20 | 2023-11-24 | 中山大学 | Single-pixel three-dimensional imaging method and system based on neural network |
| CN118068348A (en) * | 2024-04-25 | 2024-05-24 | 中国科学院长春光学精密机械与物理研究所 | Single-station photoelectric positioning measurement device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105607073A (en) * | 2015-12-18 | 2016-05-25 | 哈尔滨工业大学 | Photon-counting imaging laser radar for filtering noise in real time by adopting adjacent pixel element threshold value method |
| US20160373676A1 (en) * | 2015-06-22 | 2016-12-22 | OmniVision Technolgies, Inc. | Imaging Systems With Single-Photon-Avalanche-Diodes And Sensor Translation, And Associated Methods |
| CN110187357A (en) * | 2019-06-14 | 2019-08-30 | 中国科学技术大学 | The Laser Active Imaging System Used of 3-D image reconstruct |
| CN110579775A (en) * | 2019-10-10 | 2019-12-17 | 华中光电技术研究所（中国船舶重工集团有限公司第七一七研究所） | Ultra-long-range single-photon three-dimensional laser radar scanning imaging system |
| CN210005696U (en) * | 2018-12-25 | 2020-01-31 | 西南技术物理研究所 | Dual-wavelength adaptive distance laser radar |
| CN111562593A (en) * | 2020-04-23 | 2020-08-21 | 中国科学院西安光学精密机械研究所 | Area array push-broom type photon counting three-dimensional imaging system based on pixel multiplexing |
| CN111736168A (en) * | 2020-06-22 | 2020-10-02 | 四川科鱼光电科技有限公司 | Three-dimensional imaging laser radar and method based on unit single photon detector |
| EP3726247A1 (en) * | 2019-04-18 | 2020-10-21 | Veoneer Sweden AB | Lidar imaging apparatus for a motor vehicle |
-
2021
- 2021-02-08 CN CN202110170666.1A patent/CN112731443B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160373676A1 (en) * | 2015-06-22 | 2016-12-22 | OmniVision Technolgies, Inc. | Imaging Systems With Single-Photon-Avalanche-Diodes And Sensor Translation, And Associated Methods |
| CN105607073A (en) * | 2015-12-18 | 2016-05-25 | 哈尔滨工业大学 | Photon-counting imaging laser radar for filtering noise in real time by adopting adjacent pixel element threshold value method |
| CN210005696U (en) * | 2018-12-25 | 2020-01-31 | 西南技术物理研究所 | Dual-wavelength adaptive distance laser radar |
| EP3726247A1 (en) * | 2019-04-18 | 2020-10-21 | Veoneer Sweden AB | Lidar imaging apparatus for a motor vehicle |
| CN110187357A (en) * | 2019-06-14 | 2019-08-30 | 中国科学技术大学 | The Laser Active Imaging System Used of 3-D image reconstruct |
| CN110579775A (en) * | 2019-10-10 | 2019-12-17 | 华中光电技术研究所（中国船舶重工集团有限公司第七一七研究所） | Ultra-long-range single-photon three-dimensional laser radar scanning imaging system |
| CN111562593A (en) * | 2020-04-23 | 2020-08-21 | 中国科学院西安光学精密机械研究所 | Area array push-broom type photon counting three-dimensional imaging system based on pixel multiplexing |
| CN111736168A (en) * | 2020-06-22 | 2020-10-02 | 四川科鱼光电科技有限公司 | Three-dimensional imaging laser radar and method based on unit single photon detector |
Non-Patent Citations (3)
| Title |
|---|
| SHUMPEI KAMEYAMA ET.AL: "《Development of Long Range,Real-time,and High Resolution 3-D Imaging Ladar》", 《PROC.SPIE》 * |
| 张秀丽 等: "《红外主被动图像像素级融合的仿真研究》", 《红外与激光工程》 * |
| 王海伟: "《高速飞行器红外/激光复合凝视成像设计与约束条件分析》", 《制导与引信》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113238202A (en) * | 2021-06-08 | 2021-08-10 | 上海海洋大学 | Coordinate system point cloud computing method of photon laser three-dimensional imaging system and application thereof |
| CN113238202B (en) * | 2021-06-08 | 2023-08-15 | 上海海洋大学 | Coordinate system point cloud computing method of photon laser three-dimensional imaging system and application thereof |
| CN113296075A (en) * | 2021-07-28 | 2021-08-24 | 中国科学院西安光学精密机械研究所 | Method and system for automatically extracting target information of single photon imaging in strong noise environment |
| CN113687383A (en) * | 2021-08-23 | 2021-11-23 | 阿波罗智联(北京)科技有限公司 | Detection device containing radar and camera, roadside sensing equipment and intelligent traffic system |
| CN114545643A (en) * | 2022-04-22 | 2022-05-27 | 北京航空航天大学 | Single photon imaging system based on polarization noise reduction |
| CN115616608A (en) * | 2022-12-19 | 2023-01-17 | 中国科学院西安光学精密机械研究所 | Single-photon three-dimensional imaging distance super-resolution method and system |
| CN115616608B (en) * | 2022-12-19 | 2023-08-11 | 中国科学院西安光学精密机械研究所 | Single photon three-dimensional imaging distance super-resolution method and system |
| CN116165682A (en) * | 2023-03-14 | 2023-05-26 | 兰州大学 | Portable atmosphere monitoring laser radar system |
| CN116165682B (en) * | 2023-03-14 | 2023-09-26 | 兰州大学 | Portable atmosphere monitoring laser radar system |
| CN117111093A (en) * | 2023-10-20 | 2023-11-24 | 中山大学 | Single-pixel three-dimensional imaging method and system based on neural network |
| CN117111093B (en) * | 2023-10-20 | 2024-02-06 | 中山大学 | Single-pixel three-dimensional imaging method and system based on neural network |
| CN118068348A (en) * | 2024-04-25 | 2024-05-24 | 中国科学院长春光学精密机械与物理研究所 | Single-station photoelectric positioning measurement device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112731443B (en) | 2022-11-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112731443B (en) | Three-dimensional imaging system and method for fusing single photon laser radar and short wave infrared image | |
| US12050288B2 (en) | High resolution LiDAR using high frequency pulse firing | |
| CN104483676B (en) | A kind of 3D/2D scannerless laser radars complex imaging device | |
| CN106405572B (en) | Remote high-resolution laser Active Imaging device and method based on space encoding | |
| CN109490908B (en) | Line scanning laser radar and scanning method | |
| CN108303701A (en) | Laser radar system, the launching technique of laser pulse and medium | |
| CN110471083B (en) | Longitudinal distance laser three-dimensional imaging device and method | |
| US11581697B2 (en) | Detector system comparing pixel response with photonic energy decay | |
| CN116299550B (en) | Underwater coaxial single-photon imaging system and method | |
| CN109814128A (en) | The high-resolution fast imaging system and method that time flight is combined with relevance imaging | |
| CN104486550A (en) | Image focus detecting device and method for aerial camera | |
| CN114200425A (en) | Three-dimensional imaging system and method for hidden target | |
| CN108107441B (en) | Integrated device and method capable of realizing distance measurement and ghost imaging | |
| CN115079203B (en) | Non-vision imaging system and imaging method | |
| CN117629403A (en) | Active single photon detection array non-field imaging system | |
| WO2022195954A1 (en) | Sensing system | |
| US11802945B2 (en) | Photonic ROIC having safety features | |
| CN111257908B (en) | High-dimensional detection method based on optical synchronization | |
| CN116400379B (en) | Three-dimensional imaging system and three-dimensional imaging method of single-photon laser radar | |
| KR20220107060A (en) | Lidar with photon resolution detector | |
| CN117111093B (en) | Single-pixel three-dimensional imaging method and system based on neural network | |
| CN114325730B (en) | Large-field-of-view remote laser radar testing method | |
| CN215867094U (en) | Resolution configurable lidar | |
| CN111551911A (en) | Laser radar echo signal processing method based on spatial domain weighting | |
| CN114935766A (en) | High-speed single-photon laser radar dynamic imaging device and scanning method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |